[Openvpn-devel] cryptoapi.c: fix run-time check failure in msvc debugger

Message ID 20200213095539.11157-1-lstipakov@gmail.com
State Accepted
Headers show
Series [Openvpn-devel] cryptoapi.c: fix run-time check failure in msvc debugger | expand

Commit Message

Lev Stipakov Feb. 12, 2020, 10:55 p.m. UTC
From: Lev Stipakov <lev@openvpn.net>

When using certificate without RSA_PKCS1_PSS_PADDING padding,
"saltlen" is passed unitialized to priv_enc_CNG(), which causes

 > Run-Time Check Failure #3 - The variable 'saltlen' is being used without being initialized.

in VS debugger.

Initialize saltlen (and other variable for the sake of consistence) to zero
to avoid above failure.

Signed-off-by: Lev Stipakov <lev@openvpn.net>
---
 src/openvpn/cryptoapi.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Comments

Selva Nair Feb. 13, 2020, 6:13 a.m. UTC | #1
Hi,

On Thu, Feb 13, 2020 at 4:57 AM Lev Stipakov <lstipakov@gmail.com> wrote:
>
> From: Lev Stipakov <lev@openvpn.net>
>
> When using certificate without RSA_PKCS1_PSS_PADDING padding,
> "saltlen" is passed unitialized to priv_enc_CNG(), which causes
>
>  > Run-Time Check Failure #3 - The variable 'saltlen' is being used without being initialized.
>
> in VS debugger.
>
> Initialize saltlen (and other variable for the sake of consistence) to zero

"consistency"

> to avoid above failure.
>
> Signed-off-by: Lev Stipakov <lev@openvpn.net>
> ---
>  src/openvpn/cryptoapi.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/src/openvpn/cryptoapi.c b/src/openvpn/cryptoapi.c
> index 1bf74fcd..30eba7b2 100644
> --- a/src/openvpn/cryptoapi.c
> +++ b/src/openvpn/cryptoapi.c
> @@ -882,9 +882,9 @@ pkey_rsa_sign(EVP_PKEY_CTX *ctx, unsigned char *sig, size_t *siglen,
>      EVP_MD *md = NULL;
>      const wchar_t *alg = NULL;
>
> -    int padding;
> -    int hashlen;
> -    int saltlen;
> +    int padding = 0;
> +    int hashlen = 0;
> +    int saltlen = 0;
>
>      pkey = EVP_PKEY_CTX_get0_pkey(ctx);
>      if (pkey)

Yeah, technically it may be "undefined behaviour" to pass an
uninitialized var to a function even when its not used there.

Acked-by: Selva Nair <selva.nair@gmail.com>




Selva
Gert Doering Feb. 13, 2020, 8:44 a.m. UTC | #2
Your patch has been applied to the master branch.

(No testing done, but this is syntactically correct and does what it
says on the tin)

commit e84f430f8487cf42304fe29fd8746a91309b08d0
Author: Lev Stipakov
Date:   Thu Feb 13 11:55:39 2020 +0200

     cryptoapi.c: fix run-time check failure in msvc debugger

     Signed-off-by: Lev Stipakov <lev@openvpn.net>
     Acked-by: Selva Nair <selva.nair@gmail.com>
     Message-Id: <20200213095539.11157-1-lstipakov@gmail.com>
     URL: https://www.mail-archive.com/openvpn-devel@lists.sourceforge.net/msg19410.html
     Signed-off-by: Gert Doering <gert@greenie.muc.de>


--
kind regards,

Gert Doering

Patch

diff --git a/src/openvpn/cryptoapi.c b/src/openvpn/cryptoapi.c
index 1bf74fcd..30eba7b2 100644
--- a/src/openvpn/cryptoapi.c
+++ b/src/openvpn/cryptoapi.c
@@ -882,9 +882,9 @@  pkey_rsa_sign(EVP_PKEY_CTX *ctx, unsigned char *sig, size_t *siglen,
     EVP_MD *md = NULL;
     const wchar_t *alg = NULL;
 
-    int padding;
-    int hashlen;
-    int saltlen;
+    int padding = 0;
+    int hashlen = 0;
+    int saltlen = 0;
 
     pkey = EVP_PKEY_CTX_get0_pkey(ctx);
     if (pkey)