[Openvpn-devel,0/9] A built-in OpenSSL3.0 provider for external-keys

Message ID 20210922211254.7570-1-selva.nair@gmail.com
Headers show
Series
  • A built-in OpenSSL3.0 provider for external-keys
Related show

Message

Selva Nair Sept. 22, 2021, 9:12 p.m.
From: Selva Nair <selva.nair@gmail.com>

The following series of patches implement a built-in
provider for interfacing OpenSSL 3.0 when external 
keys are in use.

Essentially, to intercept the sign operation, the SSL_CTX
object has to be created with properties string set to 
prioritize our provider. In the provider we implement
only keymgmt and signature operations and specify the
property string as optional. That allows all operations
we do not provide to be used from the default provider.

This patch set stops at interfacing the provider with
management-external-key. For pkcs11-helper, only some glue
code is needed and is in the works. Same with cryptoapicert
 aka CNG, but I want to cleanup the old code a bit before
hooking to the provider.

I haven't attempted to remove any of the deprecated interfaces.
That is better done along with Arne's patches. There will be
only minor, if at all any, conflicts between that and this 
patch set. 

Selva Nair (9):
  A built-in provider for using external key with OpenSSL 3.0
  Initialize the xkey provider and use it in SSL context
  Implement keymgmt in the xkey provider
  Implement provider interface for signature operations
  Implement import of custom external keys
  A helper function to load key for management-external-key
  Enable signing via provider for management-external-key
  Add a function to encode digests with PKCS1 DigestInfo wrapper
  Allow management client to announce pss padding support

 configure.ac                            |   11 +
 doc/man-sections/management-options.rst |    8 +-
 doc/management-notes.txt                |   15 +-
 src/openvpn/Makefile.am                 |    2 +
 src/openvpn/crypto_openssl.c            |   19 +
 src/openvpn/manage.h                    |    1 +
 src/openvpn/openssl_compat.h            |   12 +
 src/openvpn/options.c                   |    7 +-
 src/openvpn/ssl_openssl.c               |   17 +-
 src/openvpn/xkey_common.h               |  120 +++
 src/openvpn/xkey_helper.c               |  285 ++++++
 src/openvpn/xkey_provider.c             | 1158 +++++++++++++++++++++++
 12 files changed, 1647 insertions(+), 8 deletions(-)
 create mode 100644 src/openvpn/xkey_common.h
 create mode 100644 src/openvpn/xkey_helper.c
 create mode 100644 src/openvpn/xkey_provider.c

Comments

Arne Schwabe Sept. 23, 2021, 11:11 a.m. | #1
Am 22.09.21 um 23:12 schrieb selva.nair@gmail.com:
> From: Selva Nair <selva.nair@gmail.com>
> 
> The following series of patches implement a built-in
> provider for interfacing OpenSSL 3.0 when external 
> keys are in use.
> 
> Essentially, to intercept the sign operation, the SSL_CTX
> object has to be created with properties string set to 
> prioritize our provider. In the provider we implement
> only keymgmt and signature operations and specify the
> property string as optional. That allows all operations
> we do not provide to be used from the default provider.
> 
> This patch set stops at interfacing the provider with
> management-external-key. For pkcs11-helper, only some glue
> code is needed and is in the works. Same with cryptoapicert
>  aka CNG, but I want to cleanup the old code a bit before
> hooking to the provider.
> 
> I haven't attempted to remove any of the deprecated interfaces.
> That is better done along with Arne's patches. There will be
> only minor, if at all any, conflicts between that and this 
> patch set. 
>

Great work and also extremely quick on the implementation. This
currently puts myself in a bit of dilemma. We (=OpenVPN Inc) need a
similar/same implementation in OpenVPN3 too. I think after I do the
review I am probably too biased to finish my own implementation without
running into the problem of it being derivative.

Your code is currently GPL2 only and not BSD like or you explicitly
agreeing to have it under the CLA of openvpn3 library. This is
completely fine. I am not arguing it should be different.

So I have two options here:

a) I finish my own implementation of the provider for OpenVPN3 to not be
influenced by this implementation and review this implementation after that

b) you agree that I can take parts of your code for the OpenVPN3
implementation, then I go directly into review and then base my OpenVPN3
implementation on your xkey provider implementation.

Arne
Arne Schwabe Sept. 23, 2021, 11:12 a.m. | #2
> So I have two options here:
> 
> a) I finish my own implementation of the provider for OpenVPN3 to not be
> influenced by this implementation and review this implementation after that
> 
> b) you agree that I can take parts of your code for the OpenVPN3
> implementation, then I go directly into review and then base my OpenVPN3
> implementation on your xkey provider implementation.

Just to clarify. Both ways are fine with me.

Arne
Selva Nair Sept. 23, 2021, 2:02 p.m. | #3
Hi Arne,


> So I have two options here:
>
> a) I finish my own implementation of the provider for OpenVPN3 to not be
> influenced by this implementation and review this implementation after that
>
> b) you agree that I can take parts of your code for the OpenVPN3
> implementation, then I go directly into review and then base my OpenVPN3
> implementation on your xkey provider implementation.
>

(b) is fine with me and preferred. I do not want to release it under BSD,
but can agree to  a CLA. I wrote it consulting OpenSSL sources (though no
copy paste) which is not GPL2, so not even sure what license is most
appropriate. But BSD isi not my thing.

If there is a specific wording required, I can add that to the
xkey_provider.c file which contains the implementation. Or make a separate
copy for you to pick for OpenVPN3 The rest is generic interfacing with
OpenVPN.

Thanks,

Selva



> Arne
>
<div dir="ltr"><div><br></div><div class="gmail_quote"><div>Hi Arne,</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
So I have two options here:<br>
<br>
a) I finish my own implementation of the provider for OpenVPN3 to not be<br>
influenced by this implementation and review this implementation after that<br>
<br>
b) you agree that I can take parts of your code for the OpenVPN3<br>
implementation, then I go directly into review and then base my OpenVPN3<br>
implementation on your xkey provider implementation.<br></blockquote><div><br></div><div>(b) is fine with me and preferred. I do not want to release it under BSD, but can agree to  a CLA. I wrote it consulting OpenSSL sources (though no copy paste) which is not GPL2, so not even sure what license is most appropriate. But BSD isi not my thing. </div><div><br></div><div>If there is a specific wording required, I can add that to the xkey_provider.c file which contains the implementation. Or make a separate copy for you to pick for OpenVPN3 The rest is generic interfacing with OpenVPN.</div><div><br></div><div>Thanks,</div><div><br></div><div>Selva</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Arne<br>
</blockquote></div></div>
Arne Schwabe Sept. 23, 2021, 3:27 p.m. | #4
Am 23.09.21 um 16:02 schrieb Selva Nair:
> 
> Hi Arne,
> 
> 
>     So I have two options here:
> 
>     a) I finish my own implementation of the provider for OpenVPN3 to not be
>     influenced by this implementation and review this implementation
>     after that
> 
>     b) you agree that I can take parts of your code for the OpenVPN3
>     implementation, then I go directly into review and then base my OpenVPN3
>     implementation on your xkey provider implementation.
> 
> 
> (b) is fine with me and preferred. I do not want to release it under
> BSD, but can agree to  a CLA. I wrote it consulting OpenSSL sources
> (though no copy paste) which is not GPL2, so not even sure what license
> is most appropriate. But BSD isi not my thing. 

I think a statement from you that you agree to
https://github.com/OpenVPN/openvpn3/blob/master/CLA.rst
for this patchset would be good and I think also enough.

Arne
Selva Nair Sept. 23, 2021, 3:40 p.m. | #5
Hi

On Thu, Sep 23, 2021 at 11:27 AM Arne Schwabe <arne@rfc2549.org> wrote:

> Am 23.09.21 um 16:02 schrieb Selva Nair:
> >
> > Hi Arne,
> >
> >
> >     So I have two options here:
> >
> >     a) I finish my own implementation of the provider for OpenVPN3 to
> not be
> >     influenced by this implementation and review this implementation
> >     after that
> >
> >     b) you agree that I can take parts of your code for the OpenVPN3
> >     implementation, then I go directly into review and then base my
> OpenVPN3
> >     implementation on your xkey provider implementation.
> >
> >
> > (b) is fine with me and preferred. I do not want to release it under
> > BSD, but can agree to  a CLA. I wrote it consulting OpenSSL sources
> > (though no copy paste) which is not GPL2, so not even sure what license
> > is most appropriate. But BSD isi not my thing.
>
> I think a statement from you that you agree to
> https://github.com/OpenVPN/openvpn3/blob/master/CLA.rst
> for this patchset would be good and I think also enough.


Okay, I will send you an email agreeing to that.

I'll also change the license header from "version 2" to
"either version 2 of the License, or (at your option) any later version."
following the FSF's wording for allowing any later version of GPL.

Selva


> Arne
>
<div dir="ltr"><div dir="ltr"><div>Hi</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 23, 2021 at 11:27 AM Arne Schwabe &lt;<a href="mailto:arne@rfc2549.org" target="_blank">arne@rfc2549.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Am 23.09.21 um 16:02 schrieb Selva Nair:<br>
&gt; <br>
&gt; Hi Arne,<br>
&gt; <br>
&gt; <br>
&gt;     So I have two options here:<br>
&gt; <br>
&gt;     a) I finish my own implementation of the provider for OpenVPN3 to not be<br>
&gt;     influenced by this implementation and review this implementation<br>
&gt;     after that<br>
&gt; <br>
&gt;     b) you agree that I can take parts of your code for the OpenVPN3<br>
&gt;     implementation, then I go directly into review and then base my OpenVPN3<br>
&gt;     implementation on your xkey provider implementation.<br>
&gt; <br>
&gt; <br>
&gt; (b) is fine with me and preferred. I do not want to release it under<br>
&gt; BSD, but can agree to  a CLA. I wrote it consulting OpenSSL sources<br>
&gt; (though no copy paste) which is not GPL2, so not even sure what license<br>
&gt; is most appropriate. But BSD isi not my thing. <br>
<br>
I think a statement from you that you agree to<br>
<a href="https://github.com/OpenVPN/openvpn3/blob/master/CLA.rst" rel="noreferrer" target="_blank">https://github.com/OpenVPN/openvpn3/blob/master/CLA.rst</a><br>
for this patchset would be good and I think also enough.</blockquote><div> </div><div>Okay, I will send you an email agreeing to that.</div><div><br></div><div>I&#39;ll also change the license header from &quot;version 2&quot; to</div><div>&quot;either version 2 of the License, or (at your option) any later version.&quot;<br></div><div>following the FSF&#39;s wording for allowing any later version of GPL. </div><div><br></div><div>Selva</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Arne<br>
</blockquote></div></div></div>
Arne Schwabe Sept. 23, 2021, 8:21 p.m. | #6
Am 22.09.21 um 23:12 schrieb selva.nair@gmail.com:
> From: Selva Nair <selva.nair@gmail.com>
> 
> The following series of patches implement a built-in
> provider for interfacing OpenSSL 3.0 when external 
> keys are in use.
> 
> Essentially, to intercept the sign operation, the SSL_CTX
> object has to be created with properties string set to 
> prioritize our provider. In the provider we implement
> only keymgmt and signature operations and specify the
> property string as optional. That allows all operations
> we do not provide to be used from the default provider.
> 
> This patch set stops at interfacing the provider with
> management-external-key. For pkcs11-helper, only some glue
> code is needed and is in the works. Same with cryptoapicert
>  aka CNG, but I want to cleanup the old code a bit before
> hooking to the provider.

I did a quick test with my Android client to see if it works and RSA
keys look good so far. I am getting a request like:

NC9t8IkYrjAQcCzc85zN0H5TvwfAUDwYkR4j2ga6fGw=,RSA_PKCS1_PSS_PADDING,hashalg=SHA256,saltlen=digest

from the management interface. But I haven't found the right Signature
method from java yet to actually sign it correctly:

sig = Signature.getInstance(SHA256withRSA/PSS);
sig.setParameter(new PSSParameterSpec("SHA-256", "MGF1",
MGF1ParameterSpec.SHA256, 32, 1));
sig.initSign(privkey);
sig.update(data);
signed_bytes = sig.sign();

is what I expected to be the correct signature but the server complains
with OpenSSL: error:0407E068:rsa routines:RSA_verify_PKCS1_PSS_mgf1:bad
signature

I will have to figure out where this goes wrong.

With an EC key somewhere in that stack, EC/RSA gets confuse as there is
rsa_keymgmt_import/rsa_keymgmt_name in the stack and then later
ec_keymgmt_name. I haven't digged into that as it is getting late here.


2021-09-23 22:19:56 TLS: Initial packet from
[AF_INET]192.168.188.61:1194, sid=7c606dcc fe241304
2021-09-23 22:19:56 In xkey provider query op with op = 4
2021-09-23 22:19:56 In xkey provider query op with op = 3
2021-09-23 22:19:56 In xkey provider query op with op = 10
2021-09-23 22:19:56 In xkey provider query op with op = 21
2021-09-23 22:19:56 VERIFY OK: depth=0, CN=dionysos
2021-09-23 22:19:56 In keymgmt_new
2021-09-23 22:19:56 In keydata_new
2021-09-23 22:19:56 In rsa_keymgmt_import
2021-09-23 22:19:56 In keymgmt_import
2021-09-23 22:19:56 In rsa_keymgmt_name
2021-09-23 22:19:56 In xkey signature_newctx
2021-09-23 22:19:56 In xkey digest_verify init with mdname <SHA2-256>
2021-09-23 22:19:56 In xkey digest_init_helper with mdname = <SHA2-256>
2021-09-23 22:19:56 In xkey signature_settable_ctx_params
2021-09-23 22:19:56 In signature_set_ctx_params
2021-09-23 22:19:56 xkey_sign_parameters: setting padmode to <pss>
2021-09-23 22:19:56 In xkey signature_settable_ctx_params
2021-09-23 22:19:56 In signature_set_ctx_params
2021-09-23 22:19:56 xkey_sign_parameters: setting saltlen to digest
2021-09-23 22:19:56 In xkey digest_verify
2021-09-23 22:19:56 In xkey signature_freectx
2021-09-23 22:19:56 In ec_keymgmt_name
2021-09-23 22:19:56 In xkey provider query op with op = 12
2021-09-23 22:19:56 In ec_keymgmt_name
2021-09-23 22:19:56 In xkey provider query op with op = 12
2021-09-23 22:19:56 In ec_keymgmt_name
2021-09-23 22:19:56 In xkey provider query op with op = 12


RSA for comparison:

2021-09-23 22:17:40 TLS: Initial packet from
[AF_INET]192.168.188.61:1194, sid=0e4a91a6 67f591d2
2021-09-23 22:17:40 In xkey provider query op with op = 4
2021-09-23 22:17:40 In xkey provider query op with op = 3
2021-09-23 22:17:40 In xkey provider query op with op = 10
2021-09-23 22:17:40 In xkey provider query op with op = 21
2021-09-23 22:17:40 VERIFY OK: depth=0, CN=dionysos
2021-09-23 22:17:40 In keymgmt_new
2021-09-23 22:17:40 In keydata_new
2021-09-23 22:17:40 In rsa_keymgmt_import
2021-09-23 22:17:40 In keymgmt_import
2021-09-23 22:17:40 In rsa_keymgmt_name
2021-09-23 22:17:40 In xkey signature_newctx
2021-09-23 22:17:40 In xkey digest_verify init with mdname <SHA2-256>
2021-09-23 22:17:40 In xkey digest_init_helper with mdname = <SHA2-256>
2021-09-23 22:17:40 In xkey signature_settable_ctx_params
2021-09-23 22:17:40 In signature_set_ctx_params
2021-09-23 22:17:40 xkey_sign_parameters: setting padmode to <pss>
2021-09-23 22:17:40 In xkey signature_settable_ctx_params
2021-09-23 22:17:40 In signature_set_ctx_params
2021-09-23 22:17:40 xkey_sign_parameters: setting saltlen to digest
2021-09-23 22:17:40 In xkey digest_verify
2021-09-23 22:17:40 In xkey signature_freectx
2021-09-23 22:17:40 In rsa_keymgmt_name
2021-09-23 22:17:40 In xkey signature_newctx
2021-09-23 22:17:40 In xkey digest_sign_init with mdname = SHA256>
2021-09-23 22:17:40 In signature_set_ctx_params
2021-09-23 22:17:40 In xkey signature_freectx
2021-09-23 22:17:40 In rsa_keymgmt_name
2021-09-23 22:17:40 In xkey signature_newctx
2021-09-23 22:17:40 In xkey digest_sign_init with mdname = SHA2-256>
2021-09-23 22:17:40 In signature_set_ctx_params
2021-09-23 22:17:40 In xkey signature_settable_ctx_params
2021-09-23 22:17:40 In signature_set_ctx_params
2021-09-23 22:17:40 xkey_sign_parameters: setting padmode to <pss>
2021-09-23 22:17:40 In xkey signature_settable_ctx_params
2021-09-23 22:17:40 In signature_set_ctx_params
2021-09-23 22:17:40 xkey_sign_parameters: setting saltlen to digest
2021-09-23 22:17:40 In xkey digest_sign
2021-09-23 22:17:40 In xkey digest_sign
2021-09-23 22:17:40 In xkey signature_sign with siglen = 256
2021-09-23 22:17:40 P:
2021-09-23 22:17:40 xkey management_sign: requesting sig with algorithm
<RSA_PKCS1_PSS_PADDING,hashalg=SHA256,saltlen=digest>
2021-09-23 22:17:40 MANAGEMENT: CMD 'pk-sig'
2021-09-23 22:17:40 In xkey signature_freectx
Selva Nair Sept. 23, 2021, 10:16 p.m. | #7
On Thu, Sep 23, 2021 at 4:21 PM Arne Schwabe <arne@rfc2549.org> wrote:

> Am 22.09.21 um 23:12 schrieb selva.nair@gmail.com:
> > From: Selva Nair <selva.nair@gmail.com>
> >
> > The following series of patches implement a built-in
> > provider for interfacing OpenSSL 3.0 when external
> > keys are in use.
> >
> > Essentially, to intercept the sign operation, the SSL_CTX
> > object has to be created with properties string set to
> > prioritize our provider. In the provider we implement
> > only keymgmt and signature operations and specify the
> > property string as optional. That allows all operations
> > we do not provide to be used from the default provider.
> >
> > This patch set stops at interfacing the provider with
> > management-external-key. For pkcs11-helper, only some glue
> > code is needed and is in the works. Same with cryptoapicert
> >  aka CNG, but I want to cleanup the old code a bit before
> > hooking to the provider.
>
> I did a quick test with my Android client to see if it works and RSA
> keys look good so far. I am getting a request like:
>
>
> NC9t8IkYrjAQcCzc85zN0H5TvwfAUDwYkR4j2ga6fGw=,RSA_PKCS1_PSS_PADDING,hashalg=SHA256,saltlen=digest
>
> from the management interface. But I haven't found the right Signature
> method from java yet to actually sign it correctly:
>
> sig = Signature.getInstance(SHA256withRSA/PSS);
> sig.setParameter(new PSSParameterSpec("SHA-256", "MGF1",
> MGF1ParameterSpec.SHA256, 32, 1));
> sig.initSign(privkey);
> sig.update(data);
> signed_bytes = sig.sign();
>

I'm not sure, but  can upload my implementation of pkcs11 including PSS
though it wont work yet with pkcs11-helper.. It needs my CK_MECHANISM patch
that Alon has merged, but not released.

It may give you some clue as to what could be wrong.


>
> is what I expected to be the correct signature but the server complains
> with OpenSSL: error:0407E068:rsa routines:RSA_verify_PKCS1_PSS_mgf1:bad
> signature
>
> I will have to figure out where this goes wrong.
>
> With an EC key somewhere in that stack, EC/RSA gets confuse as there is
> rsa_keymgmt_import/rsa_keymgmt_name in the stack and then later
> ec_keymgmt_name. I haven't digged into that as it is getting late here.
>

Yes, there is a bug in ec_keymgmt_name -- we should return the name for the
op (passed as id), so when id == OSSL_OP_SIGNATURE we should return
"ECDSA", not "EC". My mistake -- I testd EC key only just now.

As we do not support any other ops like ECDH key exchange we can just
always return "ECDSA" in that function. I will do it v2, later.
Unfortunately none of this is documented it seems.

Selva
<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 23, 2021 at 4:21 PM Arne Schwabe &lt;<a href="mailto:arne@rfc2549.org">arne@rfc2549.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Am 22.09.21 um 23:12 schrieb <a href="mailto:selva.nair@gmail.com" target="_blank">selva.nair@gmail.com</a>:<br>
&gt; From: Selva Nair &lt;<a href="mailto:selva.nair@gmail.com" target="_blank">selva.nair@gmail.com</a>&gt;<br>
&gt; <br>
&gt; The following series of patches implement a built-in<br>
&gt; provider for interfacing OpenSSL 3.0 when external <br>
&gt; keys are in use.<br>
&gt; <br>
&gt; Essentially, to intercept the sign operation, the SSL_CTX<br>
&gt; object has to be created with properties string set to <br>
&gt; prioritize our provider. In the provider we implement<br>
&gt; only keymgmt and signature operations and specify the<br>
&gt; property string as optional. That allows all operations<br>
&gt; we do not provide to be used from the default provider.<br>
&gt; <br>
&gt; This patch set stops at interfacing the provider with<br>
&gt; management-external-key. For pkcs11-helper, only some glue<br>
&gt; code is needed and is in the works. Same with cryptoapicert<br>
&gt;  aka CNG, but I want to cleanup the old code a bit before<br>
&gt; hooking to the provider.<br>
<br>
I did a quick test with my Android client to see if it works and RSA<br>
keys look good so far. I am getting a request like:<br>
<br>
NC9t8IkYrjAQcCzc85zN0H5TvwfAUDwYkR4j2ga6fGw=,RSA_PKCS1_PSS_PADDING,hashalg=SHA256,saltlen=digest<br>
<br>
from the management interface. But I haven&#39;t found the right Signature<br>
method from java yet to actually sign it correctly:<br>
<br>
sig = Signature.getInstance(SHA256withRSA/PSS);<br>
sig.setParameter(new PSSParameterSpec(&quot;SHA-256&quot;, &quot;MGF1&quot;,<br>
MGF1ParameterSpec.SHA256, 32, 1));<br>
sig.initSign(privkey);<br>
sig.update(data);<br>
signed_bytes = sig.sign();<br></blockquote><div><br></div><div>I&#39;m not sure, but  can upload my implementation of pkcs11 including PSS though it wont work yet with pkcs11-helper.. It needs my CK_MECHANISM patch that Alon has merged, but not released.</div><div><br></div><div>It may give you some clue as to what could be wrong.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
is what I expected to be the correct signature but the server complains<br>
with OpenSSL: error:0407E068:rsa routines:RSA_verify_PKCS1_PSS_mgf1:bad<br>
signature<br>
<br>
I will have to figure out where this goes wrong.<br>
<br>
With an EC key somewhere in that stack, EC/RSA gets confuse as there is<br>
rsa_keymgmt_import/rsa_keymgmt_name in the stack and then later<br>
ec_keymgmt_name. I haven&#39;t digged into that as it is getting late here.<br></blockquote><div><br></div><div>Yes, there is a bug in ec_keymgmt_name -- we should return the name for the op (passed as id), so when id == OSSL_OP_SIGNATURE we should return &quot;ECDSA&quot;, not &quot;EC&quot;. My mistake -- I testd EC key only just now. </div><div><br></div><div>As we do not support any other ops like ECDH key exchange we can just always return &quot;ECDSA&quot; in that function. I will do it v2, later. Unfortunately none of this is documented it seems.</div><div><br></div><div>Selva</div></div></div>
Selva Nair Sept. 23, 2021, 10:54 p.m. | #8
Hi,


> from the management interface. But I haven't found the right Signature
>> method from java yet to actually sign it correctly:
>>
>> sig = Signature.getInstance(SHA256withRSA/PSS);
>>
>
SHA256withRSA/PSS may be trying to first do Sha256 digest of the data and
then pad and sign. Instead try this: "NonewithRSASSA-PSS" or
"NonewithRSA/PSS"


> sig.setParameter(new PSSParameterSpec("SHA-256", "MGF1",
>> MGF1ParameterSpec.SHA256, 32, 1));
>> sig.initSign(privkey);
>> sig.update(data);
>> signed_bytes = sig.sign();
>
>
>>
You would still need SHA-256 in the PSSParameterSpec.

Selva
<div dir="ltr"><div class="gmail_quote"><div>Hi,</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
from the management interface. But I haven&#39;t found the right Signature<br>
method from java yet to actually sign it correctly:<br>
<br>
sig = Signature.getInstance(SHA256withRSA/PSS);<br></blockquote></div></div></blockquote><div><br></div><div>SHA256withRSA/PSS may be trying to first do Sha256 digest of the data and then pad and sign. Instead try this: &quot;NonewithRSASSA-PSS&quot; or &quot;NonewithRSA/PSS&quot;</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
sig.setParameter(new PSSParameterSpec(&quot;SHA-256&quot;, &quot;MGF1&quot;,<br>
MGF1ParameterSpec.SHA256, 32, 1));<br>
sig.initSign(privkey);<br>
sig.update(data);<br>
signed_bytes = sig.sign();</blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br></blockquote></div></div></blockquote><div><br></div><div>You would still need SHA-256 in the PSSParameterSpec.</div><div><br></div><div>Selva</div><div> </div></div></div>
Arne Schwabe Sept. 24, 2021, 11:13 a.m. | #9
Am 24.09.21 um 00:54 schrieb Selva Nair:
> Hi,
>  
> 
>         from the management interface. But I haven't found the right
>         Signature
>         method from java yet to actually sign it correctly:
> 
>         sig = Signature.getInstance(SHA256withRSA/PSS);
> 
> 
> SHA256withRSA/PSS may be trying to first do Sha256 digest of the data
> and then pad and sign. Instead try this: "NonewithRSASSA-PSS" or
> "NonewithRSA/PSS"

Yeah, That *would* be the correct algorithm for that. Unfortunately the
Android Keystore does not support that one
(https://developer.android.com/training/articles/keystore#SupportedSignatures)


Manually adding the RSA/PSS padding and then signing with
"RSA/ECB/NoPadding" like I did in OpenSSL 1.1 days works. But I would
like to avoid implementing RSA/PSS myself but the crypto libraries seem
not be helpful in providing an implementation for that.

OpenSSL has RSA_padding_add_PKCS1_PSS_mgf1 but it is deprecated in 3.0
and the Java crypto API also does not seem to expose an API for just
padding it seems.

But on the plus side, using that workaround the external key provider
works with EC and RSA on Android.

Arne
Selva Nair Sept. 24, 2021, 12:48 p.m. | #10
Hi,

On Fri, Sep 24, 2021 at 7:13 AM Arne Schwabe <arne@rfc2549.org> wrote:

> Am 24.09.21 um 00:54 schrieb Selva Nair:
> > Hi,
> >
> >
> >         from the management interface. But I haven't found the right
> >         Signature
> >         method from java yet to actually sign it correctly:
> >
> >         sig = Signature.getInstance(SHA256withRSA/PSS);
> >
> >
> > SHA256withRSA/PSS may be trying to first do Sha256 digest of the data
> > and then pad and sign. Instead try this: "NonewithRSASSA-PSS" or
> > "NonewithRSA/PSS"
>
> Yeah, That *would* be the correct algorithm for that. Unfortunately the
> Android Keystore does not support that one
> (
> https://developer.android.com/training/articles/keystore#SupportedSignatures
> )
>

We can treat management-external key as special and optionally provide the
digest to sign. OpenSSL 3.0 with provider always seem to call DigestSign
and never Sign directly so we have the info.

I already have key->origin indication which was originally meant to
distinguish between various external libraries -- in the end I simplified
that a bit. So its easy to treat management external key differently from
other external keys. So for management it would a digest_sign callback if
"--management-external-key digest" is specified, or some such. Harder part
would be to get access to options->management_flags into xkey_helper.c -- I
have tried to keep a max separation between it and the core and would like
to keep it that way.

Or make another ifdef for ANDROID... ahem..

I'll look into it. Probably your app is the only --management-external-key
"consumer" out there. We can even change the spec of PK_SIG and no one will
notice..

Manually adding the RSA/PSS padding and then signing with
> "RSA/ECB/NoPadding" like I did in OpenSSL 1.1 days works. But I would
> like to avoid implementing RSA/PSS myself but the crypto libraries seem
> not be helpful in providing an implementation for that.
>

Yeah, that's not good.


> But on the plus side, using that workaround the external key provider
> works with EC and RSA on Android.
>

Great!

Selva
<div dir="ltr"><div>Hi,</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 24, 2021 at 7:13 AM Arne Schwabe &lt;<a href="mailto:arne@rfc2549.org" target="_blank">arne@rfc2549.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Am 24.09.21 um 00:54 schrieb Selva Nair:<br>
&gt; Hi,<br>
&gt;  <br>
&gt; <br>
&gt;         from the management interface. But I haven&#39;t found the right<br>
&gt;         Signature<br>
&gt;         method from java yet to actually sign it correctly:<br>
&gt; <br>
&gt;         sig = Signature.getInstance(SHA256withRSA/PSS);<br>
&gt; <br>
&gt; <br>
&gt; SHA256withRSA/PSS may be trying to first do Sha256 digest of the data<br>
&gt; and then pad and sign. Instead try this: &quot;NonewithRSASSA-PSS&quot; or<br>
&gt; &quot;NonewithRSA/PSS&quot;<br>
<br>
Yeah, That *would* be the correct algorithm for that. Unfortunately the<br>
Android Keystore does not support that one<br>
(<a href="https://developer.android.com/training/articles/keystore#SupportedSignatures" rel="noreferrer" target="_blank">https://developer.android.com/training/articles/keystore#SupportedSignatures</a>)<br></blockquote><div><br></div><div>We can treat management-external key as special and optionally provide the digest to sign. OpenSSL 3.0 with provider always seem to call DigestSign and never Sign directly so we have the info. </div><div><br></div><div>I already have key-&gt;origin indication which was originally meant to distinguish between various external libraries -- in the end I simplified that a bit. So its easy to treat management external key differently from other external keys. So for management it would a digest_sign callback if &quot;--management-external-key digest&quot; is specified, or some such. Harder part would be to get access to options-&gt;management_flags into xkey_helper.c -- I have tried to keep a max separation between it and the core and would like to keep it that way.</div><div><br></div><div>Or make another ifdef for ANDROID... ahem..</div><div><br></div><div>I&#39;ll look into it. Probably your app is the only --management-external-key &quot;consumer&quot; out there. We can even change the spec of PK_SIG and no one will notice..</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Manually adding the RSA/PSS padding and then signing with<br>
&quot;RSA/ECB/NoPadding&quot; like I did in OpenSSL 1.1 days works. But I would<br>
like to avoid implementing RSA/PSS myself but the crypto libraries seem<br>
not be helpful in providing an implementation for that.<br></blockquote><div><br></div><div>Yeah, that&#39;s not good.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
But on the plus side, using that workaround the external key provider<br>
works with EC and RSA on Android.<br></blockquote><div><br></div><div>Great!</div><div><br></div><div>Selva </div></div></div>
Selva Nair Sept. 24, 2021, 5:14 p.m. | #11
Hi Arne,

On Fri, Sep 24, 2021 at 8:48 AM Selva Nair <selva.nair@gmail.com> wrote:

> Hi,
>
> On Fri, Sep 24, 2021 at 7:13 AM Arne Schwabe <arne@rfc2549.org> wrote:
>
>> Am 24.09.21 um 00:54 schrieb Selva Nair:
>> > Hi,
>> >
>> >
>> >         from the management interface. But I haven't found the right
>> >         Signature
>> >         method from java yet to actually sign it correctly:
>> >
>> >         sig = Signature.getInstance(SHA256withRSA/PSS);
>> >
>> >
>> > SHA256withRSA/PSS may be trying to first do Sha256 digest of the data
>> > and then pad and sign. Instead try this: "NonewithRSASSA-PSS" or
>> > "NonewithRSA/PSS"
>>
>> Yeah, That *would* be the correct algorithm for that. Unfortunately the
>> Android Keystore does not support that one
>> (
>> https://developer.android.com/training/articles/keystore#SupportedSignatures
>> )
>>
>
> We can treat management-external key as special and optionally provide the
> digest to sign. OpenSSL 3.0 with provider always seem to call DigestSign
> and never Sign directly so we have the info.
>

Turns out to be easier than I thought. I have added a patch to optionally
send the undigested message to the management client. Indicate support for
digesting operation in --management-external-key and you get the message to
sign with data=message

See the commit message. It's in
https://github.com/selvanair/openvpn/tree/xkey-provider-v3   (only compile
tested).

Selva
PS. I'm supposed to be holidaying, but basking in LCD glow instead of sun..
<div dir="ltr"><div dir="ltr">Hi Arne,</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 24, 2021 at 8:48 AM Selva Nair &lt;<a href="mailto:selva.nair@gmail.com">selva.nair@gmail.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 24, 2021 at 7:13 AM Arne Schwabe &lt;<a href="mailto:arne@rfc2549.org" target="_blank">arne@rfc2549.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Am 24.09.21 um 00:54 schrieb Selva Nair:<br>
&gt; Hi,<br>
&gt;  <br>
&gt; <br>
&gt;         from the management interface. But I haven&#39;t found the right<br>
&gt;         Signature<br>
&gt;         method from java yet to actually sign it correctly:<br>
&gt; <br>
&gt;         sig = Signature.getInstance(SHA256withRSA/PSS);<br>
&gt; <br>
&gt; <br>
&gt; SHA256withRSA/PSS may be trying to first do Sha256 digest of the data<br>
&gt; and then pad and sign. Instead try this: &quot;NonewithRSASSA-PSS&quot; or<br>
&gt; &quot;NonewithRSA/PSS&quot;<br>
<br>
Yeah, That *would* be the correct algorithm for that. Unfortunately the<br>
Android Keystore does not support that one<br>
(<a href="https://developer.android.com/training/articles/keystore#SupportedSignatures" rel="noreferrer" target="_blank">https://developer.android.com/training/articles/keystore#SupportedSignatures</a>)<br></blockquote><div><br></div><div>We can treat management-external key as special and optionally provide the digest to sign. OpenSSL 3.0 with provider always seem to call DigestSign and never Sign directly so we have the info. <br></div></div></div></blockquote><div><br></div><div>Turns out to be easier than I thought. I have added a patch to optionally send the undigested message to the management client. Indicate support for digesting operation in --management-external-key and you get the message to sign with data=message</div><div><br></div><div>See the commit message. It&#39;s in  <a href="https://github.com/selvanair/openvpn/tree/xkey-provider-v3">https://github.com/selvanair/openvpn/tree/xkey-provider-v3</a>   (only compile tested). </div><div><br></div><div>Selva</div><div>PS. I&#39;m supposed to be holidaying, but basking in LCD glow instead of sun..</div></div></div>
Gert Doering Sept. 24, 2021, 6:31 p.m. | #12
Hi,

On Fri, Sep 24, 2021 at 01:14:34PM -0400, Selva Nair wrote:
> PS. I'm supposed to be holidaying, but basking in LCD glow instead of sun..

Sometimes "I finally have time for hacking!" makes great holidays :-)

(The initial IPv6 patch set was a christmas present, sort of - my wife
let me hack a few days on it, without being offended :-) )

gert
Arne Schwabe Sept. 26, 2021, 1:15 a.m. | #13
> 
>     We can treat management-external key as special and optionally
>     provide the digest to sign. OpenSSL 3.0 with provider always seem to
>     call DigestSign and never Sign directly so we have the info. 
> 
> 
> Turns out to be easier than I thought. I have added a patch to
> optionally send the undigested message to the management client.
> Indicate support for digesting operation in --management-external-key
> and you get the message to sign with data=message

That is great. I tested it and TLS 1.3 is working nicely with
"SHA256withRSA/PSS" on the java side. So that is great.

But then it went downhill from that with things unrelated to the xkey
provider. I wanted to see if it works with TLS 1.1 (ie.. forcing a pkcs1
signature) and found out tls-version-max no longer work on OpenSSL 3.0
nor on my Ubuntu 20 with OpenSSL 1.1.

I decided then to replace SSLv23_client_method with TLS11_client_method,
which ended with an internal SSL error in my client.

So instead I tried to see tls-version-max still works with mbed TLS only
to discover that my OpenSSL 3.0 client does not connect to my mbed TLS
at all and instead the server says "TLS_ERROR: read tls_read_plaintext
error: SSL - Bad input parameters to function" (Ubuntu 20/mbed TLS
2.16.0). Switching the app back to OpenSSL 1.1.1l still works.

Then tested again with OpenSSL 3.0 but forgot to reenable
HAVE_XKEY_PROVIDER and now the RSA_method/EC_method works but I was
fairly sure that it didn't before. Maybe there is some obscure side
effect from your branch or I misremembered before ...

But for some reason having the HAVE_XKEY_PROVIDER define enabled breaks
connecting to an mbed TLS 2.16 server. I can also reproduce that with my
a version on my mac.

> 
> See the commit message. It's
> in  https://github.com/selvanair/openvpn/tree/xkey-provider-v3
> <https://github.com/selvanair/openvpn/tree/xkey-provider-v3>   (only
> compile tested). 
> 
> Selva
> PS. I'm supposed to be holidaying, but basking in LCD glow instead of sun..


Hehe, and I am supposed to be sleeping but instead I am testing some
obscure software part that is even hard to explain to other IT (non
crypto) people what it does ....
Selva Nair Oct. 5, 2021, 4:39 p.m. | #14
Hi

Here is an update on this patch set to keep all in the loop.

Arne discovered that my patch broke ECDH key exchange in some cases.  This
turns out to be due to the way providers are handled in OpenSSL especially
when used in a TLS context. It leads to the requirement that an external
provider has to handle a wide zoo of  key operations including key exchange
and key generation, even if all it wants to do is signing with an external
key. Essentially something like: "you either export the key to me or be
ready to import and handle all operations on any asymmetric key I may come
across". We can't export as the key is in a protected storage in some
backend,  we also do not want to do all that extra work that's not in the
contract, and we are not good at it either.

I have been engaging with OpenSSL developers on this and they realize this
was unintended, and is a "bug/weakness" in their implementation. They are
working on a patch to fix it at their end (
https://github.com/openssl/openssl/pull/16725). The eventual fix is very
likely to get backported to OpenSSL 3.0, so we have to wait.

I'll submit a slightly modified v2 once their fix is finalized.

Thanks,

Selva

>
<div dir="ltr"><div>Hi<br></div><div class="gmail_quote"><br></div><div class="gmail_quote">Here is an update on this patch set to keep all in the loop. </div><div class="gmail_quote"><br></div><div class="gmail_quote">Arne discovered that my patch broke ECDH key exchange in some cases.  This turns out to be due to the way providers are handled in OpenSSL especially when used in a TLS context. It leads to the requirement that an external provider has to handle a wide zoo of  key operations including key exchange and key generation, even if all it wants to do is signing with an external key. Essentially something like: &quot;you either export the key to me or be ready to import and handle all operations on any asymmetric key I may come across&quot;. We can&#39;t export as the key is in a protected storage in some backend,  we also do not want to do all that extra work that&#39;s not in the contract, and we are not good at it either.</div><div class="gmail_quote"><br></div><div class="gmail_quote">I have been engaging with OpenSSL developers on this and they realize this was unintended, and is a &quot;bug/weakness&quot; in their implementation. They are working on a patch to fix it at their end (<a href="https://github.com/openssl/openssl/pull/16725">https://github.com/openssl/openssl/pull/16725</a>). The eventual fix is very likely to get backported to OpenSSL 3.0, so we have to wait.</div><div class="gmail_quote"><br></div><div class="gmail_quote">I&#39;ll submit a slightly modified v2 once their fix is finalized.</div><div class="gmail_quote"><br></div><div class="gmail_quote">Thanks,</div><div class="gmail_quote"><br></div><div class="gmail_quote">Selva</div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
</blockquote></div></div>
Selva Nair Nov. 2, 2021, 4:40 a.m. | #15
Hi,

OpenSSL folks have merged their "fix" in the provider interface that I was
waiting for. It will be in the 3.0.1 patch release. In the meantime, I have
opened a matching version of this patch set as a PR for OpenVPN for
comments/tests/bug-reports/nitpicks. I skipped v2 and this version is
tagged v3.

Will post the patches to the list when OpenSSL 3.0.1 is released.

On top of v1 patches this also includes handling pkcs11-id and
cryptoapicert options through the provider. Requires OpenSSL from either
the master branch (3.1.0-dev) or 3.0 branch (3.0.1-dev) post Oct. 27.

Cheers,

Selva

On Tue, Oct 5, 2021 at 12:39 PM Selva Nair <selva.nair@gmail.com> wrote:

> Hi
>
> Here is an update on this patch set to keep all in the loop.
>
> Arne discovered that my patch broke ECDH key exchange in some cases.  This
> turns out to be due to the way providers are handled in OpenSSL especially
> when used in a TLS context. It leads to the requirement that an external
> provider has to handle a wide zoo of  key operations including key exchange
> and key generation, even if all it wants to do is signing with an external
> key. Essentially something like: "you either export the key to me or be
> ready to import and handle all operations on any asymmetric key I may come
> across". We can't export as the key is in a protected storage in some
> backend,  we also do not want to do all that extra work that's not in the
> contract, and we are not good at it either.
>
> I have been engaging with OpenSSL developers on this and they realize this
> was unintended, and is a "bug/weakness" in their implementation. They are
> working on a patch to fix it at their end (
> https://github.com/openssl/openssl/pull/16725). The eventual fix is very
> likely to get backported to OpenSSL 3.0, so we have to wait.
>
> I'll submit a slightly modified v2 once their fix is finalized.
>
> Thanks,
>
> Selva
>
>>
<div dir="ltr">Hi,<div><br></div><div>OpenSSL folks have merged their &quot;fix&quot; in the provider interface that I was waiting for. It will be in the 3.0.1 patch release. In the meantime, I have opened a matching version of this patch set as a PR for OpenVPN for comments/tests/bug-reports/nitpicks. I skipped v2 and this version is tagged v3.</div><div><br></div><div>Will post the patches to the list when OpenSSL 3.0.1 is released.</div><div><br></div><div>On top of v1 patches this also includes handling pkcs11-id and cryptoapicert options through the provider. Requires OpenSSL from either the master branch (3.1.0-dev) or 3.0 branch (3.0.1-dev) post Oct. 27.</div><div><br></div><div>Cheers,</div><div><br></div><div>Selva</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 5, 2021 at 12:39 PM Selva Nair &lt;<a href="mailto:selva.nair@gmail.com" target="_blank">selva.nair@gmail.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi<br></div><div class="gmail_quote"><br></div><div class="gmail_quote">Here is an update on this patch set to keep all in the loop. </div><div class="gmail_quote"><br></div><div class="gmail_quote">Arne discovered that my patch broke ECDH key exchange in some cases.  This turns out to be due to the way providers are handled in OpenSSL especially when used in a TLS context. It leads to the requirement that an external provider has to handle a wide zoo of  key operations including key exchange and key generation, even if all it wants to do is signing with an external key. Essentially something like: &quot;you either export the key to me or be ready to import and handle all operations on any asymmetric key I may come across&quot;. We can&#39;t export as the key is in a protected storage in some backend,  we also do not want to do all that extra work that&#39;s not in the contract, and we are not good at it either.</div><div class="gmail_quote"><br></div><div class="gmail_quote">I have been engaging with OpenSSL developers on this and they realize this was unintended, and is a &quot;bug/weakness&quot; in their implementation. They are working on a patch to fix it at their end (<a href="https://github.com/openssl/openssl/pull/16725" target="_blank">https://github.com/openssl/openssl/pull/16725</a>). The eventual fix is very likely to get backported to OpenSSL 3.0, so we have to wait.</div><div class="gmail_quote"><br></div><div class="gmail_quote">I&#39;ll submit a slightly modified v2 once their fix is finalized.</div><div class="gmail_quote"><br></div><div class="gmail_quote">Thanks,</div><div class="gmail_quote"><br></div><div class="gmail_quote">Selva</div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
</blockquote></div></div>
</blockquote></div>
Gert Doering Nov. 5, 2021, 8:54 p.m. | #16
Hi,

On Tue, Nov 02, 2021 at 12:40:50AM -0400, Selva Nair wrote:
> OpenSSL folks have merged their "fix" in the provider interface that I was
> waiting for. It will be in the 3.0.1 patch release. In the meantime, I have
> opened a matching version of this patch set as a PR for OpenVPN for
> comments/tests/bug-reports/nitpicks. I skipped v2 and this version is
> tagged v3.
> 
> Will post the patches to the list when OpenSSL 3.0.1 is released.
> 
> On top of v1 patches this also includes handling pkcs11-id and
> cryptoapicert options through the provider. Requires OpenSSL from either
> the master branch (3.1.0-dev) or 3.0 branch (3.0.1-dev) post Oct. 27.

This sounds really nice.

I'll see if I can get Steffan, Max and Arne interested in looking at the
PR tomorrow :-) - face to face discussions can be some much faster than
email with asynchronous schedules...

gert