-
Oct 30th, 2020, 12:31 PM
#41
Re: Simulate TLS 1.3
Originally Posted by couttsj
Any idea what that problem might be?
The problem is that most clients request X25519 exchange group in ClientHello and you are directly replying with secp256r1 together with its public key and this is an unexpected behavior. The proper way to decline the requested exchange group is to send a HelloRetryRequest in your first ServerHello with the exchange group the server prefers.
This is what HRR for secp256r1 (sent as first ServerHello) looks in my impl:
Code:
0000 - 16 03 03 Record TLS_CONTENT_TYPE_HANDSHAKE
00 58 Record Size = 0x58
02 00 Handshake TLS_HANDSHAKE_TYPE_SERVER_HELLO
00 54 Message Size = 0x54
03 03 TLS_LOCAL_LEGACY_VERSION
CF 21 AD 74 E5
0010 - 9A 61 11 BE 1D 8C 02 1E 65 B8 91 C2 A2 11 16 7A
0020 - BB 8C 5E 07 9E 09 E2 C8 A8 33 9C HelloRetryRandom
20 Session Size = 0x20
EE 3F A6 18
0030 - 91 E4 B4 31 7A E2 B4 5B DC 41 F4 65 2A A3 C9 B0
0040 - 6A 97 F8 E8 43 1C 62 27 F0 04 B6 55 RemoteSessionID
13 01 CipherSuite = TLS_CS_AES_128_GCM_SHA256
00 Legacy Compression Method
00
0050 - 0C Extensions Size = 0x0C
00 33 Extension - Key Share
00 02 Extension Size = 0x02
00 17 ExchGroup = TLS_GROUP_SECP256R1
00 2B Extension - Supported Versions
00 02 Extension Size = 0x02
03 04 TLS_PROTOCOL_VERSION_TLS13
Note that there are two caveats. First the random is special, the so called HelloRetryRandom. You can read the specifics in the RFC but I'm using something like this
Code:
Private Sub pvTlsArrayHelloRetryRandom(baRetVal() As Byte)
pvArrayByte baRetVal, &HCF, &H21, &HAD, &H74, &HE5, &H9A, &H61, &H11, &HBE, &H1D, &H8C, &H2, &H1E, &H65, &HB8, &H91, &HC2, &HA2, &H11, &H16, &H7A, &HBB, &H8C, &H5E, &H7, &H9E, &H9, &HE2, &HC8, &HA8, &H33, &H9C
End Sub
Second one is in the Key Share when returning the exchange group (TLS_GROUP_SECP256R1 in this case) the public key is missing yet.
Otherwise your ServerHello is verbatim my second ServerHello (when the client agrees the usage of secp256r1 exchange group in its ClientHello) like this
Code:
0000 - 16 03 03 Record TLS_CONTENT_TYPE_HANDSHAKE
00 9B Record Size = 0x9B
02 00 Handshake TLS_HANDSHAKE_TYPE_SERVER_HELLO
00 97 Message Size = 0x97
03 03 TLS_LOCAL_LEGACY_VERSION
37 B4 83 A7 9A
0010 - D3 6A 7B 57 48 E1 31 A4 A2 69 09 09 BB D6 98 5B
0020 - B7 46 85 DB 96 00 56 C5 C9 20 ED LocalExchRandom
20 Session Size = 0x20
98 31 99 3C
0030 - 74 FA 16 C2 26 D5 33 16 91 EA B5 B4 D9 4C 69 B8
0040 - 6F 40 08 5E 32 6D E7 7A ED CE 9F 25 RemoteSessionID
13 01 CipherSuite = TLS_CS_AES_128_GCM_SHA256
00 Legacy Compression Method
00
0050 - 4F Extensions Size = 0x4F
00 33 Extension - Key Share
00 45 Extension Size = 0x45
00 17 ExchGroup = TLS_GROUP_SECP256R1
00 41 Public Key Size = 0x41
04 FB 5C 4E 59 DD FC
0060 - 48 34 42 BE 48 FD F8 53 62 BC 7C C7 3E 26 6A 3F
0070 - E9 21 08 06 B6 AF 4E E5 16 96 AA 7F E3 33 C4 80
0080 - 1D 0F BF 71 A7 07 C0 DF 33 27 12 56 D2 58 6A 1F
0090 - BF F9 8B 80 3E 5C B6 A2 41 58 LocalExchPublic
00 2B Extension - Supported Versions
00 02 Extension Size = 0x02
03 04 TLS_PROTOCOL_VERSION_TLS13
cheers,
</wqw>
Last edited by wqweto; Oct 30th, 2020 at 01:08 PM.
-
Oct 30th, 2020, 05:25 PM
#42
Re: Simulate TLS 1.3
Originally Posted by wqweto
The problem is that most clients request X25519 exchange group in ClientHello and you are directly replying with secp256r1 together with its public key and this is an unexpected behavior. The proper way to decline the requested exchange group is to send a HelloRetryRequest in your first ServerHello with the exchange group the server prefers.
</wqw>
FireFox sent both x25519 and secp256r1 public keys. From that I assumed (rightly or wrongly) that it was prepared to accept either. The RFC is not clear on that. Why would it send both if it was not prepared to accept both? That makes no sense at all.
J.A. Coutts
-
Oct 31st, 2020, 04:03 AM
#43
Re: Simulate TLS 1.3
Of course you are right. I was testing with chrome, curl and openssl s_client and these are openssl based and send only X25519. Of course when the client sends both groups the server can choose one and respond with local public key etc. so your ServerHello looks ok.
I suppose it's the next handshake messages that are required/missing for the handshake to complete successfully. Here is a dump of Firefox 82.0.2 handshake (and a bit of traffic) against my server impl
https://gist.github.com/wqweto/35e7b...2ebc681a99b2a2
Both groups are present in the ClientHello and just configured the server to choose secp256r1 for this particular traffic dump so there is no HRR here and the ServerHello accepts a cipher and an exchange group from the advertised in CH and sends back the local random and ephemeral public key.
Note that TlsHandshake.Output starts with unencrypted ServerHello and then switches to encrypted traffic for EncryptedExtensions, ServerCertificate, etc. messages. The plaintext content of EncryptedExtensions, ServerCertificate, ServerCertificateVerify and Finished is hexdumped just before it in pvWriteEndOfRecord.Output (unencrypted) section.
Following is the code that produces these handshake messages:
Code:
'--- Record Header
lPos = pvWriteBeginOfRecord(baOutput, lPos, TLS_CONTENT_TYPE_APPDATA, uCtx)
'--- Server Encrypted Extensions
lHandshakePos = lPos
lPos = pvWriteLong(baOutput, lPos, TLS_HANDSHAKE_TYPE_ENCRYPTED_EXTENSIONS)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=3)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=2)
If LenB(.AlpnNegotiated) <> 0 Then
lPos = pvWriteLong(baOutput, lPos, TLS_EXTENSION_TYPE_ALPN, Size:=2)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=2)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=2)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack)
lPos = pvWriteString(baOutput, lPos, .AlpnNegotiated)
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
End If
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
'--- Server Certificate
lPos = pvWriteLong(baOutput, lPos, TLS_HANDSHAKE_TYPE_CERTIFICATE)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=3)
'--- certificate request context
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack)
lPos = lPos '--- empty
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=3)
For lIdx = 1 To pvCollectionCount(.LocalCertificates)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=3)
baCert = .LocalCertificates.Item(lIdx)
lPos = pvWriteArray(baOutput, lPos, baCert)
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
'--- certificate extensions
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=2)
lPos = lPos '--- empty
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
Next
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
pvWriteBuffer .HandshakeMessages, pvArraySize(.HandshakeMessages), VarPtr(baOutput(lHandshakePos)), lPos - lHandshakePos
'--- Server Certificate Verify
lHandshakePos = lPos
lPos = pvWriteLong(baOutput, lPos, TLS_HANDSHAKE_TYPE_CERTIFICATE_VERIFY)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=3)
lPos = pvWriteLong(baOutput, lPos, .LocalSignatureType, Size:=2)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=2)
pvTlsArrayHash baHandshakeHash, .DigestAlgo, .HandshakeMessages, 0
lVerifyPos = pvWriteString(baVerifyData, 0, Space$(64) & "TLS 1.3, server CertificateVerify" & Chr$(0))
lVerifyPos = pvWriteArray(baVerifyData, lVerifyPos, baHandshakeHash)
pvTlsSignatureSign .LocalPrivateKey, .LocalSignatureType, baVerifyData, baSignature
lPos = pvWriteArray(baOutput, lPos, baSignature)
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
pvWriteBuffer .HandshakeMessages, pvArraySize(.HandshakeMessages), VarPtr(baOutput(lHandshakePos)), lPos - lHandshakePos
'--- Server Handshake Finished
lHandshakePos = lPos
lPos = pvWriteLong(baOutput, lPos, TLS_HANDSHAKE_TYPE_FINISHED)
lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=3)
pvTlsArrayHash baHandshakeHash, .DigestAlgo, .HandshakeMessages, 0
pvTlsHkdfExpandLabel baTemp, .DigestAlgo, .LocalTrafficSecret, "finished", baEmpty, .DigestSize
pvTlsHkdfExtract baVerifyData, .DigestAlgo, baTemp, baHandshakeHash
lPos = pvWriteArray(baOutput, lPos, baVerifyData)
lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack)
pvWriteBuffer .HandshakeMessages, pvArraySize(.HandshakeMessages), VarPtr(baOutput(lHandshakePos)), lPos - lHandshakePos
'--- Record Type
lPos = pvWriteLong(baOutput, lPos, TLS_CONTENT_TYPE_HANDSHAKE)
lPos = pvWriteEndOfRecord(baOutput, lPos, uCtx)
cheers,
</wqw>
Last edited by wqweto; Oct 31st, 2020 at 10:37 AM.
-
Oct 31st, 2020, 10:45 PM
#44
Re: Simulate TLS 1.3
Prompted by the packet traces that you pointed to, I decided to run my own packet traces. After receiving the Client Hello from FireFox, the Server sends it's own Server hello:
Code:
6 F8:BC:12:5E:A0:20: IP-192.168.0.4
---> E8:9E:B4:36:98:F5: IP-192.168.0.6 443 50304
0 E8:9E:B4:36:98:F5:F8:BC: 12:5E:A0:20:08:00:45:00: ...6.... .^. ..E.
16 00:C8:14:81:40:00:80:06: 64:54:C0:A8:00:04:C0:A8: ....@... dT......
32 00:06:01:BB:C4:80:49:CF: A7:79:D6:E2:B0:EA:50:18: ......I. .y....P.
48 01:00:E8:0A:00:00:
16:03: 03:00:9B:02:00:00:97:03: ........ ........
64 03:7D:96:A0:33:B2:43:04: C9:8E:5D:6E:84:93:05:AA: .}..3.C. ..]n....
80 42:64:41:24:FA:DB:CC:1B: 5C:5E:0D:2F:73:57:DB:FD: BdA$.... \^./sW..
96 CD:20:CE:DF:1E:30:F2:91: DB:FB:25:DC:13:F7:F5:E2: . ...0.. ..%.....
112 1B:FF:C5:5E:02:AA:5B:D3: BA:07:EF:4B:45:59:C0:FA: ...^..[. ...KEY..
128 34:DB:13:01:00:00:4E:00: 33:00:45:00:17:00:41:04: 4.....N. 3.E...A.
144 6C:64:75:CE:6E:C2:96:C0: 05:4C:AF:8C:21:E2:12:4F: ldu.n... .L..!..O
160 97:F8:92:FF:5D:EF:03:53: 0C:FE:83:42:04:89:7C:A1: ....]..S ...B..|.
176 67:0D:89:0F:DD:29:38:37: CF:03:78:75:D6:CD:EB:FA: g....)87 ..xu....
192 A6:AD:70:49:CF:DD:A0:17: F3:E7:27:C6:4E:24:75:0B: ..pI.... ..'.N$u.
208 00:2B:00:02:03:04: .+....
7 E8:9E:B4:36:98:F5: IP-192.168.0.6
---> F8:BC:12:5E:A0:20: IP-192.168.0.4 50304 443
0 F8:BC:12:5E:A0:20:E8:9E: B4:36:98:F5:08:00:45:00: ...^. .. .6....E.
16 00:2F:5D:F0:40:00:80:06: 1B:7E:C0:A8:00:06:C0:A8: ./].@... .~......
32 00:04:C4:80:01:BB:D6:E2: B0:EA:49:CF:A8:19:50:18: ........ ..I...P.
48 02:00:A2:73:00:00:
15:03: 01:00:02:02:32: ...s.... ....2
Fatal Alert 50 = decode_error
to which FireFox responds with a Fatal Alert 50 (decode_error). It does not like something about the Server Hello, and I am at a loss to figure out what that is.
J.A. Coutts
-
Nov 1st, 2020, 03:15 AM
#45
Re: Simulate TLS 1.3
What is this part in the hexdump?
Code:
0 E8:9E:B4:36:98:F5:F8:BC: 12:5E:A0:20:08:00:45:00: ...6.... .^. ..E.
16 00:C8:14:81:40:00:80:06: 64:54:C0:A8:00:04:C0:A8: ....@... dT......
32 00:06:01:BB:C4:80:49:CF: A7:79:D6:E2:B0:EA:50:18: ......I. .y....P.
48 01:00:E8:0A:00:00:
This part is verbatim my ServerHello
Code:
16:03: 03:00:9B:02:00:00:97:03: ........ ........
64 03:7D:96:A0:33:B2:43:04: C9:8E:5D:6E:84:93:05:AA: .}..3.C. ..]n....
80 42:64:41:24:FA:DB:CC:1B: 5C:5E:0D:2F:73:57:DB:FD: BdA$.... \^./sW..
96 CD:20:CE:DF:1E:30:F2:91: DB:FB:25:DC:13:F7:F5:E2: . ...0.. ..%.....
112 1B:FF:C5:5E:02:AA:5B:D3: BA:07:EF:4B:45:59:C0:FA: ...^..[. ...KEY..
128 34:DB:13:01:00:00:4E:00: 33:00:45:00:17:00:41:04: 4.....N. 3.E...A.
144 6C:64:75:CE:6E:C2:96:C0: 05:4C:AF:8C:21:E2:12:4F: ldu.n... .L..!..O
160 97:F8:92:FF:5D:EF:03:53: 0C:FE:83:42:04:89:7C:A1: ....]..S ...B..|.
176 67:0D:89:0F:DD:29:38:37: CF:03:78:75:D6:CD:EB:FA: g....)87 ..xu....
192 A6:AD:70:49:CF:DD:A0:17: F3:E7:27:C6:4E:24:75:0B: ..pI.... ..'.N$u.
208 00:2B:00:02:03:04: .+....
but there should not be anything before "16 03 03" sent.
cheers,
</wqw>
-
Nov 1st, 2020, 11:28 AM
#46
Re: Simulate TLS 1.3
Originally Posted by wqweto
What is this part in the hexdump?
Code:
0 E8:9E:B4:36:98:F5:F8:BC: 12:5E:A0:20:08:00:45:00: ...6.... .^. ..E.
16 00:C8:14:81:40:00:80:06: 64:54:C0:A8:00:04:C0:A8: ....@... dT......
32 00:06:01:BB:C4:80:49:CF: A7:79:D6:E2:B0:EA:50:18: ......I. .y....P.
48 01:00:E8:0A:00:00:
cheers,
</wqw>
That is the standard packet header containing such things as the source and destination MAC, IP, & port, and various packet control information. For TCP packets, the data will generally start at byte 54, and I added an extra CrLf to separate the two. On my packet trace program, I simply highlight the data.
J.A. Coutts
-
Nov 1st, 2020, 12:19 PM
#47
Re: Simulate TLS 1.3
Originally Posted by couttsj
That is the standard packet header containing such things as the source and destination MAC, IP, & port, and various packet control information. For TCP packets, the data will generally start at byte 54, and I added an extra CrLf to separate the two. On my packet trace program, I simply highlight the data.
J.A. Coutts
LOL :-)) How did you capture that!
I did the traffic logging from inside my VB6 class, not with external sniffer. . . but good, whatever floats your boat :-))
cheers,
</wqw>
-
Nov 1st, 2020, 01:45 PM
#48
Re: Simulate TLS 1.3
Originally Posted by wqweto
LOL :-)) How did you capture that!
cheers,
</wqw>
I developed my own packet trace program:
https://www.vbforums.com/showthread....cket-Ananlyzer
It uses the Windows Packet Filter Kit 3.2.3 from NT Kernel Resources. I originally developed it to deal with IPv6, but it has come in handy for dealing with any network traffic, not just TCP & UDP. It does not have any packet analysis capabilities, but I usually do that in a separate program designed for a specific purpose. It was instrumental in putting together the TLS 1.2 packet trace page (non ECC) shown here:
http://www.yellowhead.com/TLS_Handshake5.htm
J.A. Coutts
-
Nov 3rd, 2020, 01:47 AM
#49
Re: Simulate TLS 1.3
Yet another dumb mistake. When I reworked the Server Hello, I set the extension length to 00:4E instead of 00:4F.
Now all I have to do is figure out how to get FireFox to accept self signed Certificates. It was easy in previous versions of FireFox.
J.A. Coutts
-
Nov 3rd, 2020, 02:34 AM
#50
Re: Simulate TLS 1.3
Originally Posted by couttsj
. . . I set the extension length to 00:4E instead of 00:4F.
Ouch!
I was thinking of using text compare on yours vs mine hexdump, unfortunately Session ID and Server Random differ by design and the output formats we both use do not align very well.
I'm preventing errors like this by using a simple nesting stack (a glorified VBA.Collection) to automatically measure nested block size.
Something like lPos = pvWriteBeginOfBlock(baOutput, lPos, .BlocksStack, Size:=2) starts a block at lPos position, leaves 2 bytes to encode block size (number of bytes to leave for encoding block size specified by Size param) and advances lPos these 2 bytes. (The parameter .BlocksStack is a simple VBA.Collection.)
The actual block size gets written later upon lPos = pvWriteEndOfBlock(baOutput, lPos, .BlocksStack) which pops the position of the last pvWriteBeginOfBlock from stack and substracts it from current lPos to come up with actual block size. This function does *not* advance lPos but is implemented like this for compatibility with the rest of the byte-array buffer management functions which have common footprint.
Originally Posted by couttsj
Now all I have to do is figure out how to get FireFox to accept self signed Certificates. It was easy in previous versions of FireFox.
In current version it's very easy to allow a one-time ignore of certificate errors for a site and it *does* persist across browser restart, contrary to chrome which needs confirming ignore on each new session.
Just on "Warning: Potential Security Risk Ahead" page press [Advanced...] button and ignore further MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT error codes with [Accept the Risk and Continue] button.
cheers,
</wqw>
Last edited by wqweto; Nov 3rd, 2020 at 02:41 AM.
-
Nov 5th, 2020, 10:31 AM
#51
Re: Simulate TLS 1.3
Originally Posted by wqweto
In current version it's very easy to allow a one-time ignore of certificate errors for a site and it *does* persist across browser restart, contrary to chrome which needs confirming ignore on each new session.
Just on "Warning: Potential Security Risk Ahead" page press [Advanced...] button and ignore further MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT error codes with [Accept the Risk and Continue] button.
cheers,
</wqw>
The problem was that when I clicked on the "Advanced" button, there was no scroll bar presented. So I could not scroll down to accept the self signed certificate. I had to use the middle mouse scroll key to see it.
J.A. Coutts
-
Nov 9th, 2020, 02:21 PM
#52
Re: Simulate TLS 1.3
I have overcome the last obstacle in the Client program, and this was a tough one. When I attempted to send an HTML request to OpenSSL, it would return an "unexpected message" error. Not knowing what that meant, or even if this version accepted HTML requests, caused a lot of trial and error solutions and a great deal of research. I ran across one article that called the extra byte added after the data to be encrypted and before the AEAD Authentication Tag as "inner content type". A check with RFC 8446 revealed several references to the "inner content type byte", but no mention of what it was supposed to be or whether or not it was supposed to be checked. A check with RFC 8448 packet traces did not show the extra byte at all. A check with tls13.ulfheim.net showed 0x16 as the Record Type for the New Session Ticket, which was sent using the Application Key/IV. A lot of contradictory or misleading information.
So I started searching for code examples, and I found one that clearly showed Application data being sent using RT_APPLICATION_DATA (0x17). Armed with this information, I changed the inner content type to reflect the real Record type, and it worked. I had been using &H16 for all records.
Why this byte is being checked is beyond me. It is not required by the RFC. Because it is being used as a flag to separate padding from the actual data, it could be any value except zero. It serves no other practical purpose. If you are using the wrong keys to decrypt it, you won't be able to tell what it is anyway. And why New Session Tickets use RT_HANDSHAKE (0x16) when they are sent as Application Data is beyond me. Just another idiosyncrasy caused by this middle-box dilemma.
I still have a remaining problem with FireFox returning an error when receiving the Encrypted Extensions, Cerfificate, Certificate Verify, and Server Finished. I can't even tell what the error is because I can't decrypt it, but fireFox itself says that it is an Authentication error. Once I solve this problem, I will update the download.
J.A. Coutts
-
Nov 11th, 2020, 07:17 PM
#53
Re: Simulate TLS 1.3
On the off chance that FireFox would not accept the second key that they offered in the Client Hello (secp256r1), I decided to try a Hello Retry. I had to yet implement it in the Server software, so it had to be done anyway. This is what I sent:
Code:
16 03 03 00 58
02 00 00 54
03 03
CF 21 AD 74 E5 9A 61 11 BE 1D 8C 02 1E 65 B8 91
C2 A2 11 16 7A BB 8C 5E 07 9E 09 E2 C8 A8 33 9C
20
88 F3 B8 06 DE 88 A6 B8 A7 CE 45 CD 40 79 DC 45
E2 B3 0B 8D 46 A6 1C F7 66 4C 25 15 D4 5B 0D 30
13 01
00
00 0C
00 33
00 02
00 17
00 2B
00 02
03 04
for which I received a Fatal Alert 47. FireFox says that it is a malformed Hello Retry Request. Use the key that they provided and I get an authentication error. Send a Hello Retry Request, and I get a Malformed Hello Retry Request error. This is getting very frustrating.
J.A. Coutts
-
Nov 12th, 2020, 02:44 AM
#54
Re: Simulate TLS 1.3
This HRR looks verbatim to the one I posted above so something else must be the problem.
You can grab one of my TLS backends (e.g. mdTlsThunks) and use its TlsHandshake function in parallel to your implementation. These backend modules are standalone (so called sans-IO) implementation of the protocol so they don't do network I/O but expect someone to feed them input byte-arrays and send resultant output byte-arrays on their behalf.
You can compare this TlsHandshake output to yours and monitor/modify its TLS internal state stored in the UcsTlsContext user-defined type so that both TLS randoms and session IDs match. The idea is to derive identical handshake keys/IVs so that encrypted hadshake traffic must match in both parallel impl.
You can turn off X25519 support (in pvCryptoIsSupported function) for the module to fallback to secp256r1 group for the ECDHE to match bcrypt primitives you are using. I can send you a sample project with this tweak based on traffic with standard Winsock control if you think this would be useful.
cheers,
</wqw>
-
Nov 17th, 2020, 07:13 PM
#55
Re: Simulate TLS 1.3
Originally Posted by wqweto
SNI is obsolete and replaced by ALPN. I'm supporting the latter only as SNI is viable only up to TLS 1.2
cheers,
</wqw>
In the process of testing connectivity to Gmail, I removed various elements of the Client Hello one at a time in order to produce a minimum. The one element that I wanted to eliminate was the SNI, but Gmail produced an alert when I removed it. Gmail obviously doesn't consider it obsolete.
J.A. Coutts
-
Nov 18th, 2020, 01:42 AM
#56
Re: Simulate TLS 1.3
Cannot even remember what was I thinking but this is plain wrong. SNI and ALPN deliver orthogonal data -- it's like machine vs service or IP vs port.
There is some experimental effort to replace SNI with Encrypted SNI which failed mostly (only Firefox implemented it) and now there is Encrypted Client Hello draft being discussed that will protect ALPN as well.
cheers,
</wqw>
-
Nov 18th, 2020, 05:27 AM
#57
Re: Simulate TLS 1.3
Originally Posted by wqweto
I can send you a sample project with this tweak based on traffic with standard Winsock control if you think this would be useful.
Btw, just comitted BareboneTls sample which uses the TLS backends with traffic over standard MS Winsock Control.
The sample includes an https client in Form1 (~200 LOC) and a minimal https server in Form2 (~120 LOC) and both of these work with the ASM thunks based backend and the native SSPI/Schannel based one depending on if Project1.vbp or Project2.vbp is loaded in IDE. (The minimal server in Form2 generates a 1024-bit RSA self-signed certificate on startup too.)
The idea is to be able to easily put breakpoints on Winsock's SendData method and DataArrival event to dump/inspect encrypted traffic.
cheers,
</wqw>
Last edited by wqweto; Nov 19th, 2020 at 03:30 AM.
-
Nov 18th, 2020, 10:01 AM
#58
Re: Simulate TLS 1.3
First of all, thanks to you two for your ongoing efforts on this topic...
Originally Posted by wqweto
Btw, just comitted BareboneTls sample which uses the TLS backends with traffic over standard MS Winsock Control.
Interesting (and my first real contact with vbAsyncSocket and this TLS-implementation).
I have a few questions...:
1) I take it, that the MS-Winsock control was used "to keep the two Test-Projects simple"
... (and that your vbAsyncSocket could've been used in a stable manner as well)?
2) And that with "native" (Project2) you mean the "direct use of the MS-Crypto-APIs"...?
3)... and that Project1 works "MS-dependency-free" via thunks (derived from libSodium)...?
4) If 3) is a "Yes"... is there any sub-dependencies behind libSodium, which I'm not aware of (OpenSSL and stuff) -
... (or does libSodium support a compile-switch, which makes it work with via the Crypto-API of the given OS internally)?
BackGround:
I plan to make the cTCP(Client/Server) classes IP-v6 capable in the next weeks (for a newer RC6-release)...
And at this occasion (in case you say, your TLS-implementation is already "mature enough for TLS 1.3 - and the versions below it") -
I'd like to implement also a WinHttp51 compatible http(s) Class... which (under the covers) works "as platform-independent as possible"
(libSodium then statically linked into my accompanying flat-dll - accessed without thunking).
So I'm basically asking for your "do or don't" (or "wait a few weeks/months").
Olaf
-
Nov 18th, 2020, 11:52 AM
#59
Re: Simulate TLS 1.3
Originally Posted by Schmidt
1) I take it, that the MS-Winsock control was used "to keep the two Test-Projects simple"
... (and that your vbAsyncSocket could've been used in a stable manner as well)?
Yes. The TLS "backends" are written in Sans-IO manner so that the actual networking can be done with whatever socket library anyone would like to use.
Originally Posted by Schmidt
2) And that with "native" (Project2) you mean the "direct use of the MS-Crypto-APIs"...?
Yes. Newer CNG calls are kept to a minimum so this works on Win2000 and NT 4.0 too.
Originally Posted by Schmidt
3)... and that Project1 works "MS-dependency-free" via thunks (derived from libSodium)...?
Yes, but the libsodium dependency is optional (and not very useful) so this is 100% dependency free.
Originally Posted by Schmidt
4) If 3) is a "Yes"... is there any sub-dependencies behind libSodium, which I'm not aware of (OpenSSL and stuff) -
... (or does libSodium support a compile-switch, which makes it work with via the Crypto-API of the given OS internally)?
No, the thunks are *not* based on libsodum but mostly on cifra and putty (for AES-NI accelerated symmetric ciphers) because these are more succinct sources targeting embedded projects and are easier to convert to position independent code (aka "to thunk":-)). The thunks are overall 33477 bytes of code total with 1952 bytes static tables.
The original project started with stdcall compiled libsodium and "native" crypto API calls but it soon turned out libsodium is missing a lot of primitives for TLS. The idea of libsodium is to be very opinionated and to include only crypto primitives that its author considers strong enough and which he can implement in production manner -- fast and with optimized support for x86_64 and ARM w/ intrinsics and so on.
That is why there is no AES-128 included (just use AES-256) and no CBC mode (just use GCM) in libsodium. Also there is no ECC (ellipic curves) in libsodium at all, so ECDHE is no go with it.
Originally Posted by Schmidt
BackGround:
I plan to make the cTCP(Client/Server) classes IP-v6 capable in the next weeks (for a newer RC6-release)...
And at this occasion (in case you say, your TLS-implementation is already "mature enough for TLS 1.3 - and the versions below it") -
I'd like to implement also a WinHttp51 compatible http(s) Class... which (under the covers) works "as platform-independent as possible"
(libSodium then statically linked into my accompanying flat-dll - accessed without thunking).
This cannot be done with libsodium only. For complete TLS support you will need *various* libs/sources for the crypto primitives. I have them thunked so my C/C++ sources are not good for what you plan to do, although they are complete (meaning they include enough primitives for a TLS backend).
Frankly, I cannot give you good advice here. Thunks raise false positives with AV software so could be a real PITA, better not to include any in RC6.
You cannot grab the TLS module as is but you'll need to convert the thunks to a DLL and tweak the TLS backend to use crypto primitives from this DLL which is not a lot of work if DLL is compiled from my C/C++ sources and match function prototypes, I guess. . .
cheers,
</wqw>
Last edited by wqweto; Nov 18th, 2020 at 02:49 PM.
-
Dec 7th, 2020, 04:48 PM
#60
Re: Simulate TLS 1.3
I am in the process of streamlining the client code, but a question has come to mind about the server code. The client has to complete the handshake and retain the derived keys until it is disconnected. The server can complete handshakes one at a time, but because it services multiple connections, I assume that it must retain multiple sets of the derived keys, but not the intermediate secrets
I ask the question, but I don't see any way around it.
J.A. Coutts
-
Dec 8th, 2020, 03:38 AM
#61
Re: Simulate TLS 1.3
Originally Posted by couttsj
I ask the question, but I don't see any way around it.
Why? You can have several instances of the server class each with it's own socket, protocol state and derived keys -- generally all the internal object state is separate.
On accepting a new client request the listener has to spawn a new instance of the server class and initialize it with the accepted socket and server certficate + key (at least).
cheers,
</wqw>
-
Dec 8th, 2020, 07:15 PM
#62
Re: Simulate TLS 1.3
Originally Posted by wqweto
Why? You can have several instances of the server class each with it's own socket, protocol state and derived keys -- generally all the internal object state is separate.
On accepting a new client request the listener has to spawn a new instance of the server class and initialize it with the accepted socket and server certficate + key (at least).
cheers,
</wqw>
I was trying to determine if the system used different Public/Private keys when calculating the Agreed Secret for each connection. The server software worked fine on the first connection, but failed miserably on the second connection. Tracing the problem revealed that some of the variables I was using were shared between instances. I have separated the HKDF and Crypto functions into 2 classes, but the 2 classes cannot communicate with each other (which is understandable). I am going to have to rethink this whole server software, and I may have to put it all into a single class.
J.A. Coutts
-
Dec 10th, 2020, 06:52 AM
#63
Re: Simulate TLS 1.3
@Olaf: I just comitted mdTlsSodium.bas to VbAsyncSocket repo which is a libsodium based backend with no thunks of any kind. You can check out the sample Project3.vbp in test/BareboneTls directory for a Winsock based TLS implementation.
The module is 88-100KB compiled (depending on compile options) and you can set ASYNCSOCKET_NO_TLSSERVER = 1 in condition compilation settings to shave off another 24KB if no server-side TLS is needed (and no client certificates) for as little as 64KB executable size. Speaking of server-side implementation, unfortunately this cannot be done with libsodium.dll crypto primitives only so the mdTlsSodium backend uses native OS provided BCrypt calls for both RSA and ECC certificate signatures which means XP is not supported (for server-side TLS only).
Another caveat is that even though libsodium.dll provided in the repo is compiled w/ XP compatibility SDK it does not work on Win2000 and earlier (contrary to thunks which work fine on NT 4.0). Also notice that from the complete libsodium.dll only a handful of exports are used by the backend so IMO this dependency can be reduced to less than 50KB if some effort is spent there.
Overall the mdTlsSodium backend does not support CBC ciphers in TLS 1.2 (only AES256-GCM and Chacha20) and NIST curves for ECDHE (only X25519) which is non fatal in 95% of the connections to public servers on the internet. Some TLS servers that do not work are ecc256.badssl.com and ecc384.badssl.com which are specially configured to accept only NIST curves for key exchange in TLS 1.2 while in TLS 1.3 specification support for X25519 is mandatory so this cannot happen.
Another exception is websocket.org which is configured to use AES in CBC mode as a single cipher for TLS 1.2 and secure connections cannot be negotiated with this mdTlsSodium backend. My estimate is that the new backend is a 100% compatible TLS 1.3 client and about 90% compatible TLS 1.2 client.
cheers,
</wqw>
Last edited by wqweto; Dec 10th, 2020 at 08:58 AM.
-
Dec 29th, 2020, 05:40 PM
#64
Re: Simulate TLS 1.3
@couttsj: Here is a Curve25519 scalar multiplication impl in pure VB6. This can be used for X25519 with sub Curve25519ScalarMulBase to generate priv/pub key pair and sub Curve25519ScalarMultiply to calculate the shared secret.
Code:
'--- mdCurve25519.bas
Option Explicit
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (Destination As Any, Source As Any, ByVal Length As Long)
Private Declare Function VariantChangeType Lib "oleaut32" (Dest As Variant, src As Variant, ByVal wFlags As Integer, ByVal vt As Long) As Long
Private Const LNG_CURVE25519_GFSZ As Long = 16
Private Const LNG_CURVE25519_KEYSZ As Long = 32
Private Const LNG_POW2_16 As Long = 2 ^ 16
Private Type UcsGaloisField
Item(0 To LNG_CURVE25519_GFSZ - 1) As Variant
End Type
Private Function CLngLng(vValue As Variant) As Variant
Const VT_I8 As Long = &H14
Call VariantChangeType(CLngLng, vValue, 0, VT_I8)
End Function
Private Sub pvCurve25519Sel(uA As UcsGaloisField, uB As UcsGaloisField, ByVal bSwap As Boolean)
Dim lIdx As Long
Dim vTemp As Variant
For lIdx = 0 To LNG_CURVE25519_GFSZ - 1
vTemp = (uA.Item(lIdx) Xor uB.Item(lIdx)) And bSwap
uA.Item(lIdx) = uA.Item(lIdx) Xor vTemp
uB.Item(lIdx) = uB.Item(lIdx) Xor vTemp
Next
End Sub
Private Sub pvCurve25519Car(uRetVal As UcsGaloisField)
Dim lIdx As Long
Dim vCarry As Variant
Dim lNext As Long
For lIdx = 0 To LNG_CURVE25519_GFSZ - 1
uRetVal.Item(lIdx) = uRetVal.Item(lIdx) + LNG_POW2_16
vCarry = uRetVal.Item(lIdx) \ LNG_POW2_16
uRetVal.Item(lIdx) = uRetVal.Item(lIdx) - vCarry * LNG_POW2_16
If lIdx = LNG_CURVE25519_GFSZ - 1 Then
vCarry = vCarry - 1 + 37 * (vCarry - 1)
Else
vCarry = vCarry - 1
End If
lNext = (lIdx + 1) Mod LNG_CURVE25519_GFSZ
uRetVal.Item(lNext) = uRetVal.Item(lNext) + vCarry
Next
End Sub
Private Sub pvCurve25519Add(uRetVal As UcsGaloisField, uA As UcsGaloisField, uB As UcsGaloisField)
Dim lIdx As Long
For lIdx = 0 To LNG_CURVE25519_GFSZ - 1
uRetVal.Item(lIdx) = uA.Item(lIdx) + uB.Item(lIdx)
Next
End Sub
Private Sub pvCurve25519Sub(uRetVal As UcsGaloisField, uA As UcsGaloisField, uB As UcsGaloisField)
Dim lIdx As Long
For lIdx = 0 To LNG_CURVE25519_GFSZ - 1
uRetVal.Item(lIdx) = uA.Item(lIdx) - uB.Item(lIdx)
Next
End Sub
Private Sub pvCurve25519Mul(uRetVal As UcsGaloisField, uA As UcsGaloisField, uB As UcsGaloisField)
Dim aTemp(0 To LNG_CURVE25519_GFSZ * 2 - 1) As Variant
Dim lIdx As Long
Dim lJdx As Long
For lIdx = 0 To UBound(aTemp)
aTemp(lIdx) = CLngLng(0)
Next
For lIdx = 0 To LNG_CURVE25519_GFSZ - 1
For lJdx = 0 To LNG_CURVE25519_GFSZ - 1
aTemp(lIdx + lJdx) = aTemp(lIdx + lJdx) + uA.Item(lIdx) * uB.Item(lJdx)
Next
Next
For lIdx = 0 To LNG_CURVE25519_GFSZ - 1
If lIdx < LNG_CURVE25519_GFSZ - 1 Then
uRetVal.Item(lIdx) = aTemp(lIdx) + 38 * aTemp(lIdx + LNG_CURVE25519_GFSZ)
Else
uRetVal.Item(lIdx) = aTemp(lIdx)
End If
Next
pvCurve25519Car uRetVal
pvCurve25519Car uRetVal
End Sub
Private Sub pvCurve25519Inv(uA As UcsGaloisField)
Dim uTemp As UcsGaloisField
Dim lIdx As Long
uTemp = uA
For lIdx = 253 To 0 Step -1
pvCurve25519Mul uA, uA, uA
If lIdx <> 2 And lIdx <> 4 Then
pvCurve25519Mul uA, uA, uTemp
End If
Next
End Sub
Private Function pvCurve25519ToGF(baInput() As Byte) As UcsGaloisField
Dim uRetVal As UcsGaloisField
Dim aTemp(0 To LNG_CURVE25519_GFSZ - 1) As Integer
Dim lIdx As Long
If UBound(baInput) >= 0 Then
Debug.Assert (UBound(aTemp) + 1) * 2 >= UBound(baInput) + 1
Call CopyMemory(aTemp(0), baInput(0), UBound(baInput) + 1)
End If
For lIdx = 0 To LNG_CURVE25519_GFSZ - 1
If aTemp(lIdx) < 0 Then
uRetVal.Item(lIdx) = CLngLng(&H10000) + aTemp(lIdx)
Else
uRetVal.Item(lIdx) = CLngLng(aTemp(lIdx))
End If
Next
pvCurve25519ToGF = uRetVal
End Function
Private Function pvCurve25519FromGF(uA As UcsGaloisField) As Byte()
Dim lRetry As Long
Dim lIdx As Long
Dim uTemp As UcsGaloisField
Dim lFlag As Long
Dim baRetVal(0 To LNG_CURVE25519_KEYSZ - 1) As Byte
For lRetry = 0 To 1
uTemp.Item(0) = uA.Item(0) - &HFFED&
For lIdx = 1 To LNG_CURVE25519_GFSZ - 1
lFlag = -((uTemp.Item(lIdx - 1) And LNG_POW2_16) <> 0)
If lIdx = LNG_CURVE25519_GFSZ - 1 Then
lFlag = &H7FFF& + lFlag
Else
lFlag = &HFFFF& + lFlag
End If
uTemp.Item(lIdx) = uA.Item(lIdx) - lFlag
uTemp.Item(lIdx - 1) = uTemp.Item(lIdx - 1) And &HFFFF&
Next
lFlag = -((uTemp.Item(LNG_CURVE25519_GFSZ - 1) And LNG_POW2_16) <> 0)
pvCurve25519Sel uA, uTemp, lFlag = 0
Next
For lIdx = 0 To LNG_CURVE25519_GFSZ - 1
lFlag = uA.Item(lIdx) And (LNG_POW2_16 - 1)
Call CopyMemory(baRetVal(2 * lIdx), lFlag, 2)
Next
pvCurve25519FromGF = baRetVal
End Function
Public Sub Curve25519ScalarMultiply(baRetVal() As Byte, baPriv() As Byte, baPub() As Byte)
Static LNG_POW2(0 To 7) As Long
Static EmptyByteArray() As Byte
Dim baKey() As Byte
Dim uX As UcsGaloisField
Dim uA As UcsGaloisField
Dim uB As UcsGaloisField
Dim uC As UcsGaloisField
Dim uD As UcsGaloisField
Dim uE As UcsGaloisField
Dim uF As UcsGaloisField
Dim uG As UcsGaloisField
Dim lIdx As Long
Dim lFlag As Long
Dim lPrev As Long
If LNG_POW2(0) = 0 Then
For lIdx = 0 To UBound(LNG_POW2)
LNG_POW2(lIdx) = 2 ^ lIdx
Next
EmptyByteArray = vbNullString
End If
baKey = baPriv
baKey(0) = baKey(0) And &HF8
baKey(31) = baKey(31) And &H7F Or &H40
uA = pvCurve25519ToGF(EmptyByteArray)
uX = pvCurve25519ToGF(baPub)
uB = uX
uC = uA
uD = uA
uG = uA
uG.Item(0) = uG.Item(0) + &HDB41&
uG.Item(1) = uG.Item(1) + 1
uA.Item(0) = uG.Item(1) ' a[0] = 1
uD.Item(0) = uG.Item(1) ' d[0] = 1
For lIdx = 254 To 0 Step -1
lPrev = lFlag
lFlag = (baKey(lIdx \ 8) \ LNG_POW2(lIdx And 7)) And 1
pvCurve25519Sel uA, uB, lFlag Xor lPrev
pvCurve25519Sel uC, uD, lFlag Xor lPrev
pvCurve25519Add uE, uA, uC ' e = a + c
pvCurve25519Sub uA, uA, uC ' a = a - c
pvCurve25519Add uC, uB, uD ' c = b + d
pvCurve25519Sub uB, uB, uD ' b = b - d
pvCurve25519Mul uD, uE, uE ' d = e * e
pvCurve25519Mul uF, uA, uA ' f = a * a
pvCurve25519Mul uA, uC, uA ' a = c * a
pvCurve25519Mul uC, uB, uE ' c = b * e
pvCurve25519Add uE, uA, uC ' e = a + c
pvCurve25519Sub uA, uA, uC ' a = a - c
pvCurve25519Mul uB, uA, uA ' b = a * a
pvCurve25519Sub uC, uD, uF ' c = d - f
pvCurve25519Mul uA, uC, uG ' a = c * g
pvCurve25519Add uA, uA, uD ' a = a + d
pvCurve25519Mul uC, uC, uA ' c = c * a
pvCurve25519Mul uA, uD, uF ' a = d * f
pvCurve25519Mul uD, uB, uX ' d = b * x
pvCurve25519Mul uB, uE, uE ' b = e * e
Next
pvCurve25519Inv uC
pvCurve25519Mul uX, uA, uC
baRetVal = pvCurve25519FromGF(uX)
End Sub
Public Sub Curve25519ScalarMulBase(baRetVal() As Byte, baPriv() As Byte)
Dim baBase(0 To LNG_CURVE25519_KEYSZ - 1) As Byte
baBase(0) = 9
Curve25519ScalarMultiply baRetVal, baPriv, baBase
End Sub
For instance here is the X25519 impl in my libsodium TLS backend
Code:
Private Function pvCryptoEcdhCurve25519MakeKey(baPrivate() As Byte, baPublic() As Byte) As Boolean
Const FUNC_NAME As String = "CryptoEccCurve25519MakeKey"
pvArrayAllocate baPrivate, LNG_X25519_KEYSZ, FUNC_NAME & ".baPrivate"
pvArrayAllocate baPublic, LNG_X25519_KEYSZ, FUNC_NAME & ".baPublic"
pvCryptoRandomBytes VarPtr(baPrivate(0)), LNG_X25519_KEYSZ
'--- fix privkey randomness
baPrivate(0) = baPrivate(0) And 248
baPrivate(UBound(baPrivate)) = (baPrivate(UBound(baPrivate)) And 127) Or 64
Call crypto_scalarmult_curve25519_base(baPublic(0), baPrivate(0))
'--- success
pvCryptoEcdhCurve25519MakeKey = True
End Function
Private Function pvCryptoEcdhCurve25519SharedSecret(baRetVal() As Byte, baPrivate() As Byte, baPublic() As Byte) As Boolean
Const FUNC_NAME As String = "CryptoEccCurve25519SharedSecret"
Debug.Assert UBound(baPrivate) >= LNG_X25519_KEYSZ - 1
Debug.Assert UBound(baPublic) >= LNG_X25519_KEYSZ - 1
pvArrayAllocate baRetVal, LNG_X25519_KEYSZ, FUNC_NAME & ".baRetVal"
Call crypto_scalarmult_curve25519(baRetVal(0), baPrivate(0), baPublic(0))
'--- success
pvCryptoEcdhCurve25519SharedSecret = True
End Function
Note that output/retval byte-arrays do not need to be pre-allocated for the Curve25519 subs as impl in mdCurve25519 module above.
Also note that "fix privkey randomness" is not needed as sub Curve25519ScalarMultiply takes care of out-of-bounds private key itself so any random 32 bytes for priv key will do the job with no "fixing".
cheers,
</wqw>
Last edited by wqweto; Dec 29th, 2020 at 05:48 PM.
-
Dec 30th, 2020, 02:35 AM
#65
Re: Simulate TLS 1.3
Originally Posted by wqweto
@couttsj: Here is a Curve25519 scalar multiplication impl in pure VB6. This can be used for X25519 with sub Curve25519ScalarMulBase to generate priv/pub key pair and sub Curve25519ScalarMultiply to calculate the shared secret.
cheers,
</wqw>
Very Interesting. I tested it against RFC 8448, and it verified. I should be able to add x25519 to the simulation now.
J.A. Coutts
Code:
Public Key:
99 38 1D E5 60 E4 BD 43 D2 3D 8E 43 5A 7D BA FE
B3 C0 6E 51 C1 3C AE 4D 54 13 69 1E 52 9A AF 2C
Private Key:
B1 58 0E EA DF 6D D5 89 B8 EF 4F 2D 56 52 57 8C
C8 10 E9 98 01 91 EC 8D 05 83 08 CE A2 16 A2 1E
Agreed Secret:
8B D4 05 4F B5 5B 9D 63 FD FB AC F9 F0 4B 9F 0D
35 E6 D6 3F 53 75 63 EF D4 62 72 90 0F 89 49 2D
Public Key:
C9 82 88 76 11 20 95 FE 66 76 2B DB F7 C6 72 E1
56 D6 CC 25 3B 83 3D F1 DD 69 B1 B0 4E 75 1F 0F
Private Key:
49 AF 42 BA 7F 79 94 85 2D 71 3E F2 78 4B CB CA
A7 91 1D E2 6A DC 56 42 CB 63 45 40 E7 EA 50 05
Agreed Secret:
8B D4 05 4F B5 5B 9D 63 FD FB AC F9 F0 4B 9F 0D
35 E6 D6 3F 53 75 63 EF D4 62 72 90 0F 89 49 2D
-
Dec 30th, 2020, 09:20 AM
#66
Re: Simulate TLS 1.3
I just did some quick benchmarking of mdCurve25519 above and in compiled executable with all optimizations turned on, for 100 multiplications it take 5.212 seconds while for the C/C++ version in release build the same takes 0.434 seconds which is about 12x slower for the VB6 impl.
This can be easily explained by the usage of VT_I8 type of Variants for the 64-bit integer arithmethics. I'll try some more tweaks, probably benchmark x64 VBA with its native LongLong data-type for the internal Galois field members.
cheers,
</wqw>
-
Dec 31st, 2020, 06:52 AM
#67
Re: Simulate TLS 1.3
Btw, I just pushed native BCrypt impl of X25519 in commit 1e53ee7 but this support for ECC curves seems like a Win10 only feature, not sure if this is backported to Win8.1 and earlier versions.
The gist of it is something like this (key pair generation, error handling omitted)
Code:
hResult = BCryptOpenAlgorithmProvider(hAlgEcdh, StrPtr("ECDH"), 0, 0)
hResult = BCryptSetProperty(hAlgEcdh, StrPtr("ECCCurveName"), StrPtr("Curve25519"), 22, 0) '--- 22 = LenB("Curve25519") + 2
'--- note: use 0 for key size or API fails
hResult = BCryptGenerateKeyPair(hAlgEcdh, hKeyPair, 0, 0)
hResult = BCryptFinalizeKeyPair(hKeyPair, 0)
hResult = BCryptExportKey(hKeyPair, 0, StrPtr("ECCPRIVATEBLOB"), VarPtr(baBlob(0)), UBound(baBlob) + 1, cbResult, 0)
Call pvCryptoEcdhFromKeyBlob(baPriv, baBlob, cbResult)
hResult = BCryptExportKey(hKeyPair, 0, StrPtr("ECCPUBLICBLOB"), VarPtr(baBlob(0)), UBound(baBlob) + 1, cbResult, 0)
Call pvCryptoEcdhFromKeyBlob(baPub, baBlob, cbResult)
There is a new "ECDH" provider to which an "ECCCurveName" parameter can be subsequently set. This parameter supports names like "secP256r1" too and all other curves from Win7 previously supported under "ECDH_P256" etc. separate providers.
Here is a dump of all supported named curves
Code:
c:>certutil -displayEccCurve
Microsoft SSL Protocol Provider:
--------------------------------
Curve Name Curve OID Public Key Length CurveType EccCurveFlags
-----------------------------------------------------------------------------------------------
curve25519 255 29 0xa
nistP256 1.2.840.10045.3.1.7 256 23 0x7
nistP384 1.3.132.0.34 384 24 0x7
brainpoolP256r1 1.3.36.3.3.2.8.1.1.7 256 26 0x7
brainpoolP384r1 1.3.36.3.3.2.8.1.1.11 384 27 0x7
brainpoolP512r1 1.3.36.3.3.2.8.1.1.13 512 28 0x7
nistP192 1.2.840.10045.3.1.1 192 19 0x7
nistP224 1.3.132.0.33 224 21 0x7
nistP521 1.3.132.0.35 521 25 0x7
secP160k1 1.3.132.0.9 160 15 0x7
secP160r1 1.3.132.0.8 160 16 0x7
secP160r2 1.3.132.0.30 160 17 0x7
secP192k1 1.3.132.0.31 192 18 0x7
secP192r1 1.2.840.10045.3.1.1 192 19 0x7
secP224k1 1.3.132.0.32 224 20 0x7
secP224r1 1.3.132.0.33 224 21 0x7
secP256k1 1.3.132.0.10 256 22 0x7
secP256r1 1.2.840.10045.3.1.7 256 23 0x7
secP384r1 1.3.132.0.34 384 24 0x7
secP521r1 1.3.132.0.35 521 25 0x7
CNG Curves:
-----------
Curve Name Curve OID Public Key Length
-------------------------------------------------------------------
brainpoolP160r1 1.3.36.3.3.2.8.1.1.1 160
brainpoolP160t1 1.3.36.3.3.2.8.1.1.2 160
brainpoolP192r1 1.3.36.3.3.2.8.1.1.3 192
brainpoolP192t1 1.3.36.3.3.2.8.1.1.4 192
brainpoolP224r1 1.3.36.3.3.2.8.1.1.5 224
brainpoolP224t1 1.3.36.3.3.2.8.1.1.6 224
brainpoolP256r1 1.3.36.3.3.2.8.1.1.7 256
brainpoolP256t1 1.3.36.3.3.2.8.1.1.8 256
brainpoolP320r1 1.3.36.3.3.2.8.1.1.9 320
brainpoolP320t1 1.3.36.3.3.2.8.1.1.10 320
brainpoolP384r1 1.3.36.3.3.2.8.1.1.11 384
brainpoolP384t1 1.3.36.3.3.2.8.1.1.12 384
brainpoolP512r1 1.3.36.3.3.2.8.1.1.13 512
brainpoolP512t1 1.3.36.3.3.2.8.1.1.14 512
curve25519 255
ec192wapi 1.2.156.11235.1.1.2.1 192
nistP192 1.2.840.10045.3.1.1 192
nistP224 1.3.132.0.33 224
nistP256 1.2.840.10045.3.1.7 256
nistP384 1.3.132.0.34 384
nistP521 1.3.132.0.35 521
numsP256t1 256
numsP384t1 384
numsP512t1 512
secP160k1 1.3.132.0.9 160
secP160r1 1.3.132.0.8 160
secP160r2 1.3.132.0.30 160
secP192k1 1.3.132.0.31 192
secP192r1 1.2.840.10045.3.1.1 192
secP224k1 1.3.132.0.32 224
secP224r1 1.3.132.0.33 224
secP256k1 1.3.132.0.10 256
secP256r1 1.2.840.10045.3.1.7 256
secP384r1 1.3.132.0.34 384
secP521r1 1.3.132.0.35 521
wtls7 1.3.132.0.30 160
wtls9 2.23.43.1.4.9 160
wtls12 1.3.132.0.33 224
x962P192v1 1.2.840.10045.3.1.1 192
x962P192v2 1.2.840.10045.3.1.2 192
x962P192v3 1.2.840.10045.3.1.3 192
x962P239v1 1.2.840.10045.3.1.4 239
x962P239v2 1.2.840.10045.3.1.5 239
x962P239v3 1.2.840.10045.3.1.6 239
x962P256v1 1.2.840.10045.3.1.7 256
CertUtil: -displayEccCurve command completed successfully.
cheers,
</wqw>
-
Jan 1st, 2021, 12:17 PM
#68
Re: Simulate TLS 1.3
Originally Posted by wqweto
Btw, I just pushed native BCrypt impl of X25519 in commit 1e53ee7 but this support for ECC curves seems like a Win10 only feature, not sure if this is backported to Win8.1 and earlier versions.
cheers,
</wqw>
Ya, I had looked at this some time ago, and it is Win 10 only. First call returns error &HC0000225.
J.A. Coutts
-
Jan 3rd, 2021, 03:08 PM
#69
Re: Simulate TLS 1.3
Originally Posted by wqweto
I just did some quick benchmarking of mdCurve25519 above and in compiled executable with all optimizations turned on, for 100 multiplications it take 5.212 seconds while for the C/C++ version in release build the same takes 0.434 seconds which is about 12x slower for the VB6 impl.
This can be easily explained by the usage of VT_I8 type of Variants for the 64-bit integer arithmethics. I'll try some more tweaks, probably benchmark x64 VBA with its native LongLong data-type for the internal Galois field members.
cheers,
</wqw>
I did notice a slight hesitation when I ran this, so I timed it. One cycle would take 250 ms in the IDE, and 141 ms compiled. The part that I found interesting was that times were fairly consistent each time I ran it. Normally there is a fairly wide variance and the times are much shorter. I speculate this is due to the large number of loops that have to be setup and run.
Is this functionality available in a C++ compiled DLL or OCX?
J.A. Coutts
-
Jan 4th, 2021, 02:00 AM
#70
Re: Simulate TLS 1.3
Originally Posted by couttsj
The part that I found interesting was that times were fairly consistent each time I ran it.
The implementation is side-channel resistant, meaning it's constant time no matter the input. For instance pvCurve25519Sel is used to swaps contents of its first two arguments and becomes a noop if bSwap is false, but it still executes the same amount of XORs as not to leak timing information that can be used for side-channel attacks.
Originally Posted by couttsj
Is this functionality available in a C++ compiled DLL or OCX?
It's implemented in openssl DLL for sure. I have it thunked from cifra sources and saw the same code in tinycrypt only shorter. Not sure for any other DLL/OCX.
Edit: This impl seems to originate from tweetnacl. Here is an explanation of its code: https://tweetnacl.cr.yp.to/tweetnacl-20131229.pdf (the final 5 Curve25519 and Ed25519 chapter is relevant)
cheers,
</wqw>
Last edited by wqweto; Jan 4th, 2021 at 02:15 AM.
-
Jan 10th, 2021, 10:30 PM
#71
Re: Simulate TLS 1.3
Originally Posted by couttsj
On the off chance that FireFox would not accept the second key that they offered in the Client Hello (secp256r1), I decided to try a Hello Retry. I had to yet implement it in the Server software, so it had to be done anyway. This is what I sent:
for which I received a Fatal Alert 47. FireFox says that it is a malformed Hello Retry Request. Use the key that they provided and I get an authentication error. Send a Hello Retry Request, and I get a Malformed Hello Retry Request error. This is getting very frustrating.
J.A. Coutts
wqweto;
I added mdCurve25519.bas to a server program and tested x25519 with a modified client program (not a simple task). Then I tested the server program with FireFox. Low and behold it actually completed the handshake, but FireFox came back with SEC_ERROR_BAD_SIGNATURE. I used the same Certificate and Certificate Verify as OpenSSL used. Any idea what FireFox doesn't like now?
J.A. Coutts
-
Jan 11th, 2021, 02:53 AM
#72
Re: Simulate TLS 1.3
Originally Posted by couttsj
Then I tested the server program with FireFox. Low and behold it actually completed the handshake, but FireFox came back with SEC_ERROR_BAD_SIGNATURE.
Do you use self-signed (RSA) server certificate or read it from PEM/PFX files or use system certificate store?
Producing signatures for authenticating the peer certificate is obviously always required for server-side TLS implementation and optional for client-side (only when autheticating with a client-side certificate) and I spent significant effort implementing these in my TLS backends.
For RSA certificates you will need RSA-PSS signatures and for ECC certificates you need separate ECDSA signatures implementation.
Both links above use CNG's BCryptSignHash API on Win10 but I'm not sure if the impl will work on Win8.1 properly (ECDSA should work on Win7+, not sure about BCRYPT_PAD_PSS flag).
cheers,
</wqw>
-
Jan 11th, 2021, 11:49 AM
#73
Re: Simulate TLS 1.3
Originally Posted by wqweto
Do you use self-signed (RSA) server certificate or read it from PEM/PFX files or use system certificate store?
Producing signatures for authenticating the peer certificate is obviously always required for server-side TLS implementation and optional for client-side (only when autheticating with a client-side certificate) and I spent significant effort implementing these in my TLS backends.
For RSA certificates you will need RSA-PSS signatures and for ECC certificates you need separate ECDSA signatures implementation.
Both links above use CNG's BCryptSignHash API on Win10 but I'm not sure if the impl will work on Win8.1 properly (ECDSA should work on Win7+, not sure about BCRYPT_PAD_PSS flag).
cheers,
</wqw>
I simply used the Certificate and Certificate Verify used by OpenSSL.exe:
Code:
0F 00 00 4A
04 03 - Signature scheme (ECDSA SECP256r1 SHA 256)
00 46 - Length (70)
30 44 02 20 09 3F 64 62 A9 A6 37 33 52 CB C7 B3
B8 83 51 EF E6 F9 C0 89 93 DC 14 93 9E 3F 1B 39
6B 0D E1 3C 02 20 27 B2 D9 BD EA 32 8D 10 6D 2E
FB EC 00 A4 9A 28 3B E3 A4 45 E6 63 D5 17 1E A5
44 5B B4 E6 62 4E
When FireFox connected to OpenSSL, it said that it was self-signed, and I had to allow it. If it worked for OpenSSL, why would it not work for my server on the same PC? My knowledge of Certificates is very limited.
J.A. Coutts
-
Jan 11th, 2021, 12:27 PM
#74
Re: Simulate TLS 1.3
This will not work as CertificateVerify is based on handshake transmitted hash i.e. depends on all messages exchanged between the peers incl. the initial peer randoms, so that this is designed to prevent replay attacks.
Otherwise it would be trivial to auth with any google’s server certificate by replaying this pair of messages as you try to. The VerifyData that is signed has to include current handshake hash.
-
Jan 11th, 2021, 06:54 PM
#75
Re: Simulate TLS 1.3
Originally Posted by wqweto
This will not work as CertificateVerify is based on handshake transmitted hash i.e. depends on all messages exchanged between the peers incl. the initial peer randoms, so that this is designed to prevent replay attacks.
Otherwise it would be trivial to auth with any google’s server certificate by replaying this pair of messages as you try to. The VerifyData that is signed has to include current handshake hash.
That makes sense. I assumed (incorrectly) that the encryption provided by TLS 1.3 would provide that protection. But I neglected to include the provisions of previous versions. By the way. the original problem with FireFox was that I was not handling the dual KeyShare properly. I assumed that my server program would handle both public keys, but that was not the case.
J.A. Coutts
-
Jan 17th, 2021, 06:16 PM
#76
Re: Simulate TLS 1.3
Originally Posted by wqweto
This will not work as CertificateVerify is based on handshake transmitted hash i.e. depends on all messages exchanged between the peers incl. the initial peer randoms, so that this is designed to prevent replay attacks.
Otherwise it would be trivial to auth with any google’s server certificate by replaying this pair of messages as you try to. The VerifyData that is signed has to include current handshake hash.
This is certainly different from any signatures that I have dealt with in the past, and I am trying to duplicate what OpenSSL produced. I got the format from RFC 8446:
Code:
20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
54 4c 53 20 31 2e 33 2c 20 73 65 72 76 65 72 20
43 65 72 74 69 66 69 63 61 74 65 56 65 72 69 66
79
00
01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01
01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01
where the "01" bytes represent the Session Hash for SHA-256. I assumed the Private key used to sign was contained in "ecprivkey.pem", and the Public key used to verify was contained in the Certificate itself. I extracted these and captured the Session Hash and assembled the array. Then I used BCryptImportKeyPair to get the handle to the Private key and BCryptSignHash to get the signature.
The first problem was that the "No Compression" (&H04) byte in the Private key had to be removed for BCryptSignHash to work at all. The second problem is that the signature I produce is 64 bytes, and the one that OpenSSL produces is 71 bytes. And the third thing is that the signature is different every time I run it with the same inputs.
I need help please.
J.A. Coutts
BTW: Your "mdTlsSodium" is extensive but very well organized. I don't know if you are aware or not, but Win 10 has a very simplified Hash function you may want to have a look at:
Code:
Private Declare Function BCryptHash Lib "bcrypt.dll" (ByVal hAlgorithm As Long, ByVal pbSecret As Long, ByVal cbSecret As Long, ByVal pbInput As Long, ByVal cbInput As Long, ByVal pbOutput As Long, ByVal cbOutput As Long) As Long
If BCryptOpenAlgorithmProvider(hAlgorithm, StrPtr(HashAlg), StrPtr("Microsoft Primitive Provider"), 0) Then GoTo ReleaseHandles
If BCryptGetProperty(hAlgorithm, StrPtr("HashDigestLength"), VarPtr(cbHashBlock), 4, cbResult, 0) Then GoTo ReleaseHandles
ReDim Hash(cbHashBlock - 1)
On Error GoTo NotW10
If BCryptHash(hAlgorithm, 0&, 0, VarPtr(bInBuffer(0)), GetbSize(bInBuffer), VarPtr(Hash(0)), cbHashBlock) Then GoTo ReleaseHandles
-
Jan 18th, 2021, 02:21 AM
#77
Re: Simulate TLS 1.3
Originally Posted by couttsj
The second problem is that the signature I produce is 64 bytes, and the one that OpenSSL produces is 71 bytes. And the third thing is that the signature is different every time I run it with the same inputs.
The ECDSA plain signature has to be ASN1 encoded. Check out the pvAsn1EncodeEcdsaSignature in mdTlsSodium for sample implementation.
There might be API based implementation with CryptEncodeObjectEx and something like CERT_ECC_SIGNATURE but I have not researched this route. (Do share if you find alternative impl of ASN1 encoding for ECDSA signatures please.)
ECDSA signatures use random padding (most padding schemes do) and that is why the results from each invocation differ so this is normal behavior.
Originally Posted by couttsj
I don't know if you are aware or not, but Win 10 has a very simplified Hash function you may want to have a look at:
Yes, but in mdTlsSodium module I try to use CNG minimally only when there is no equivalent Crypto API impl so that it stays XP compatible as much as possible. Currently on XP only server-side TLS is dropped as there are no Crypto API functions for RSA-PSS and ECDSA signatures. (The client-side TLS currently just *ignores* signature verification so this "works" on most OS versions :-))
Edit: Found some C/C++ code on ASN1 encoding (in CVE-2014-6321), the call is as simple as this
Code:
CERT_ECC_SIGNATURE sig;
...
CryptEncodeObject(X509_ASN_ENCODING, X509_ECC_SIGNATURE, &sig, (BYTE*)pEnc, &cbEncoded)
cheers,
</wqw>
Last edited by wqweto; Jan 18th, 2021 at 02:36 AM.
-
Jan 19th, 2021, 11:37 AM
#78
Re: Simulate TLS 1.3
Originally Posted by wqweto
The ECDSA plain signature has to be ASN1 encoded.
cheers,
</wqw>
It took a lot of searching to find information on the ASN.1 format, so I decided to show it here for completeness.
Code:
Signature:
99 1A 32 A4 9C F4 CF 4F A6 27 A5 77 14 CB 56 01
DC BB 9D DB 0F 44 0F 2C 69 E8 BF 59 0E AA C6 96
1A 89 2F 1B E6 6B 26 AA 32 A0 19 23 AD 51 2A CA
66 FA 1C 88 D5 0D 7B 0A C5 73 91 96 76 21 3B EC
Cert Verify:
0F 00 00 4B
04 03 - Signature scheme (ECDSA SECP256r1 SHA 256)
00 47 - Length (71)
30 - ASN.1
45 - Len=69
02 21 - r length
00 - extra zero byte because of leading 8 bit
99 1A 32 A4 9C F4 CF 4F A6 27 A5 77 14 CB 56 01
DC BB 9D DB 0F 44 0F 2C 69 E8 BF 59 0E AA C6 96
02 20 - s length
1A 89 2F 1B E6 6B 26 AA 32 A0 19 23 AD 51 2A CA
66 FA 1C 88 D5 0D 7B 0A C5 73 91 96 76 21 3B EC
Note that if the high order bit (&H80) is true, a zero byte is added to the r or s field. This means that the length can be 70, 71, or 72 for an SHA256 signature. Not all ASN.1 implementations support this feature, as ASN.1 was designed primarily for ASCII data, not binary data.
I really don't understand the purpose of this addition, as it adds complexity without adding any real value.
J.A. Coutts
-
Jan 19th, 2021, 03:35 PM
#79
Re: Simulate TLS 1.3
Btw, I just tested CryptEncodeObject idea above and the API based encoding works as far back as NT 4.0 (tested on Win7, XP and Win2000) something like this
Code:
Option Explicit
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (Destination As Any, Source As Any, ByVal Length As Long)
Private Declare Function IsBadReadPtr Lib "kernel32" (ByVal lp As Long, ByVal ucb As Long) As Long
Private Declare Function CryptEncodeObjectEx Lib "crypt32" (ByVal dwCertEncodingType As Long, ByVal lpszStructType As Any, pvStructInfo As Any, ByVal dwFlags As Long, ByVal pEncodePara As Long, pvEncoded As Any, pcbEncoded As Long) As Long
Private Type CRYPT_BLOB_DATA
cbData As Long
pbData As Long
End Type
Private Type CERT_ECC_SIGNATURE
r As CRYPT_BLOB_DATA
s As CRYPT_BLOB_DATA
End Type
Private Sub Form_Click()
Const X509_ASN_ENCODING As Long = 1
Const X509_ECC_SIGNATURE As Long = 47
Dim baR(0 To 32) As Byte
Dim baS(0 To 32) As Byte
Dim uSig As CERT_ECC_SIGNATURE
Dim lSize As Long
Dim baOutput() As Byte
For lSize = 0 To UBound(baR)
baR(lSize) = lSize + 1
baS(lSize) = lSize + 1
Next
uSig.r.pbData = VarPtr(baR(0))
uSig.r.cbData = UBound(baR) + 1
uSig.s.pbData = VarPtr(baS(0))
uSig.s.cbData = UBound(baS) + 1
If CryptEncodeObjectEx(X509_ASN_ENCODING, X509_ECC_SIGNATURE, uSig, 0, 0, ByVal 0, lSize) = 0 Then
GoTo QH
End If
ReDim baOutput(0 To lSize - 1) As Byte
If CryptEncodeObjectEx(X509_ASN_ENCODING, X509_ECC_SIGNATURE, uSig, 0, 0, baOutput(0), lSize) = 0 Then
GoTo QH
End If
Print DesignDumpArray(baOutput)
QH:
End Sub
. . . and produces this
Code:
0000 - 30 46 02 21 21 20 1F 1E 1D 1C 1B 1A 19 18 17 16 0F.!! ..........
0010 - 15 14 13 12 11 10 0F 0E 0D 0C 0B 0A 09 08 07 06 ................
0020 - 05 04 03 02 01 02 21 21 20 1F 1E 1D 1C 1B 1A 19 ......!! .......
0030 - 18 17 16 15 14 13 12 11 10 0F 0E 0D 0C 0B 0A 09 ................
0040 - 08 07 06 05 04 03 02 01 ........
The problem is that the arrays in the CERT_ECC_SIGNATURE are little-endian encoded (reversed) so this is a bit inconvenient but takes care of all ASN.1 encoding edge cases.
This works on legacy OSes because X509_ECC_SIGNATURE = 47 matches X509_DH_PARAMETERS = 47 as constant value and a comment in the SDK explains that "Its data structure is identical except for the names of the fields" -- indeed CERT_ECC_SIGNATURE struct nicely coincides with CERT_DH_PARAMETERS struct.
Edit: Ooops, the arrays above have 33 elements (not 32) but the ASN.1 encoding is correct.
cheers,
</wqw>
-
Feb 24th, 2021, 02:26 PM
#80
Re: Simulate TLS 1.3
I still cannot get my server to work with Firefox. Firefox responds with:
"Error code:SSL_ERROR_RX_MALFORMED_HANDSHAKE" and a Fatal Alert code 50 (Decode_Error), but it is already well past the handshake. The server has received and decrypted the Get Request, and sent the encrypted Response Header and page data as two separate records.
One web site (ugetfix.com) has suggested that this is a common problem for Firefox users, as the browser establishes a secure connection while attempting to verify the Certificate via DNS. I am using the same Certificate as OpenSSL, which does not produce this error.
J.A. Coutts
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|