SSLStream Zero Test Coverage for Renegotiate

[Drawaes]

There is a fair amount of complex code inside SSLStream to handle the case of renegotiation, including

1. What happens to partial data inflight
2. What happens to data encrypted with the old key
3. A lot of locking

But there is zero testing in the inner or outerloop which makes it possible that during refactoring someone could break it and no one would know until its released.

As there is no way to trigger a renegotiation from .net I would say a website on IIS with a client cert required for a certain path in an outerloop test should kick off a renegotiation?

/cc @stephentoub, @Priya91 and @geoffkizer
ref dotnet/corefx#24352

[Drawaes]

/cc @bartonjs @wfurt as well from the original mention in the PR

[karelz]

We should do this in 2.1 as we were changing code in the area.
Estimation: 0.5 day work

@Drawaes are you interested in adding the tests?

[Drawaes]

As said in the linked issue (this is the way I would see it having to work, but if someone knows different then I can implement it but as I see it now I am not sure its practical for me to do it).

Its not something you can do from .net and I don't know how to sort this out in your outerloop infra. So it will need to be done by a microsoftie, it will either need to be an "OpenSSL" server setup or IIS with certificate validation on a certain path to trigger the renegotiation.

[Priya91]

We already have echo servers setup, could this functionality be added to the echo servers @davidsh ?

[davidsh]

We already have echo servers setup, could this functionality be added to the echo servers @davidsh ?

Not simply. The current "echo" server for most of CoreFx is http://corefx-net.cloudapp.net. That server does support TLS/SSL but it doesn't request any client certificates. It is based on an Azure Cloud Service and isn't easy to change that configuration.

I did add another Azure test server endpoint specifically for testing client certificates being sent to a server.

See:
https://github.com/dotnet/corefx/blob/master/src/Common/tests/System/Net/Configuration.Http.cs#L42

and the current tests that use it. That endpoint is based on an Azure WebSite and hitting that endpoint will require client certificates to be sent. However, I am not sure whether it demand client certificates immediately or whether it does TLS/SSL renegotiation as a later step. Perhaps you can try it out and use WireShark to see if it does what we need. Otherwise, I can work on setting up a new Azure test server endpoint but that will take several weeks.

[Drawaes]

When you hit that server it does a normal negotiation. When you do an http request to that path it then triggers a renegotiation with a cert request.

I confirmed it by just hand writing the HTTP request and using openssl s_client with message logging ;)

[davidsh]

I confirmed it by just hand writing the HTTP request and using openssl s_client with message logging ;)

Ok. Thanks for confirming. So, you should be able to write some interesting tests for SslStream against that endpoint. Make sure to mark those tests as outerloop.

[Drawaes]

I hope you are referring to the "royal you". 😆

[Priya91]

I checked the client certificate echo server, it requires a client cert for TLS handshake, so no secure connection is established without the cert, 403 forbidden is thrown. It's just a normal negotiation. I'm not sure what you mean by sending an http request, there's no TLS/SSL with that.. When i sent just a http request, the response returned 200 OK, didn't request for https upgrade.

[Drawaes]

No you need to connect as TLS initially. Should not need a certificate. Then you make an http request over the TLS channel for the page that has client certs. At this point it will send you a hello request (without dropping the TLS and encrypted still) but this time its asking for a client cert.

Basically using openssl I did

openssl s_client -connect corefx-net-tls.azurewebsites.net:443 -msg

This will then connect and it's waiting for input. I just copied pasted this into the console and hit return 2x to denote the end of the http request

GET /EchoClientCertificate.ashx HTTP/1.1
Host: corefx-net-tls.azurewebsites.net

It then goes through the cycle again with the first message the server sending being

<<< TLS 1.2, Handshake [length 0004], HelloRequest
    00 00 00 00
>>> ??? [length 0005]
    16 03 03 00 f2
>>> TLS 1.2, Handshake [length 00da], ClientHello
    01 00 00 d6 03 03 6c 74 df a3 4b 6f de a7 98 20
    c3 7d fe d6 b7 bf 59 46 6c 8b cd 8e 01 0b 36 b2
    2f 83 9e a5 5a 8f 20 ba 28 00 00 e6 67 c6 a5 90
    48 29 f9 17 64 d2 d9 25 b1 95 9d 50 39 1a 79 57
    98 6b 89 91 46 85 b6 00 36 c0 2c c0 30 00 9f cc
    a9 cc a8 cc aa c0 2b c0 2f 00 9e c0 24 c0 28 00
    6b c0 23 c0 27 00 67 c0 0a c0 14 00 39 c0 09 c0
    13 00 33 00 9d 00 9c 00 3d 00 3c 00 35 00 2f 01
    00 00 57 ff 01 00 0d 0c bb ef 72 a2 d1 4d 76 60
    93 5d 48 62 00 0b 00 04 03 00 01 02 00 0a 00 0a
    00 08 00 1d 00 17 00 19 00 18 00 23 00 00 00 0d
    00 20 00 1e 04 03 05 03 06 03 08 04 08 05 08 06
    04 01 05 01 06 01 02 03 02 01 02 02 04 02 05 02
    06 02 00 16 00 00 00 17 00 00
<<< ??? [length 0005]
    16 03 03 0e d3
<<< TLS 1.2, Handshake [length 006d], ServerHello
    02 00 00 69 03 03 5a 4c 0e 60 0c 82 a4 90 d5 69
    c2 3e 28 82 c7 cc 66 a6 5a 9c 13 82 ba 9f b3 fa
    b7 65 75 c2 cf b8 20 a7 2f 00 00 fd 88 74 3d d9
    6d e0 15 a3 f2 25 be 10 37 53 6b ec a7 3f e3 d8
    9c 50 42 5a 62 77 b7 c0 2f 00 00 21 00 17 00 00
    ff 01 00 19 18 bb ef 72 a2 d1 4d 76 60 93 5d 48
    62 c5 ff 39 fd ee 3e c7 7c ac 9a d9 b8

I have included a gist of my console session log you can see the re-handshake at

https://gist.github.com/Drawaes/a8a8d451ab4cd7ae33944a6186e8da7c#file-openssl-log-L377

[Drawaes]

The reason the above works is because the server itself doesn't seem to require a cert, but that page does so you can connect without the cert, and then it forces you to provide a cert when you hit the page.

[Priya91]

So, I tried to simulate that repro with .NET APIs, and it does result in a renegotiation request. But looks like there's a bug, results in NullReferenceException.

            Socket s = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
            await s.ConnectAsync("corefx-net-tls.azurewebsites.net", 443);
            using (NetworkStream ns = new NetworkStream(s))
            using (SslStream ssl = new SslStream(ns, true))
            {
                await ssl.AuthenticateAsClientAsync("corefx-net-tls.azurewebsites.net");
                Assert.True(ssl.IsAuthenticated);

                byte[] message = Encoding.UTF8.GetBytes("GET /EchoClientCertificate.ashx HTTP/1.1\r\nHost: corefx-net-tls.azurewebsites.net\r\n\r\n");
                await ssl.WriteAsync(message, 0, message.Length);

                await ssl.ReadAsync(message, 0, message.Length);
                Console.WriteLine(Encoding.UTF8.GetString(message));
            }

outputs,

 System.Net.Http.Functional.Tests.HttpClientHandler_ClientCertificates_Test.TestRenegotiation [FAIL]
        System.NullReferenceException : Object reference not set to an instance of an object.
        Stack Trace:
           E:\corefx\src\System.Net.Security\src\System\Net\Security\SslState.cs(1287,0): at System.Net.Security.SslState.FinishRead(Byte[] renegotiateBuffer)
           E:\corefx\src\System.Net.Security\src\System\Net\Security\SslStreamInternal.cs(294,0): at System.Net.Security.SslStreamInternal.<ReadAsyncInternal>d__32`1.MoveNext()
           --- End of stack trace from previous location where exception was thrown ---
  Finished:    System.Net.Http.Functional.Tests

  === TEST EXECUTION SUMMARY ===

[Priya91]

It could be that the handshake failed because client cert was not provided or some other issue.

[Drawaes]

I can take a look this weekend if you want

[Priya91]

@Drawaes Thanks, i'm working on this, after I get the PR on sslstream ciphers in.