TLS handshake error when bumping netty to 4.1.69.Final and tcnative to 2.0.44.Final

[dapengzhang0]

https://source.cloud.google.com/results/invocations/71e5d11c-8ed1-421a-86a0-b646f0eecb74/targets/grpc%2Fjava%2Fmaster%2Fpresubmit%2Fmacos/log

io.grpc.netty.ProtocolNegotiatorsTest > from_tls_clientAuthRequire_noClientCert FAILED
    expected:
        ssl exception
    but was:
        io exception
        Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]
        at io.grpc.netty.ProtocolNegotiatorsTest.from_tls_clientAuthRequire_noClientCert(ProtocolNegotiatorsTest.java:418)

io.grpc.netty.NettyClientTransportTest > tlsNegotiationServerExecutorShouldSucceed FAILED
    java.util.concurrent.TimeoutException: Waited 10 seconds (plus 10086801 nanoseconds delay) for SettableFuture@2fab9b61[status=PENDING]
        at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:508)
        at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:111)
        at io.grpc.netty.NettyClientTransportTest$Rpc.waitForResponse(NettyClientTransportTest.java:847)
        at io.grpc.netty.NettyClientTransportTest.tlsNegotiationServerExecutorShouldSucceed(NettyClientTransportTest.java:720)

[ljnelson]

(Lurking. Curious if this test failure is due in part or whole to Netty having turned off TLS 1.)

[sanjaypujare]

The first failure is because of error mismatch (io exception instead of ssl exception) which does not seem like a big issue.

The second one - I don't see any connection with disabling TLS1 but needs to be investigated

[ljnelson]

I've noticed the first failure doesn't occur (on my Mac) with JDK 8u301 in case that helps (if I understand right this project is still pinned on JDK 8).

I also now receive the following on the console when I run JAVA_HOME=$(/usr/libexec/java_home -v1.8) ./gradlew :grpc-netty:test:

[…]
> Task :grpc-netty:test
[jetty-alpn-agent][ info] alpn-boot not necessary for Java version: 1.8.0_301

…suggesting that the problem was more than a simple exception mismatch brought on by an upgrade and may indeed have to do with protocol negotiation. But I am still just speculating.

I'm still looking into the second issue.

[ljnelson]

More findings: I can make all our Helidon gRPC-related tests pass by setting the io.netty.handler.ssl.openssl.useTasks system property to false. I wonder if that would make the second failure here go away as well.

[ljnelson]

By adding:

systemProperty 'io.netty.handler.ssl.openssl.useTasks', 'false'

…to netty/build.gradle in the test block I make the remaining failure go away. This suggests an underlying problem in Netty.

[ljnelson]

Filed: netty/netty-tcnative#680

[sanjaypujare]

@normanmaurer this issue is still occurring on Macos and Windows. Will the fix for netty/netty-tcnative#680 (which is netty/netty#11854) not work on MacOS and Windows? What else is needed?

More information is available from tests in #8780 and #8813 . Some findings:

• On MacOS before the netty upgrade (to 4.1.72.Final) it was using TLSv1.3 but after the upgrade it was using TLSv1.2. This causes a test to fail because of an unexpected exception (IOException instead of SSLException)
• On Windows too we see the same behavior: dropping down to TLSv1.2 with the latest netty but using TLSv1.3 before the netty upgrade

[normanmaurer]

thats interesting.. there shouldn't be any other changes needed.

[normanmaurer]

@sanjaypujare can I somehow get the debug logs of netty for these failed test runs ?