cannot use tls.TLSSocket created from net.Socket after first 'data' event

[coolaj86]

tried on node v6.3 and v6.6

My goal is to mux a socket to use either tcp or tls after inspecting the first packet to see if it's a hello.

The problem I have is that I could not pass the socket to tls.TLSSocket (or tls.Server, or https.Server) after the 'data' event (even though I replay it) and have it emit data.

issue-8752.js:

//
//
// SETUP
//
//
'use strict';

var net = require('net');
var tls = require('tls');
var http = require('http');
var tlsOpts = {
  key: require('fs').readFileSync('./privkey.pem')
, cert: require('fs').readFileSync('./fullchain.pem')
};
var httpServer = http.createServer(function (req, res) {
  res.end('Hello, World!');
});
var tcp3000 = net.createServer();

tcp3000.listen(3000, function () {
  console.log('listening on 3000');
});



tcp3000.on('connection', function (socket) {
  console.log('connection incoming');
  // this works when I put it here, but I don't know if it's tls yet here
  // httpsServer.emit('connection', socket);

  socket.once('data', function (chunk) {
    console.log('data incoming');

    if (/http\/1/i.test(chunk.toString())) {

      console.log("looks like http, continue");

      // this works as expected
      httpServer.emit('connection', socket);

    } else {

      //**********************************************************
      //
      // PROBLEM
      //
      //**********************************************************
      console.log("doesn't look like http, try tls");

      // none of these methods work:
      // httpsServer.emit('connection', socket);  // this didn't work
      // tlsServer.emit('connection', socket);    // this didn't work either
      var tlsSocket = new tls.TLSSocket(socket, { secureContext: tls.createSecureContext(tlsOpts) });
      tlsSocket.on('data', function (chunk) {
        // never called
        console.log('chunk', chunk);
      });

    }

    // replay first packet
    socket.emit('data', chunk);
    // I've also tried this, which didn't work at all:
    // socket.unshift(chunk);
    // socket.emit('readable');
  });

});

curl https://local.test.ppl.family

connection incoming
data incoming
looks like http, continue
curl: (35) gnutls_handshake() failed: An unexpected TLS packet was received.

From the documentation I've found I don't think I'm doing anything wrong and the same process works with a plain tcp socket, so it makes me wonder if there isn't something amiss in the internals.

Also, the following does work (even though it's useless):

'use strict';

var net = require('net');
var http = require('http');
var https = require('https');
var tlsOpts = {
  key: require('fs').readFileSync('./privkey.pem')
, cert: require('fs').readFileSync('./fullchain.pem')
};
var httpServer = http.createServer(function (req, res) {
  res.end('Hello, World!');
});
var httpsServer = https.createServer(tlsOpts, function (req, res) {
  res.end('Hello, Encrypted World!');
});
var tcp3000 = net.createServer();

tcp3000.listen(3000, function () {
  console.log('listening on 3000');
});



tcp3000.on('connection', function (socket) {
  // if I never let the 'data' event fire, it works as expected.
  httpsServer.emit('connection', socket);
});

[coolaj86]

I found a workaround which suggests this must be a bug and not intentional behavior.

Even with the workaround, if I try to use tls.TLSSocket directly I get back an error:

curl: (35) Unknown SSL protocol error in connection to example.com:-9800

However, this allows me to at least use tls.Server.emit('connection'):

      console.log("doesn't look like http, try tls");

      var myDuplex = new Duplex();
      myDuplex._write = function (chunk, encoding, cb) {
        socket.write(chunk, encoding);
        cb();
      };
      myDuplex._read = function (size) {
        var chunk = socket.read(size);
        if (chunk) { this.push(chunk); }
      };
      socket.on('data', function (chunk) {
        myDuplex.push(chunk);
      });

      tlsServer.emit('connection', myDuplex);

And tlsServer looks like this:

var tlsServer = tls.createServer(tlsOpts, function (tlsSocket) {
  httpServer.emit('connection', tlsSocket);
});

[mscdex]

Did you try using the recommended way of upgrading a socket? For example:

var tlssocket = tls.connect({ socket: socket }, function() {
  console.log('Socket upgraded!');
  // use `tlssocket`
});

Also on a semi-related note, you shouldn't make any assumptions about chunks of data from a stream. The substring you're checking for could be cut across boundaries.

[mscdex]

If you're wanting to serve http and https over the same port, you might try httpolyglot.

[coolaj86]

I'm not trying to perform a socket upgrade. The socket is already tls. Nevertheless, when I replace my hacky code with your code I got this result:

events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: socket hang up
    at TLSSocket.onHangUp (_tls_wrap.js:1092:19)
    at TLSSocket.g (events.js:291:16)
    at emitNone (events.js:91:20)
    at TLSSocket.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)

I'm working on a tunneling service, which happens to include serving both http and https on the same port, but is a bit more complicated than just that.

The end goal is to allow many authenticated clients to tunnel anything - http, https, and even ssh or openvpn (wrapped in openssl as to appear to be https and to get the sni header) - and that will work in school dorms, HOAs, public libraries, corporate networks, airplanes, and other harsh network environments (and on all OSes without requiring admin privileges, iptables, etc).

Similar concept to localtunnel.me but with arbitrary encrypted traffic and eventually that could even work on a chrome book in the browser.

[mscdex]

FWIW as far as SSH goes, you may run into issues with buggy SSH clients since I've seen some that actually wait for the server to respond with their ident string first before sending theirs (even though this kind of goes against the RFC, depending on your interpretation).

Also, tunneling OpenVPN over TCP? I think most clients would be using UDP for performance? Either way, wrapping OpenVPN traffic with encrypted OpenSSL traffic seems like a lot of overhead.

[coolaj86]

I don't want to pull this thread away from the core issue of figuring out how to wrap and unwrap net and tls sockets, so I continued the tangent here: https://gist.github.com/coolaj86/f7c39baf83c32525f7c88ed13643e11d

[Trott]

This is still an issue in Node.js 8.x?

[coolaj86]

This is still an issues in node v9.11.1

[Trott]

@nodejs/crypto @nodejs/streams

[bnoordhuis]

The reason the example doesn't work is that tls.TLSSocket short-circuits the net.Socket machinery for performance reasons. That's its sole raison d'être really, and not something we'll change.

You can use tls.createSecurePair(). It's deprecated but if it supports a use case that tls.TLSSocket does not, we can look into undeprecating it.

[coolaj86]

@bnoordhuis The use case is building a net proxy service that can inspect incoming traffic and route it accordingly.

We don't know ahead of time if the incoming message is tls or not (or if it's http or https), so we need to inspect the first packet and then pump the stream through the rest of the stack.

Could you add a peek() function into the tcp API so that we can inspect and determine what it is?

We'd also like to be able to track the bytes going over the wire per-session. For some traffic we authenticate with https or wss much later, but we need access to the byte count from the tcp object (to know total packet size, including the increase caused by encryption).