host vs hostname inconsistency

[sholladay]

Version: Node 8 and others

Problem

The semantics of the host option for server.listen() are incompatible with the conventions of other APIs, both in Node itself and in browsers, etc. This is a footgun that could be fixed by making the options for .listen() be consistent with other APIs.

Context

Most systems I interact with distinguish between a hostname and a host, where the latter includes a port number and the former does not.

In general, Node behaves this way, too (see os.hostname(), WHATWG URL, and the legacy url module APIs). However, the option object passed to server.listen(option) is weird. It only understands host (not hostname), but a port number is disallowed. This is confusing and leads to very surprising results in some cases.

Prior art

In browsers

location.href = 'https://localhost:3000';
console.log(location.host !== location.hostname);  // true
const whatwgParsed = new URL('https://localhost:3000');
console.log(whatwgParsed.host !== whatwgParsed.hostname);  // true

In Node.js

const url = require('url');
const target = 'https://localhost:3000';
const legacyParsed = url.parse(target);
console.log(legacyParsed.host !== legacyParsed.hostname);  // true
const whatwgParsed = new url.URL(target);
console.log(whatwgParsed.host !== whatwgParsed.hostname);  // true

Current behavior

The server.listen() API understands a host option, but it is incompatible with host from other APIs, including WHATWG URL and others. This is because .listen() refuses perfectly valid hosts such as localhost:3000. A reasonable expectation would be to try hostname instead, which is the more common name for the current behavior of the host option. Unfortunately, that doesn't work either, as .listen() does not understand hostname, and it will be silently ignored.

// Reports an error correctly, since the API lacks a default port, but the error message is somewhat vague
server.listen({ host : 'localhost' }, () => {});
Error: Invalid listen argument: { host: 'localhost' }

// Listens on 0.0.0.0 rather than localhost (works as documented, but very confusing and as a footgun is arguably unsafe)
server.listen({ hostname : 'localhost', port : 3000}, () => {});

// Reports an error even though the host is valid (works as documented, but confusing and incompatible with other specs and APIs)
server.listen({ host : 'localhost:3000' }, () => {});
Error: Invalid listen argument: { host: 'localhost:3000' }

// Succeeds, even though host usually takes precedence over separate hostname and port options and the host lacks a port (works as documented, but a bit strange)
server.listen({ host : 'localhost', port : 3000 }, () => {});

Expected behavior

APIs that accept a host should respect their port number to avoid confusion and inconsistency. Having separate options for hostname and port is also awesome, but in that case it should be named hostname instead of host. I would argue that Node should only support hostname instead of host (leaving parsing hosts to userland) but that would be a breaking change, so I'm not sure how feasible that is. At the very least, I think a new hostname option could be introduced with the correct behavior, and then host could be extended to respect port numbers, and the fact that host continues to be in the API would purely be for backwards compatibility reasons.

// Should report an intuitive error. The host is perfectly valid, but the API lacks a default port.
server.listen({ host : 'localhost' }, () => {});
Error: No port specified

// Should listen on localhost, whereas it currently listens on 0.0.0.0 instead because hostname is not understood.
server.listen({ hostname : 'localhost', port : 3000}, () => {});

// If host needs to be supported, then this should listen on localhost and port 3000, whereas it currently throws an error.
server.listen({ host : 'localhost:3000' }, () => {});

// Should probably throw an error, whereas it currently does not.
server.listen({ host : 'localhost', port : 3000 }, () => {});

[maclover7]

@sholladay Would it make sense as a first step to add some more documentation about this behavior?

[PatrickHeneise]

What's the expected behaviour?

[sholladay]

@maclover7 Perhaps calling out the inconsistency would be nice. I don't find the documentation to be bad, per se, though. It's the API design that should be improved.

@PatrickHeneise The expected behavior is that host and hostname have consistent definitions throughout Node's API surface. I expanded upon my examples above, so hopefully it is more clear now. Also, here is a demo of incompatibility with the whatwg/url spec. Does this help?

const { URL } = require('url');
const server = require('net').createServer();
const { host, hostname, port } = new URL('http://localhost:3000');
server.listen({ host }, () => {});  // errors
server.listen({ hostname, port }, () => {});  // listens on the wrong hostname

[bnoordhuis]

The { host : 'localhost' } error message would be good to fix but the other "expected behaviors" could potentially create (possibly severe) ecosystem fallout for not enough value to take the risk.

Perhaps a good middle ground is to follow what http.request() does and prefer .hostname over .host, that makes url.parse() and new url.URL() work transparently.

[sholladay]

@bnoordhuis nothing I am asking for here should be a breaking change, as far as I can tell.

• All host values that Node currently accepts could still be valid, no need for users to do anything.
• The host option, if you want to keep it around for backwards compat (as mentioned above), would be extended to also allow ports in it (not require them).
• A new, discrete hostname option could be introduced, which would be exactly the same as the current host option.

There seems to be a clear path towards whatwg URL compatibility without any breakage changes.

[bnoordhuis]

The host option, if you want to keep it around for backwards compat (as mentioned above), would be extended to also allow ports in it

Whether that can be done in a backwards compatible way depends on whether host+port strings can always be unambiguously distinguished from strings that just look like them. I'm thinking of strings like abba::1972 that are valid IPv6 addresses.

(I made ^ up on the spot. More realistic examples can be construed with a bit more effort.)

[sholladay]

From what I can tell playing around in the REPL, Node doesn't allow : in the host option today. I could be wrong, though. Do you have any actual examples?

http.createServer().listen({ host : 'abba::1972' })
Error: Invalid listen argument: { host: 'abba::1972' }

http.createServer().listen({ host : '::1' })
Error: Invalid listen argument: { host: '::1' }

[bnoordhuis]

That's the other thing you commented on, the mandatory .port property. =)

(Touches on another issue: what if .host and .port both contain port numbers? What if they don't agree?)

[sholladay]

Oh, right. True enough... the error message is misleading. At any rate, from what I've read, part of the purpose of the URL spec was to clean up the grammar. I imagine they have a good way to distinguish IPv6 and such from hosts with ports. But I confess I don't know the details of how that works. Since Node already has the URL parser implemented, perhaps it should just use that directly and not worry about re-implementing anything?

As for conflicting input about the port number, here again I would go for consistency with the existing APIs. I don't think WHATWG provides any functionality that is prone to that, but Node's url.format() does and it treats host as taking precedence.

url.format({ host : 'localhost:4000', port : 5000 });
// 'localhost:4000'

It could even do something like const { hostname, port } = new url.URL(url.format(...args)) if you really wanted to get fancy.

[tswaters]

Using url parser directly won't really work without the other bits to make a valid url... e.g.:

const myUrl = new url.URL(url.format({host: 'localhost:3000'}))
myUrl.protocol // "localhost:"
myUrl.pathname // "3000"

It would need a dummy protocol (or maybe just http) to pull out hostname/port/host properly. And, even then, it wouldn't work for the current case, passing both host and port:

const myUrl = new url.URL(url.format({protocol: 'http', host: 'localhost', port: 4000}))
myUrl.host     // "localhost"
myUrl.hostname // "localhost"
myUrl.port     // "" - port is lost when `host` provided over `hostname`

[sholladay]

I'm not necessarily asking for Node itself to directly use the WHATWG URL parser. That is an implementation detail that may or may not be appropriate (though it seems attractive, if it can be made to work).

Personally, I just want one of the following to correctly listen on localhost port 3000 without reporting an error.

const { hostname, port } = new URL('http://localhost:3000');
server.listen({ hostname, port }, () => {});

const { host } = new URL('http://localhost:3000');
server.listen({ host }, () => {});