Provide a way to override HTTP parser

[gajus]

What problem are you trying to solve?

const got = require('got');

const main = async () => {
  const response = await got('https://entradas.cinesa.es/compra/?performanceCode=185560&s=192');

  console.log(response);
};

main();

throws:

(node:52462) UnhandledPromiseRejectionWarning: RequestError: Parse Error: Invalid header value char
    at ClientRequest.<anonymous> (/Users/gajus/Documents/dev/temp/node_modules/got/source/request-as-event-emitter.js:178:14)
    at Object.onceWrapper (events.js:291:20)
    at ClientRequest.emit (events.js:208:15)
    at ClientRequest.origin.emit (/Users/gajus/Documents/dev/temp/node_modules/@szmarczak/http-timer/source/index.js:37:11)
    at TLSSocket.socketOnData (_http_client.js:456:9)
    at TLSSocket.emit (events.js:203:13)
    at addChunk (_stream_readable.js:294:12)
    at readableAddChunk (_stream_readable.js:275:11)
    at TLSSocket.Readable.push (_stream_readable.js:210:10)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:166:17)
(node:52462) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:52462) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Notes:

• works as expected with--http-parser=legacy
• works as expected with https://github.com/creationix/http-parser-js

process.binding('http_parser').HTTPParser = require('http-parser-js').HTTPParser;

const got = require('got');

const main = async () => {
  const response = await got('https://entradas.cinesa.es/compra/?performanceCode=185560&s=192');

  console.log(response);
};

main();

It would be nice if we didn't need to override process.binding('http_parser').HTTPParser and Got accepted this as a configuration.

[gajus]

#646

Looks like I have already had similar issue in the past.

It seems that Got could at the very least allow to override the parser that it will use to make requests.

[gajus]

Annoyingly, it looks like the issue is intermittent.

However, I was able to capture Got response when it was happening (using --http-parser=legacy):

{
  'cache-control': 'private',
  'content-type': 'text/html; charset=utf-8',
  server: '',
  'set-cookie': [
    'ASP.NET_SessionId=s44eyjpra41kfbzf2thltx3w; domain=.cinesa.es; path=/; HttpOnly',
    'ReceiveComunications=; domain=.cinesa.es; path=/',
    'visid_incap_626733=Hz5ALXzaRU2XVWfrrQUnv4Q+TF0AAAAAQUIPAAAAAABJs5Fln5jLbHGPPxpgbfUk; expires=Fri, 07 Aug 2020 08:40:57 GMT; path=/; Domain=.cinesa.es',
    'nlbi_626733=jiFkFti8tFSxW7tfrg7CCgAAAADaVlWeOHrNRlTVZ1XRbc9Y; path=/; Domain=.cinesa.es',
    'incap_ses_505_626733=cYXeHS6e205z8wbcXiACB4Q+TF0AAAAAq9/6MKcPgKjM2iROYAtIxQ==; path=/; Domain=.cinesa.es',
    '___utmvmtRuEBFs=SefeVlEgWJF; path=/; Max-Age=900',
    '___utmvatRuEBFs=aIm\u0001dPfD; path=/; Max-Age=900',
+    '___utmvbtRuEBFs=tZu    XmsOvalH: Ity; path=/; Max-Age=900'
  ],
  'x-aspnet-version': '4.0.30319',
  'x-content-type-options': 'nosniff',
  'x-frame-options': 'SAMEORIGIN',
  'x-xss-protection': '1; mode=block',
  date: 'Thu, 08 Aug 2019 15:23:47 GMT',
  connection: 'close',
  'x-iinfo': '2-4598390-4598419 NNNN CT(34 35 0) RT(1565277826197 663) q(0 0 1 -1) r(12 12) U12',
  'x-cdn': 'Incapsula',
  'content-encoding': 'gzip',
  'transfer-encoding': 'chunked'
}

Note the highlighted header. It is invalid. And that is what is causing the issue.

[gajus]

This is what cURL output looks like:

HTTP/1.1 200 OK
Cache-Control: private
Content-Length: 85151
Content-Type: text/html; charset=utf-8
Server:
Set-Cookie: ASP.NET_SessionId=j3ot1vtvpdjd4azav4kx4jy4; domain=.cinesa.es; path=/; HttpOnly
X-AspNet-Version: 4.0.30319
Set-Cookie: ReceiveComunications=; domain=.cinesa.es; path=/
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
Date: Thu, 08 Aug 2019 15:29:44 GMT
Set-Cookie: visid_incap_626733=VgZX+172S9m+3qs6LZb+6uo/TF0AAAAAQUIPAAAAAADhyReZXzYt0z5Nl0lFaHpQ; expires=Fri, 07 Aug 2020 08:40:55 GMT; path=/; Domain=.cinesa.es
Set-Cookie: nlbi_626733=nidDRBmxkDhYqfjvrg7CCgAAAACQ8uIVl9PHfMK8a29HPDmY; path=/; Domain=.cinesa.es
Set-Cookie: incap_ses_505_626733=b/NLSlpGpC69NAjcXiACB+o/TF0AAAAAGLLmZGxEC4nXDMzm5AKWFw==; path=/; Domain=.cinesa.es
Set-Cookie: ___utmvmtRuEBFs=QrssVCikPEc; path=/; Max-Age=900
+Set-Cookie: ___utmvatRuEBFs=cvd^Ajeia; path=/; Max-Age=900
Set-Cookie: ___utmvbtRuEBFs=DZh
    XoFONalR: vtW; path=/; Max-Age=900
X-Iinfo: 10-7040743-7035806 PNNN RT(1565278185321 761) q(0 0 0 -1) r(7 7) U6
X-CDN: Incapsula

That ^A is not supposed to be there.

[gajus]

There appears to be no way to fix this at the moment.

nodejs/node#22064 (comment)

[gajus]

I have documented a solution in this thread.

nodejs/node#22064 (comment)