cli(extra-headers): Enable sending additional HTTP Headers via the CLI

[rupesh1]

I can across #2746 which mentioned a SO question, that had a link to commit by @fdn - after taking a look it seemed abandoned but functional. So based off their initial commit I've:

• merged in master (had to resolve some conflicts - initial commit was v1.6.0)
• fixed unit tests
• updated some references and function locations to match the current codebase
• added unit tests and basic docs

I've tested the CLI locally and it's working as expected for me.

P.s. I had to run yarn install-all a few times for it to run cleanly hence adding yarn-error.log to the gitignore

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If your company signed a CLA, they designated a Point of Contact who decides which employees are authorized to participate. You may need to contact the Point of Contact for your company and ask to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot.
• In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again.

[rupesh1]

TODO: Hmm this should probably be of type string

[rupesh1]

I signed it!

[googlebot]

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this State. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

[fdn]

Merge away. Glad this commit is going to see the light of day! Good stuff @rupesh1

[johntron]

@paulirish This PR looks useful for both CLI and programmatic use. If the conflicts are resolved, is there a reason it can't be merged? I know there were other issues where we were discussing using Chrome's setCookie.

[johnboxall]

This feature would be handy for us: we have conditionals in our code the can be activated/deactivated using HTTP Headers. Specifically cookies.

Our current workflow is:

1. Launch an instance of Chrome: ./node_modules/.bin/chrome-debug. Note the port number!
2. In the Chrome instance, set the state you want to test. For example, navigate to a page and set cookies using the devtools.
3. Clear cached images / files via chrome://settings/clearBrowserData?search=cache
4. In another terminal, run Lighthouse, providing the --port argument you noted.

We could script this behaviour using the Chrome Debug Protocol, but ... wouldn't it be sweet if you could just pass an option into 💡 🏠 ?

❤️

[patrickhulce]

thanks for the PR @rupesh1!!

sorry for the long delay but it seems to be quite popular these days! 🎉 we like the idea and just need some 🚲 🏠 on the CLI usage a bit

[patrickhulce]

wdyt about handling this in the CLI instead?

this way we could fail early and also do something fancy like accept files :)

let extraHeaders
if (argv.extraHeaders) {
  if (argv.extraHeaders.substr(0, 1) === '{') {
    extraHeaders = JSON.parse(argv.extraHeaders)
  } else {
    extraHeaders = JSON.parse(fs.readFileSync(argv.extraHeaders))
  }
}

[paulirish]

@rupesh1 holler if you can make this change. if you've moved on, we can pick it up;

[patrickhulce]

we could probably skip this entirely if headers is invalid/empty instead

[rupesh1]

@patrickhulce @paulirish I've had a go at implementing your suggestions (everything passes and runs locally for me node --version 8.9.1, yarn --version 1.3.2), but now bumped in to some typings issue with yargs when the CI runs yarn unit-cli (weirdly yarn type-check passes on TravisCI):

/home/travis/build/GoogleChrome/lighthouse/lighthouse-cli/cli-flags.js:137
      .coerce('extra-headers', /** @param {string} arg */ (arg) => {
       ^
TypeError: y.help(...).version(...).showHelpOnFail(...).usage(...).example(...).example(...).example(...).example(...).example(...).example(...).example(...).example(...).group(...).describe(...).group(...).describe(...).alias(...).group(...).describe(...).boolean(...).choices(...).array(...).string(...).default(...).default(...).default(...).default(...).default(...).default(...).check(...).coerce is not a function

I can't reproduce the error locally so any help is appreciated.

[patrickhulce]

Thanks for updating @rupesh1!! It looks like you might have a newer version of yargs installed locally in lighthouse-cli/node_modules we're quite behind at v3 which doesn't have the .coerce option which is why travis is failing

how about we move the parsing logic over to bin.js for now instead?

[rupesh1]

@patrickhulce thanks for the tip, CI is looking much better now

[patrickhulce]

this looks awesome! great job @rupesh1! 🎉 💯

everything a-ok by me, @paulirish over to you for any opinions on the CLI usage

[paulirish]

I'm not 100% on using a file rather than a string, but neither seems like the clear winner. Let's do this.

[paulirish]

thanks @rupesh1 !

[rupesh1]

Awesome! Really appreciate everyone's help with this PR 👍

[JoelTW]

This is great. Is there any example of the format required for the json file?

[patrickhulce]

@JoelTW it should just be key/value header name/value

{
  "Cookie": "token=12345acdfe",
  "x-secret-header": "it's a secret"
}

[Pagan-Idel]

Is there an example we can use these extra headers through programmatical Lighthouse?