Adds option to download caches using AzCopy

[dhadka]

Adds option to download caches using AzCopy. This improves download speed and helps reduce flakiness. AzCopy downloads the file in 4 MB chunks, so it has more granular control over retries and parallelism.

This download option is selected when

1. The cache URL is from Azure blob storage
2. The DISABLE_AZCOPY env var is not set (used to test the http-client download path)
3. azcopy is available on the runner

For hosted runners, azcopy is available on Ubuntu and Mac, but not Windows. On Ubuntu, we use the azcopy10 alias since azcopy refers to a much earlier version (7.x.x).

Stats

The table below shows the average time to download a 1 GB file from an EastUS storage account to a VM hosted in EastUS2 or WestUS:

Client           EastUS2 VM          WestUS VM
---------------  ------------------  ----------------
http-client      43.6 s              141 s
azcopy           7.5 s               22.4 s

TODO

• Get AzCopy installed on Windows runners (Add azcopy on Windows runner-images#1004)

[aiqiaoy]

What do you mean by correct version? What happens if we just do the version 10 or newer check?

[dhadka]

azcopy on Ubuntu is version 7.x.x, which uses a completely different CLI. So we need to use the azcopy10 alias on Ubuntu to get to version 10.

This is different on a Mac, where azcopy is v10. Windows currently does not have AzCopy installed, but I already filed an issue to fix that.

I'm not sure why it's setup this way, but this logic is needed to ensure we get the correct version on Ubuntu.

[dhadka]

Closing, PR to use the NodeJS library for Azure storage used instead of AzCopy.