Allow azure-cli to trust on OS certificates

[joaocc]

Related command
any command (prob REQUESTS_CA_BUNDLE is the closest)

Is your feature request related to a problem? Please describe.
Yes. Enabling azure-cli to work with custom trusted cert roots.

On macOS it is not simple to configure https://learn.microsoft.com/en-us/cli/azure/use-cli-effectively?tabs=bash%2Cbash2#work-behind-a-proxy as:

1. the ca_bundle is located in a folder that depends on 2 variables: client version and python version (/usr/local/Cellar/azure-cli/<cliversion>/libexec/lib/python<version>/site-packages/certifi/cacert.pem)
2. every time we upgrade the client, the bundle has to be recreated

Describe the solution you'd like
Allow azure-cli to trust the certificates that are trusted by the OS (keychain on macOS, others on other OS).
This could be the default behaviour or, in order to keep compat with current behaviour, allow enabling it via both env var (such as CA_BUNDLE_TRUST_OS) and CLI option (--ca-bundle-trust-os)

Alternatively (or in conjuntcion), allow specification of an additional bundle vi both env var (such as REQUESTS_CA_BUNDLE_EXTRA) and CLI option (--requests-ca-bundle-extra)

Describe alternatives you've considered
Implementing bash scripts that need to try to guess the correct location to place new certs, generate a new bundle, and then rewriting.

Additional context

[yonzhan]

Thank you for opening this issue, we will look into it.

[codepic]

Looking through the source code of azure-cli, it seems like you already have 2 options:

1. Use the AZURE_CLI_DISABLE_CONNECTION_VERIFICATION (not recommended)
2. Use the REQUESTS_CA_BUNDLE

For the latter, you should be able to export all CA certificates from Apple KeyChain into a single PEM bundle

If the Azure CLI team were to implement a trust OS certificates feature, I think a better route would be to do it through Azure CLI configuration az config set.

The problem with passing arguments to commands for this purpose is that it really depends on the environment. On some environments you do want to trust OS certificates, on others you don't. This adds complexity into your scripts because now you need to do additional if statements depending on the environment you are in. The hypothetical if statement would need to have something to check, and that would in 99% of the cases be a custom environment variable. Which is kind of weird situation because now you've moved from using one environment variable to using another (probably undocumented one that nobody knows of).

In the end of the day, you need to ask yourself why do you even need to deal with this stuff?. If the reason is that your company is using a self-signed certificate at the proxy due to cost reasons, you can inform them how much does it cost to work around the issues that arise from the lack of proper certificate at the proxy and how many developers will simply disable cert checks completely to overcome such issues.

[joaocc]

HI @codepic. Thanks for the update.
In our case, the reason for using custom certs is different - we are using a corp firewall that does traffic inspection, so we need to deploy the cert to all machines. As you can understand, this is not a matter of short-sighted cost savings.
In a more general note, any tool that doesn't use the OS certs requires additional effort (to plan, test, deploy, maintain) instead of trusting the user/developer/architect/IT staff to decide what is more suitable for their particular environment. Even Firefox ended up implementing a way to enable users to decide whether or not to trust the certs that the OS trusts (Edge and Chrome do that by default, as do many other tools and apps).
For the reasons above, the "trust OS certificates" option (or default) would be a much cleaner and simpler solution, and one that would not have any downside for existing users, but would facilitate the lives of many others.
I agree that the specific mechanics to enable it would be best determined by the team (as I was just providing illustrative examples).

[jiasli]

@joaocc, I fully understand your concern. @codepic, thank you very much for the detailed explanation.

The fact is that, Azure CLI internally depends on requests library and has no special logic for choosing and using CA bundle. There is an issue opened in requests repo for this feature request:

• Consider using system trust stores by default in 3.0.0. psf/requests#2966

[joaocc]

@jiasli, looking at the issue you pointed, it seems there is now what seems to be a simple(r) way to address this without waiting for requests (sorry if I am oversimplifying :)), but using truststore.

• https://truststore.readthedocs.io/en/latest/#using-truststore-with-requests

# Using truststore with Requests
# Just like with urllib3 using truststore.inject_into_ssl() is the easiest method for using Truststore with Requests:

import requests
import truststore

truststore.inject_into_ssl()

resp = requests.request("GET", "https://example.com")

I understand it might not be so simple to make a change in a core are as this, but... Worth trying to ask for this to be considered, especially due to the upside for the community of azure-cli users.
Thanks

[Ousret]

There's an another option. In fact two options.

• The first would be to replace requests by (drop-in replacement) niquests, with implemented support for Consider using system trust stores by default in 3.0.0. psf/requests#2966 (amongst other things)
• Finally, you could consider using the wassima library to extract the system CA bundle automatically.

The caveat of truststore is the monkeypatching of the stdlib and that it is exclusive to Python 3.10+

[bodnarbm]

@joaocc, I fully understand your concern. @codepic, thank you very much for the detailed explanation.

The fact is that, Azure CLI internally depends on requests library and has no special logic for choosing and using CA bundle. There is an issue opened in requests repo for this feature request:

• Consider using system trust stores by default in 3.0.0. psf/requests#2966

The referenced issue with requests is now closed as completed, with the resolution being the recommendation to use truststore to patch requests.

@jiasli could we evaluate conditionally adding/importing truststore. For most systems on python 3.10+, this should solve the issue with a better user experience then messing with the bundled certs. For systems where truststore does not work (python 3.9 or old versions of macos), then the conditional import should not affect preset behavior.