{Profile} `az account get-access-token`: Show `expiresOn` for managed identity by jiasli · Pull Request #20219 · Azure/azure-cli

jiasli · 2021-11-05T09:37:03Z

Fix #20211
Fix getporter/azure-plugins#39
Fix microsoft/AzureTRE#1067
Fix Azure/azure-cli-extensions#4076

Description

Issue: During the migration to MSAL, for managed identity, the expiresOn property in az account get-access-token's output is lost.

$ az account get-access-token
{
  "accessToken": "...",
  "expiresOn": null,
  "subscription": "0b1f6471-1bf0-4dda-aec3-cb9272f09590",
  "tenant": "54826b22-38d6-4fb2-bad9-b7b93a3e9c5a",
  "tokenType": "Bearer"
}

This is due to the complexity and inconsistency of expiresOn/expires_on properties across different services and tools.

There are 4 forms of expiresOn/expires_on:

epoch str (VM managed identity endpoint 2018-02-01): "expires_on": "1605238724"
epoch int (Azure Python SDK Track 2): "expires_on": 1605238724
datetime in ISO str representing local time (ADAL token entry): "expiresOn": "2020-11-12 13:50:47.114324"
datetime in Month/Day/Year str with timezone (App service managed identity endpoint 2017-09-01): "expires_on": "11/05/2021 15:18:31 +00:00"

This PR unified these 4 types:

For MSAL tokens, epoch int expires_on is returned.
For managed identity and Cloud Shell, expires_on is unified to epoch int ([App Service] Support managed identity in App Service container #20215).

get_raw_token returns

expiresOn: datetime string for backward compatibility, like "2020-11-12 13:50:47.114324"
[To be decided] expires_on: epoch int, like 1605238724 (az account get-access-token: Use epoch expiresOn/expires_on #19700)

Testing Guide

apt install git
git clone https://github.com/jiasli/azure-cli --branch expires-on --depth 1
apt install python3-venv --yes
python3 -m venv py
. py/bin/activate
pip install -U pip
pip install azdev
azdev setup -c
az login --identity --debug
az account get-access-token --debug

yonzhan · 2021-11-05T13:45:48Z

Profile

kairu-ms · 2021-11-08T02:44:16Z

src/azure-cli/azure/cli/command_modules/profile/custom.py

        'tokenType': creds[0],
        'accessToken': creds[1],
        # 'expires_on': creds[2].get('expires_on', None),
-        'expiresOn': creds[2].get('expiresOn', None),


Why do you need to change this line?

We need to guarantee that expiresOn always exists in creds[2] - CLI should fail if expiresOn is not set, instead of returning None which will cause more trouble.

jiasli · 2021-11-08T09:42:00Z

src/azure-cli-core/azure/cli/core/_profile.py

+            from .auth.util import scopes_to_resource
+            msi_creds = MsiAccountTypes.msi_auth_factory(identity_type, identity_id,
+                                                         scopes_to_resource(scopes))
+            sdk_token = msi_creds.get_token(*scopes)


The old implementation of ADAL-based Azure CLI returns token_entry containing inconsistent fields:

expires_on for managed identity

expiresOn for ADAL credential

They are unified later at command module level:

azure-cli/src/azure-cli/azure/cli/command_modules/profile/custom.py

Lines 82 to 85 in df737ed

if 'expires_on' in token_entry:

# https://docs.python.org/3.8/library/datetime.html#strftime-and-strptime-format-codes

token_entry['expiresOn'] = _fromtimestamp(int(token_entry['expires_on']))\

.strftime("%Y-%m-%d %H:%M:%S.%f")

We use get_token to unify them to epoch int expires_on in core instead.

Just out of curiosity, will token_entry['expiresOn'] has its own value in some cases before it is overwritten by _fromtimestamp(int(token_entry['expires_on'])).strftime("%Y-%m-%d %H:%M:%S.%f")?
May I ask their values should be the same in all cases, but only in different formats, right?

No. get_token only returns AccessToken which must have int epoch expires_on.

waylew-lexis · 2021-11-15T14:52:30Z

any update on when this issue will be merged and released? getting similar issue when trying to use azure app service with custom container.
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: invalid literal for int() with base 10: '11/16/2021 14:26:35 +00:00'
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/knack/cli.py", line 231, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 657, in execute
raise ex
File "/usr/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 720, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 691, in _run_job
result = cmd_copy(params)
File "/usr/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 328, in call
return self.handler(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
return op(**command_args)
File "/usr/lib/python3.8/site-packages/azure/cli/command_modules/profile/custom.py", line 84, in get_access_token
token_entry['expiresOn'] = _fromtimestamp(int(token_entry['expires_on']))
ValueError: invalid literal for int() with base 10: '11/16/2021 14:26:35 +00:00'
To open an issue, please run: 'az feedback'

jiasli · 2021-11-16T06:31:28Z

@waylew-lexis your issue has been resolved by #20215 and will be released in Azure CLI 2.31.0.

# Conflicts: # src/azure-cli-core/azure/cli/core/tests/test_profile.py

jiasli · 2021-11-17T03:10:45Z

src/azure-cli-core/azure/cli/core/tests/test_profile.py

        self.object_id = kwargs.get('object_id')
        self.msi_res_id = kwargs.get('msi_res_id')
        self.resource = kwargs.get('resource')
-        MSRestAzureAuthStub.return_value = self


Don't save the instance as class attribute so that tests won't interfere with each other.

zhoxing-ms · 2021-11-17T03:18:50Z

src/azure-cli-core/azure/cli/core/_profile.py

-            msi_creds.set_token()
-            token_entry = msi_creds.token
-            creds = (token_entry['token_type'], token_entry['access_token'], token_entry)
+                raise CLIError("Tenant shouldn't be specified for managed identity account")


Could we use a specific error type (such as ArgumentUsageError) instead of CLIError?

There are so many places in core that don't comply with the error handling rule. Let's refine them together later.

Maarc-D · 2021-11-18T12:58:10Z

Hello,

do you know when this merged modification will be integrated to a release ?
Because I have a function app that faild because of :

File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/cli/core/adal_authentication.py", line 168, in get_token return AccessToken(self.token['access_token'], int(self.token['expires_on'])) ValueError: invalid literal for int() with base 10: '11/19/2021 10:40:07 +00:00'

That should be fixed with this modification ;)

Regards Marc.

yonzhan · 2021-11-18T14:14:39Z

Build to Cloud Shell: 12/03/2021 Official Release: 12/07/2021

jiasli · 2021-11-19T02:02:18Z

@Maarc-D, your ValueError: invalid literal for int() with base 10: '11/19/2021 10:40:07 +00:00' issue is resolved by #20215.

jiasli added 2 commits November 5, 2021 16:41

expires_on

bfaacc2

unittest

5859d97

jiasli requested review from evelyn-ys, jsntcy, kairu-ms and zhoxing-ms as code owners November 5, 2021 09:37

jiasli self-assigned this Nov 5, 2021

yonzhan added this to the Nov 2021 (2021-12-07) milestone Nov 5, 2021

squillace mentioned this pull request Nov 5, 2021

[BUG]Resource Processor error running porter install when creating Base Workspace microsoft/AzureTRE#1067

Closed

kairu-ms reviewed Nov 8, 2021

View reviewed changes

jiasli added 4 commits November 8, 2021 10:55

datetime

71d161e

Merge remote-tracking branch 'azure/dev' into expires-on

141ab62

resouce

f65d76a

Add tests

4cf877e

jiasli force-pushed the expires-on branch from 145da9b to 4cf877e Compare November 8, 2021 07:52

jiasli commented Nov 8, 2021

View reviewed changes

jiasli mentioned this pull request Nov 9, 2021

az account get-access-token returns expiresOn as null for managed identity #20211

Closed

calvinhzy approved these changes Nov 12, 2021

View reviewed changes

evelyn-ys approved these changes Nov 15, 2021

View reviewed changes

jiasli added 2 commits November 15, 2021 18:07

comment

173d469

Merge remote-tracking branch 'azure/dev' into expires-on

7aa714e

jiasli changed the title ~~{Profile} az account get-access-token show expiresOn for managed identity~~ {Profile} az account get-access-token: Show expiresOn for managed identity Nov 16, 2021

jiasli added 2 commits November 16, 2021 20:24

Merge remote-tracking branch 'azure/dev' into expires-on

eae0254

# Conflicts: # src/azure-cli-core/azure/cli/core/tests/test_profile.py

refine tests

f0746f0

jiasli commented Nov 17, 2021

View reviewed changes

jiasli requested a review from evelyn-ys November 17, 2021 03:11

evelyn-ys approved these changes Nov 17, 2021

View reviewed changes

zhoxing-ms reviewed Nov 17, 2021

View reviewed changes

zhoxing-ms approved these changes Nov 17, 2021

View reviewed changes

jiasli merged commit d7e5ede into Azure:dev Nov 17, 2021

jiasli deleted the expires-on branch November 17, 2021 08:33

jiasli mentioned this pull request Nov 17, 2021

Breaking change latest CLI version Azure/cli#56

Closed

jiasli mentioned this pull request Dec 7, 2021

DefaultAzureCredential not working with Managed Identity (aka "Service Principal") #20628

Closed

This was referenced Feb 9, 2022

AzureCliCredential fails to get token because strptime() argument 1 must be str, not None Azure/azure-sdk-for-python#21653

Closed

handle null expireson Azure/azure-sdk-for-python#22899

Closed

carolynvs mentioned this pull request Feb 22, 2022

did azure change some auth or format? getporter/azure-plugins#39

Closed

pencels mentioned this pull request Sep 13, 2023

AzureCliCredential parses an incorrect token expiry time in multi-threaded Unix applications Azure/azure-sdk-for-rust#1371

Closed

	if 'expires_on' in token_entry:
	# https://docs.python.org/3.8/library/datetime.html#strftime-and-strptime-format-codes
	token_entry['expiresOn'] = _fromtimestamp(int(token_entry['expires_on']))\
	.strftime("%Y-%m-%d %H:%M:%S.%f")

Conversation

jiasli commented Nov 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing Guide

Uh oh!

yonzhan commented Nov 5, 2021

Uh oh!

kairu-ms Nov 8, 2021

Choose a reason for hiding this comment

Uh oh!

jiasli Nov 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jiasli Nov 8, 2021

Choose a reason for hiding this comment

Uh oh!

zhoxing-ms Nov 17, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jiasli Nov 17, 2021

Choose a reason for hiding this comment

Uh oh!

waylew-lexis commented Nov 15, 2021

Uh oh!

jiasli commented Nov 16, 2021

Uh oh!

jiasli Nov 17, 2021

Choose a reason for hiding this comment

Uh oh!

zhoxing-ms Nov 17, 2021

Choose a reason for hiding this comment

Uh oh!

jiasli Nov 17, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Maarc-D commented Nov 18, 2021

Uh oh!

yonzhan commented Nov 18, 2021

Uh oh!

jiasli commented Nov 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

jiasli commented Nov 5, 2021 •

edited

Loading

jiasli Nov 8, 2021 •

edited

Loading

zhoxing-ms Nov 17, 2021 •

edited

Loading

jiasli Nov 17, 2021 •

edited

Loading

jiasli commented Nov 19, 2021 •

edited

Loading