Revoked ATs are returned after re-login

[jiasli]

Describe the bug

Revoked ATs are returned after re-login.

To Reproduce

1. Log in with auth code flow with organizations tenant and https://management.core.windows.net/ resource.

2. Get an AT explicitly with tenant ID 54826b22-38d6-4fb2-bad9-b7b93a3e9c5a.

   At this point, in token cache the AccessToken contains 2 entries:

   • AT1: 14387eca3-bb26-4047-922a-6452cae1e9c7.54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-login.microsoftonline.com-accesstoken-04b07795-8ddb-461a-bbee-02f9e1bf7b46-organizations-https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default
   • AT2: 4387eca3-bb26-4047-922a-6452cae1e9c7.54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-login.microsoftonline.com-accesstoken-04b07795-8ddb-461a-bbee-02f9e1bf7b46-54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default

3. Revoke the session with

   az rest -m POST -u https://graph.microsoft.com/v1.0/me/revokeSignInSessions
   

   Doing so adds a new AT for https://graph.microsoft.com/:

   • AT3: 4387eca3-bb26-4047-922a-6452cae1e9c7.54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-login.microsoftonline.com-accesstoken-04b07795-8ddb-461a-bbee-02f9e1bf7b46-54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-email openid profile https://graph.microsoft.com//auditlog.read.all https://graph.microsoft.com//directory.accessasuser.all https://graph.microsoft.com//group.readwrite.all https://graph.microsoft.com//user.readwrite.all https://graph.microsoft.com//.default

4. Re-login with auth code flow with organizations.

   At this point, in token cache the AccessToken contains 3 entries and only AT1 is updated:

   • AT1 (Updated): 14387eca3-bb26-4047-922a-6452cae1e9c7.54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-login.microsoftonline.com-accesstoken-04b07795-8ddb-461a-bbee-02f9e1bf7b46-organizations-https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default
   • AT2 (Old): 4387eca3-bb26-4047-922a-6452cae1e9c7.54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-login.microsoftonline.com-accesstoken-04b07795-8ddb-461a-bbee-02f9e1bf7b46-54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default
   • AT3 (Old): 4387eca3-bb26-4047-922a-6452cae1e9c7.54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-login.microsoftonline.com-accesstoken-04b07795-8ddb-461a-bbee-02f9e1bf7b46-54826b22-38d6-4fb2-bad9-b7b93a3e9c5a-email openid profile https://graph.microsoft.com//auditlog.read.all https://graph.microsoft.com//directory.accessasuser.all https://graph.microsoft.com//group.readwrite.all https://graph.microsoft.com//user.readwrite.all https://graph.microsoft.com//.default

5. Get an AT explicitly with tenant ID 54826b22-38d6-4fb2-bad9-b7b93a3e9c5a -> the old revoked AT2 is returned

6. Use AT2 with ARM -> ❌

Expected behavior

MSAL should purge all ATs if RT is updated for this user.

What you see instead

Old ATs are still preserved and returned by MSAL.

The MSAL Python version you are using

1.10.0

Additional context

To solve this issue I have to manually delete the MSAL cache to get all ATs purged.

[rayluo]

MSAL should purge all ATs if RT is updated for this user.

This was not a required behavior in the design goal. And, in general, MSAL will not know an AT in cache has been revoked, because the revocation could even be performed by another app which does not necessarily share the same token cache.

If an AT returned by old_at = app.acquire_token_silent(...) is not usable for whatever reason:

(1) The application is currently expected to call new_at = app.acquire_token_silent(..., force_refresh=True) again. But even that does not always result in the old_at being purged (more on this in a subsequent message), which brings me to my next point:
(2) I used to contemplate it would be logically meaningful to have a new acquire_token_silent(..., by_the_way_please_purge_this_useless_access_token=old_at) behavior.
(3) It is also possible to wrap behavior 1 and the future behavior 2 (if we'd go that route) into a higher-level helper/api.

Do you think the repro step 3 "revoke a session" is frequent enough to justify the changes of (2) and/or (3)?

Besides, I think you might not run into all above, at least not as easily, if all the repro steps stay consistent with same tenant (using either "organizations" or tenant_id, but not back and forth).

[jiasli]

The application is currently expected to call new_at = app.acquire_token_silent(..., force_refresh=True) again. But even that does not result in the old_at being purged, which brings me to my next point:

Really? Wouldn't calling force_refresh=True reach the same code path as when the AT expires? The old AT should be overwritten, right?

microsoft-authentication-library-for-python/msal/application.py

Lines 842 to 843 in 96614ec

	if expires_in < 5*60:
	continue # Removal is not necessary, it will be overwritten

I used to contemplate it would be logically meaningful to have a new acquire_token_silent(..., by_the_way_please_purge_this_useless_access_token=old_at) behavior.

Ugh, this is a little bit ugly.

Besides, I think you might not run into all above, at least not as easily, if all the repro steps stay consistent with same tenant (using either organizations or tenant_id, but not back and forth).

Unfortunately, this is indeed how Azure CLI works, so Azure CLI can easily run into such situation and can't recover.

Without specifying --tenant, az login will

1. Log in with auth code flow with organizations as tenant.
2. Once the RT an AT are retrieved, use the AT to call Tenants - List ARM REST API to list all tenants the user has access to, say T1, T2.
3. Use the RT retrieved previously to get new ATs specific to T1, T2 and call Subscriptions - List to find out all subscriptions under those tenants.

Here comes the tricky part:

1. In case of Continuous Access Evaluation, those ATs are revoked.
2. Now even I run az login again, only the AT for organizations is updated and MSAL will continue to return revoked ATs for T1, T2, making subsequence ARM invocation to fail.
3. Without deleting the whole MSAL cache, az login can't recover.

In my mind, asking MSAL to purge ATs may not be a good idea. If auth code flow is re-initiated, it is supposed to start everything from scratch and forget about all previous outdated information.

Of course Azure CLI can purge the MSAL cache before az login, but other MSAL users may not know this pitfall, like Azure Identity which doesn't have an API to purge MSAL cache at all.

This problem doesn't occur in ADAL because we never enabled CAE (client_capabilities CP1). Even we call revokeSignInSessions, those ATs in the token cache will still be valid until they expire.

[rayluo]

The application is currently expected to call new_at = app.acquire_token_silent(..., force_refresh=True) again. But even that does not result in the old_at being purged, which brings me to my next point:

Really? Wouldn't calling force_refresh=True reach the same code path as when the AT expires? The old AT should be overwritten, right?

OK, I should be more precise to say that, acquire_token_silent(...) itself does not explicitly remove an old AT, the old AT will usually be automatically overwritten when/if a new AT with same scopes is issued. This mechanism works fine for expired AT because an un-purged but expired AT will not be utilized anyway.

In CAE case, an AT can be still within 1-hour lifespan but already revoked. In such case, ideally a more reliable purge mechanism should be in place. The acquire_token_silent(..., by_the_way_please_purge_this_useless_access_token=old_at) idea may look ugly because of that long parameter name so we could of course use a shorter parameter (how about acquire_token_silent(..., exclude=old_at)?). Regardless of the naming, my point is there needs to be a way to convey your need "hey librarian, I want to borrow any book about time travelling. Any recommendation? By the way, I already borrow the first book from your top 10 list, The Time Machine of H.G. Wells, I do not like it very much, so please do not recommend that again."

All that being said, have your tried the low hanging fruit already? Pattern "(1) The application is currently expected to call new_at = app.acquire_token_silent(..., force_refresh=True) again" is needed anyway, and it might just work if your new request uses same set of scopes.

We still want to work with you to come up with a solution. We will try our best to avoid having downstream apps need to nuke the entire token cache. (That is even uglier than acquire_token_silent(..., exclude=old_at), in my opinion.)

[jiasli]

force_refresh=True

have your tried the low hanging fruit already? Pattern "(1) The application is currently expected to call new_at = app.acquire_token_silent(..., force_refresh=True) again" is needed anyway, and it might just work if your new request uses same set of scopes.

Not really.

1. For az login, we can definitely do this for ARM operations, and it actually has the exact same effect as removing all ARM ATs of this user first.
2. But later on, for other resources (e.g. MS Graph), the old AT will still be returned, because we don't want to use force_refresh=True for every get_token invocation. Then we will need to try&error (shown below).

exclude=old_at

The acquire_token_silent(..., by_the_way_please_purge_this_useless_access_token=old_at) idea may look ugly because of that long parameter name so we could of course use a shorter parameter (how about acquire_token_silent(..., exclude=old_at)?).

My concern is not about the long name, but the try&error approach which certainly adds extra complexity to the downstream app's code path.

The downstream app must

1. try the old_at first at the resource (e.g. ARM, MS Graph, etc.)
2. find out the AT has been revoked
3. call acquire_token_silent(..., exclude=old_at)
4. use the new_at on the resource again

The extra step 1 will adds performance penalty. To make things worse, if the user can access 10 tenants, the above try&error routine will be called 10 times!

Conclusion

In a word, for either approach:

• acquire_token_silent(..., force_refresh=True)
• acquire_token_silent(..., exclude=old_at)

the downstream app must be aware that "there is something old, something wrong within the MSAL cache", we have to deal with it.

We will try our best to avoid having downstream apps need to nuke the entire token cache. (That is even uglier than acquire_token_silent(..., exclude=old_at), in my opinion.)

Not really nuke the entire token cache. remove_account (purging the AT (not only ARM) and RT for specific user) will be the best approach if auth code flow is re-initiated. Not calling remove_account in that case will also make the following call of MSAL non-idempotent - if I start everything over, MSAL should behave the same, no matter whether there is an existing token cache or not.

Azure CLI can definitely call remove_account by itself, but other downstream apps like Azure Identity may hit this intriguing pitfall. So MSAL should be the best place to call remove_account. (My 2 cents.)

[rayluo]

This is one of the most difficult thing in computer science - (token) cache invalidation. :-) Conceptually, the consumer of the cache would have to know when and how to invalidate a cached item. We could, however, come up with higher level API for that.

A short-term workaround would be Azure CLI to call remove_account() after "step 3: Revoke the session az rest -m POST -u https://graph.microsoft.com/v1.0/me/revokeSignInSessions". MSAL does not have that signal to trigger remove_account(). MSAL would not do remove_account() for every init_auth_code_flow().

[jiasli]

A short-term workaround would be Azure CLI to call remove_account() after "step 3: Revoke the session az rest -m POST -u https://graph.microsoft.com/v1.0/me/revokeSignInSessions".

This doesn't sound like a plan, because az rest is a generic command. It doesn't care what the call is. The ATs may also be revoked by other tools, like Azure Portal.

MSAL would not do remove_account() for every init_auth_code_flow().

When to call remove_account

During az login, CLI can only know the username after the AT and RT are update:

• Before finishing auth code flow, CLI doesn't know the username
• After finishing auth code flow, calling remove_account for username will remove the new AT and RT

So MSAL seems to be the only feasible place to call remove_account once the ID Token is returned, before updating AT and RT. 🤔

Otherwise, we have to ☢ the whole MSAL cache indeed.

[jiasli]

MSAL.NET is more resilient but still affected

This issue doesn't reveal in MSAL.NET easily because MSAL.NET doesn't save an AT for organizations at all (#341).

In a single-tenant scenario with MSAL.NET, there is only one AT with tenant_id (realm) as GUID-> there is no back and forth between organizations and tenant_id -> this issue will not be triggered.

However, in a multi-tenant scenario, this issue can still happen for MSAL.NET.

[jiasli]

Mitigation with silent reauth

As tested, after reinitiating auth code flow, this issue can be overcome following these steps:

1. MSAL returns a revoked AT
2. Using the revoked AT on ARM -> 401 failure with claims challenge ❌
3. Do silent reauth with claims challenge (same effect as force_refresh) and the new RT -> Get a new AT
4. Use the new AT on ARM -> 200 success ✔

However, this relies on the client to support silent reauth. It doesn't work in many CLI commands:

• commands still using Track 1 SDK which has no silent reauth support, like those under az group, az ad
• commands calling REST API directly, like those related to private links
• az rest

In a word, returning revoked ATs is a heavy burden for the client to handle.

[jiasli]

Steps to repro with Azure CLI

First install Azure CLI CAE b3 edge build following Azure/azure-cli#17612 (comment).

Then execute following commands:

> az login

# Revoke session, including RTs and ATs
> az rest -m POST -u https://graph.microsoft.com/v1.0/me/revokeSignInSessions

# Wait several minutes for the session revocation to propagate.

# A failed command using Track 2 SDK. Make sure you see an error.
> az storage account list

# A failed command using Track 1 SDK. Make sure you see an error.
> az group list

In #335 (comment) I mentioned az login will fail and can't recover because it can't silently reauth with claims_challenge. It now works because azure-cli-core now vendors a Track 2 subscription SDK which supports CAE silent reauth.

# Log in again with the same account.
> az login --debug

# It is supposed to see a 401 from ARM because MSAL provided a revoked access token
azure.core.pipeline.policies._universal: Request URL: 'https://eastus2euap.management.azure.com/subscriptions?api-version=2019-11-01'
azure.core.pipeline.policies._universal: Request method: 'GET'

azure.core.pipeline.policies._universal: Response status: 401
azure.core.pipeline.policies._universal: Response headers:
azure.core.pipeline.policies._universal:     'WWW-Authenticate': 'Bearer authorization_uri="https://login.windows.net/", error="invalid_token", error_description="User session has been revoked", claims="eyJhY2Nlc3NfdG9rZW4iOnsibmJmIjp7ImVzc2VudGlhbCI6dHJ1ZSwgInZhbHVlIjoiMTYxNzk0OTcwNSJ9fX0="'
azure.core.pipeline.policies._universal: Response content:
azure.core.pipeline.policies._universal: {"error":{"code":"AuthenticationFailed","message":"Authentication failed."}}
cli.azure.cli.core.credential: CredentialAdaptor.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={'claims': '{"access_token":{"nbf":{"essential":true, "value":"1617949705"}}}'}

# Silent reauth with claims
azure.core.pipeline.policies._universal: Request URL: 'https://login.microsoftonline.com/54826b22-38d6-4fb2-bad9-b7b93a3e9c5a/oauth2/v2.0/token'
azure.core.pipeline.policies._universal: Request method: 'POST'
azure.core.pipeline.policies._universal: Request body:
azure.core.pipeline.policies._universal: {'client_id': '04b07795-8ddb-461a-bbee-02f9e1bf7b46', 'grant_type': 'refresh_token', 'client_info': 1, 'claims': '{"access_token": {"nbf": {"essential": true, "value": "1617949705"} ...

az rest will fail because there is no silent reauth support

> az rest -m POST -u https://graph.microsoft.com/v1.0/me/revokeSignInSessions --debug

cli.azure.cli.core.util: Request URL: 'https://graph.microsoft.com/v1.0/me/revokeSignInSessions'
cli.azure.cli.core.util: Request method: 'POST'

cli.azure.cli.core.util: Response status: 401
cli.azure.cli.core.util: Response headers:
cli.azure.cli.core.util:     'WWW-Authenticate': 'Bearer realm="", authorization_uri="https://login.microsoftonline.com/common/oauth2/authorize", client_id="00000003-0000-0000-c000-000000000000", error="insufficient_claims", claims="eyJhY2Nlc3NfdG9rZW4iOnsibmJmIjp7ImVzc2VudGlhbCI6dHJ1ZSwgInZhbHVlIjoiMTYxNzk0OTgyMSJ9fX0="'
cli.azure.cli.core.util: Response content:
cli.azure.cli.core.util: {
  "error": {
    "code": "InvalidAuthenticationToken",
    "message": "Continuous access evaluation resulted in claims challenge with result: InteractionRequired and code: TokenIssuedBeforeRevocationTimestamp",
    "innerError": {
      "date": "2021-04-09T06:30:21",
      "request-id": "83512124-7723-46a4-bbae-62c5dfe446a9",
      "client-request-id": "83512124-7723-46a4-bbae-62c5dfe446a9"
    }
  }
}

knack.util.CLIError: Unauthorized({
  "error": {
    "code": "InvalidAuthenticationToken",
    "message": "Continuous access evaluation resulted in claims challenge with result: InteractionRequired and code: TokenIssuedBeforeRevocationTimestamp",
    "innerError": {
      "date": "2021-04-09T06:30:21",
      "request-id": "83512124-7723-46a4-bbae-62c5dfe446a9",
      "client-request-id": "83512124-7723-46a4-bbae-62c5dfe446a9"
    }
  }
})