[BUG Bash] _key_vault_client.py ImportError: cannot import name 'Deserializer'

[dakooka]

Describe the bug
We are using some python modules in our python scripts, and when gitlab-runner is trying to run test for this python script we are getting error message:
from msrest import Deserializer, Serializer
ImportError: cannot import name 'Deserializer'

requirements.txt
**We are also tried to use the last versions of the Python Azure SDK and clear the caches for gitlab-runner but the issue is still present **

PyYAML
colorama
azure-common~=1.1
azure-identity>=1.0.1
azure-keyvault-secrets~=4.0
azure-mgmt-keyvault~=1.0
azure-mgmt-storage~=2.0
azure-mgmt-resource~=4.0
azure-mgmt-subscription~=0.2.0
python-terraform
pyodbc
pyhcl
requests

Exception or Stack Trace

Traceback (most recent call last):
File "tools/manage.py", line 11, in
import preterraform
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/tools/preterraform.py", line 4, in
from azure.mgmt.storage import models
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/azure/mgmt/storage/init.py", line 12, in
from .storage_management_client import StorageManagementClient
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/azure/mgmt/storage/storage_management_client.py", line 12, in
from msrest.service_client import SDKClient
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/msrest/init.py", line 28, in
from .configuration import Configuration
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/msrest/configuration.py", line 38, in
from .universal_http.requests import (
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/msrest/universal_http/requests.py", line 41, in
from oauthlib import oauth2
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/oauthlib/oauth2/init.py", line 8, in
from .rfc6749.clients import (
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/oauthlib/oauth2/rfc6749/clients/init.py", line 9, in
from .backend_application import BackendApplicationClient
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/oauthlib/oauth2/rfc6749/clients/backend_application.py", line 10, in
from .base import Client
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/oauthlib/oauth2/rfc6749/clients/base.py", line 11, in
import secrets
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/tools/secrets.py", line 6, in
from azure.keyvault import secrets as client_secrets
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/azure/keyvault/secrets/init.py", line 5, in
from ._models import DeletedSecret, KeyVaultSecret, KeyVaultSecretIdentifier, SecretProperties
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/azure/keyvault/secrets/_models.py", line 5, in
from ._shared import parse_key_vault_id
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/azure/keyvault/secrets/_shared/init.py", line 13, in
from .client_base import KeyVaultClientBase
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/azure/keyvault/secrets/_shared/client_base.py", line 12, in
from .._generated import KeyVaultClient as _KeyVaultClient
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/azure/keyvault/secrets/_generated/init.py", line 9, in
from ._key_vault_client import KeyVaultClient
File "/home/gitlab-runner/builds/kgZdCMGe/0/smart-azure/terraform-profiles/.venv-test/lib/python3.6/site-packages/azure/keyvault/secrets/_generated/_key_vault_client.py", line 17, in
from msrest import Deserializer, Serializer
ImportError: cannot import name 'Deserializer'

Setup (please complete the following information):

• Python Version: Python 3.6
• azure-keyvault-secrets-4.3.0
• msrest 0.6.21
• azure_core-1.21.1
• azure_mgmt_subscription-3.0.0
• azure_mgmt_keyvault-9.3.0
• azure_identity-1.7.1
• azure_mgmt_storage-19.0.0
• azure_mgmt_resource-20.1.0
• azure_common-1.1.27
• pip 21.3.1
• virtual environment CPython3.6.9.final.0-64

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• Bug Description Added
• Repro Steps Added
• Setup information Added

[swathipil]

Hi @dakooka - thanks for reporting the issue! Would you be able to add the snippet of code that causes this issue? Thanks!

[dakooka]

Hi @swathipil ,

Thank You for Your quick reaction

Unfortunately, there is nothing to add, since it does not work from the model import line.
from azure.mgmt.storage import models

On the bottom of the traceback (which I added also) we have found that the problem is occurring with
entire keyvaults module when it's trying to import Deserializer and Serializer
from msrest import Deserializer, Serializer
in _key_vault_client.py and _operations_mixin.py

Deserializer and Serializer are not used in these files at all

Next module which is failing is azure.mgmt for keyvault

Imported modules

from azure.common import credentials
from azure.mgmt import resource
from azure.mgmt import subscription
from azure.mgmt import storage
from azure.mgmt import keyvault

Snipped of code:

return keyvault.KeyVaultManagementClient(credentials=self.creds, subscription_id=self.subscription)

Error message

name 'Serializer' is not defined

It should failed for _key_vault_management_client.py from azure.mgmt module -> Serializer and Derializer are used there

PS Small remark
we see one pull request in Your repo [Key Vault] Drop 2.7, 3.6 support (#22367)
But a few days later the support was added back with the following pull request
[Key Vault] Add back 3.6 support (#22556)
Maybe it could broke module for using with python 3.6 version, I don't know

[mccoyp]

Hi @dakooka, thank you for opening an issue! This seems like a packaging error. msrest 0.6.21 does expose Serializer and Deserializer, and I'm able to successfully run

from msrest import Serializer, Deserializer

and

import azure.keyvault.secrets

with msrest==0.6.21 and azure-keyvault-secrets==4.3.0 installed, on Python 3.6.

Could I ask you to run pip freeze in the environment that had this issue, and share the output?