Skip to content

[Storage] _queue_client.py mypy fixes#29736

Merged
vincenttran-msft merged 12 commits intoAzure:feature/storage-queue-typingfrom
vincenttran-msft:vincenttran/mypy_queue_1
Apr 7, 2023
Merged

[Storage] _queue_client.py mypy fixes#29736
vincenttran-msft merged 12 commits intoAzure:feature/storage-queue-typingfrom
vincenttran-msft:vincenttran/mypy_queue_1

Conversation

@vincenttran-msft
Copy link
Member

@vincenttran-msft vincenttran-msft commented Apr 4, 2023

Current overarching issue to resolve:

  • Biggest issue is "Configuration" has no attribute xxxxx. The issue is that the type Configuration is from azure.core.configuration and these don't have the fields we are using (are we custom populating these?)
    • Let's try to make something like StorageConfiguration and then replace all instances of Configuration with this (make sure to inherit) lets slap this into models
  • Some methods we are defined as having a return type (i.e. QueueMessage). These go through alot of try-except blocks to make sure things are in place before returning, and so we don't have a default return statement and so Mypy complains. Currently I have put a return None to resolve the "missing return" statement, but now the types don't match. Not sure how to approach this
  • Swagger code needs to be updated for some return types (giving None when they should be non-None). So I'm thinking either the resolution is that we update generated code return types OR just throw an ignore on there

Original Mypy output:

  • azure\storage\queue_queue_client.py:105: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:106: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:117: error: Incompatible types in assignment (expression has type "bytes", variable has type "str") [assignment] ✔️
  • azure\storage\queue_queue_client.py:315: error: "None" has no attribute "name" [attr-defined] # response.name = self.queue_name but resopnse is self._client.queue.get_properties and generated is improperly typed to return None...
  • azure\storage\queue_queue_client.py:316: error: Incompatible return value type (got "None", expected "QueueProperties") [return-value] # Same issue as above, improperly typed generated function
  • azure\storage\queue_queue_client.py:381: error: "SignedIdentifier" has no attribute "iter" (not iterable) [attr-defined] ✔️ # cast
  • azure\storage\queue_queue_client.py:434: error: Incompatible types in assignment (expression has type "List[SignedIdentifier]", variable has type "Dict[str, AccessPolicy]") ✔️ [assignment] # Don't reassign processed List back to our Dict variable...
  • azure\storage\queue_queue_client.py:437: error: Argument "queue_acl" to "set_access_policy" of "QueueOperations" has incompatible type "Optional[Dict[str, AccessPolicy]]" ✔️ # Resolved by fixing above line
  • azure\storage\queue_queue_client.py:444: error: Missing return statement [return]
  • azure\storage\queue_queue_client.py:502: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:514: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:518: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:539: error: Missing return statement [return]
  • azure\storage\queue_queue_client.py:582: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:591: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:601: error: Missing return statement [return]
  • azure\storage\queue_queue_client.py:666: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:675: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:687: error: Missing return statement [return]
  • azure\storage\queue_queue_client.py:747: error: Item "str" of "Union[str, QueueMessage]" has no attribute "id" [union-attr] ✔️ # Resolved with a isinstance NOTE: Fixing this generated an issue wtih Optional[str] and str assignment, fixed with cast
  • azure\storage\queue_queue_client.py:748: error: Item "str" of "Union[str, QueueMessage]" has no attribute "content" [union-attr] ✔️
  • azure\storage\queue_queue_client.py:749: error: Item "str" of "Union[str, QueueMessage]" has no attribute "pop_receipt" [union-attr] ✔️
  • azure\storage\queue_queue_client.py:750: error: Item "str" of "Union[str, QueueMessage]" has no attribute "inserted_on" [union-attr] ✔️
  • azure\storage\queue_queue_client.py:751: error: Item "str" of "Union[str, QueueMessage]" has no attribute "expires_on" [union-attr] ✔️
  • azure\storage\queue_queue_client.py:752: error: Item "str" of "Union[str, QueueMessage]" has no attribute "dequeue_count" [union-attr] ✔️
  • azure\storage\queue_queue_client.py:765: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:777: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:781: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:796: error: Incompatible types in assignment (expression has type "Union[Any, datetime, None]", variable has type "datetime") [assignment] ✔️ # cast
  • azure\storage\queue_queue_client.py:797: error: Incompatible types in assignment (expression has type "Union[Any, datetime, None]", variable has type "datetime") [assignment] ✔️ # cast
  • azure\storage\queue_queue_client.py:798: error: Incompatible types in assignment (expression has type "Union[Any, int, None]", variable has type "int") [assignment] ✔️ # cast
  • azure\storage\queue_queue_client.py:799: error: Value of type "None" is not indexable [index] # response is wrongly typed in generated method as None
  • azure\storage\queue_queue_client.py:800: error: Value of type "None" is not indexable [index] # response is wrongly typed in generated method as None
  • azure\storage\queue_queue_client.py:806: error: Missing return statement [return]
  • azure\storage\queue_queue_client.py:852: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:860: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:937: error: Item "str" of "Union[str, QueueMessage]" has no attribute "id" [union-attr] ✔️ # Resolved with a isinstance NOTE: Fixing this generated an issue wtih Optional[str] and str assignment, fixed with cast
  • azure\storage\queue_queue_client.py:938: error: Item "str" of "Union[str, QueueMessage]" has no attribute "pop_receipt" [union-attr] ✔️

New Mypy output:

  • azure\storage\queue_queue_client.py:106: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:107: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:317: error: "None" has no attribute "name" [attr-defined]
  • azure\storage\queue_queue_client.py:318: error: Incompatible return value type (got "None", expected "QueueProperties") [return-value]
  • azure\storage\queue_queue_client.py:503: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:515: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:519: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:538: error: Incompatible return value type (got "None", expected "QueueMessage") [return-value]
  • azure\storage\queue_queue_client.py:584: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:593: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:669: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:678: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:688: error: Incompatible return value type (got "None", expected "ItemPaged[QueueMessage]") [return-value]
  • azure\storage\queue_queue_client.py:769: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:781: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:785: error: "Configuration" has no attribute "message_encode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:803: error: Value of type "None" is not indexable [index]
  • azure\storage\queue_queue_client.py:804: error: Value of type "None" is not indexable [index]
  • azure\storage\queue_queue_client.py:808: error: Incompatible return value type (got "None", expected "QueueMessage") [return-value]
  • azure\storage\queue_queue_client.py:857: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:865: error: "Configuration" has no attribute "message_decode_policy" [attr-defined]
  • azure\storage\queue_queue_client.py:873: error: Incompatible return value type (got "None", expected "List[QueueMessage]") [return-value]

@ghost ghost added the Storage Storage Service (Queues, Blobs, Files) label Apr 4, 2023
@azure-sdk
Copy link
Collaborator

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-storage-queue

@vincenttran-msft vincenttran-msft merged commit f9290ee into Azure:feature/storage-queue-typing Apr 7, 2023
@vincenttran-msft vincenttran-msft deleted the vincenttran/mypy_queue_1 branch April 7, 2023 23:17
vincenttran-msft added a commit that referenced this pull request Nov 10, 2023
* [Storage] Update typing for `queue_service_client` (Queue) (#28949)

* qsc done sync&async

* Remove #type ignore, fix dict()

* Fix indents

* Fix indent again :D

* [Storage] Update typing for `queue_client` (Queue) (#28952)

* sync 80% done need *, logic

* First attempt at *,

* Async + unify sync spacing

* Fix CI

* Revert recordings

* PR feedback

* Revert recording, fix indents

* indent, again :D

* Fix content docstring

* [Storage] Update typing for miscellaneous files (queue) (#29173)

* [Storage] `_queue_client.py` mypy fixes (#29736)

* First draft MyPy

* Mypy config fix + just Config Mypy left

* Fully done w/ custom StorageConfiguration

* Pylint

* Double import

* Fix cast

* PR feedback

* Unused imports

* Pylint

* Configuration errors in async BE GONE

* Fix tests that assert against _config

* [Storage] _queue_client_async.py mypy fixes  (#29824)

* Init done for _queue_client

* Fix async calling into sync

* Back to the drawing board

* Finished __init__

* Fix failing tests

* queue_client done and decoupled

* Queue specific files done

* Shared, some base_client leftovers, some model leftovers

* 3 _model 1 response_handlers

* Mypy congrats, time to fight pylint & CI :D

* unused import asyncio

* New PyLint rules fixes

* Pylint 2

* Fix failing test cases

* Be gone test failures

* Finally found root cause of infinite loop

* All encryption tests passing locally, 32 pylint errors remaining

* All tests passing locally, mypy green locally

* Pylint

* Encryption feedback done

* Shared, sample, and lint output left

* More comments addressed, need bc_async, policies, and samples + output

* Just samples, added more helpers

* More feedback

* More fixes

* Lint

* Feedback 2

* CI

* Fix weird invisible unicode char

* No mypy errors hopefully no CI

* Pylint and CI, but now hinting issues

* Fix Lint which should fix CI

* Attempted overload

* Just hanging encryption and message_encoding Qs, fix import ordering

* Addressed encryption feedback

* Feedback, need base client decouple, need CI passing

* Fixing CI

* CI

* CI

* Ignores :(

* More fixes

* CI

* Pylint again

* Adjust typehint

* Try SO workaround 41207128

* Reimport Pylint

* First round of feedback

* Fix CorsRule to_generated method

* Fix PyLint & test failures

* Feedback

* Revert moving out sas token helper, causes circular dependency

* Pylint

* Typing feedback

* Fix CI

* Use QueueMessage ctor

* Pylint

* Privatize CorsRule method

* Disable for protected access

* Flip on mypy

* Missed configuration value parsing

* Double-spaced

* Typo

* Last minute nits

* Extra newline in encryption
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Storage Storage Service (Queues, Blobs, Files)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants