* [EventHub] basic receive event scenario with pyamqp (#19748)
* initial changes for receiving
* undo __init__ aio
* vendor
* remove c/pyx files
* adams comments
* [EventHubs & AMQP Python] Send Port (#19745)
* draft send port
* copy and paste code changes in amqp
* simple stress test scripts for sending and receiving
* review feedbacks
* [EventHubs&AMQP Python] Port amqp send large message (#19937)
* port amqp send large message
* send perf test in parallel
* use context manager for executor
* add throughput
* improve test code
* update test matrix
* update test
* handle server busy
* fix timeout
* precision to 2 decimal points
* update pyamqp changes and update tests
* [EventHubs] Pure Python AMQP Sycn Implementation Integration (#22397)
* copy amqp changes
* eh python amqp integration
* fix time unit
* rename module pyamqp to _pyamqp
* more pyamqp to _pyamqp
* simplify todo
* [EH Pyproto] Release preparation (#22433)
* cherry pick changes
* update docs
* cherry pick fixed retry PR
* minor fix
* fix mypy, pylint, brokenlink
* update doc
* opt out mypy/pylint/api stub
* try opt out checkpointstore in ci and test
* fix
* more fixes
* furuther opt out tests
* update tests
* bump version
* fix __str__
* add test play holder
* ignore azure checkpoinstore aio in ci
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* update readme to drop uamqp
* revert async tests
* revert aio module in eventhub
* [EH Pyproto] Async support (#22957)
* async port
* add scripts for tests
* update async perf test scripts
* fix test scripts
* amqp implementation update + eh update + sync perf test scripts
* update pyamqp and eh async impl and test scripts
* update pyamqp async impl
* fix bug
* fix pyamqp transport ssl setting and asyncio exception module import
* use ensure future for 3.6
* update token generation to return bytes to avoid breaking changes
* update docs
* Increment version for eventhub releases (#22994)
Increment package version after release of azure-eventhub
* [EH Pyproto] Async recv perf improvement (#23122)
* stop spawning too much coroutines
* improve send
* async recv perf improvement
* async perf improve
* update version
* align with sync imple
* update method name
* remove redundant except catch
* [EH Pyproto] Release updates (#23349)
* update docs
* add todo
* Increment version for eventhub releases (#23420)
Increment package version after release of azure-eventhub
* AMQP websocket implementation (#23722)
* Initial implementation
* http proxy support
* change impl
* more changes
* working sol
* async impl
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* more changes
* sasl mixin
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/message.py
* refactor
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* oops
* comments
* comment
* Apply suggestions from code review
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* comments
* changes
* async test
* rasie
* lint
* changelog
* version
* comments
* move path to EH
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* Revert "AMQP websocket implementation (#23722)" (#24344)
This reverts commit 0123f4d.
* AMQP websocket implementation (#24345)
* Initial implementation
* http proxy support
* change impl
* more changes
* working sol
* async impl
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* more changes
* sasl mixin
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/message.py
* refactor
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* oops
* comments
* comment
* Apply suggestions from code review
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* comments
* changes
* async test
* rasie
* lint
* changelog
* version
* comments
* move path to EH
* Fix typo
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* [EventHub] basic receive event scenario with pyamqp (#19748)
* initial changes for receiving
* undo __init__ aio
* vendor
* remove c/pyx files
* adams comments
* [EventHubs & AMQP Python] Send Port (#19745)
* draft send port
* copy and paste code changes in amqp
* simple stress test scripts for sending and receiving
* review feedbacks
* [EventHubs&AMQP Python] Port amqp send large message (#19937)
* port amqp send large message
* send perf test in parallel
* use context manager for executor
* add throughput
* improve test code
* update test matrix
* update test
* handle server busy
* fix timeout
* precision to 2 decimal points
* update pyamqp changes and update tests
* [EventHubs] Pure Python AMQP Sycn Implementation Integration (#22397)
* copy amqp changes
* eh python amqp integration
* fix time unit
* rename module pyamqp to _pyamqp
* more pyamqp to _pyamqp
* simplify todo
* [EH Pyproto] Release preparation (#22433)
* cherry pick changes
* update docs
* cherry pick fixed retry PR
* minor fix
* fix mypy, pylint, brokenlink
* update doc
* opt out mypy/pylint/api stub
* try opt out checkpointstore in ci and test
* fix
* more fixes
* furuther opt out tests
* update tests
* bump version
* fix __str__
* add test play holder
* ignore azure checkpoinstore aio in ci
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* update readme to drop uamqp
* revert async tests
* revert aio module in eventhub
* [EH Pyproto] Async support (#22957)
* async port
* add scripts for tests
* update async perf test scripts
* fix test scripts
* amqp implementation update + eh update + sync perf test scripts
* update pyamqp and eh async impl and test scripts
* update pyamqp async impl
* fix bug
* fix pyamqp transport ssl setting and asyncio exception module import
* use ensure future for 3.6
* update token generation to return bytes to avoid breaking changes
* update docs
* Increment version for eventhub releases (#22994)
Increment package version after release of azure-eventhub
* [EH Pyproto] Async recv perf improvement (#23122)
* stop spawning too much coroutines
* improve send
* async recv perf improvement
* async perf improve
* update version
* align with sync imple
* update method name
* remove redundant except catch
* [EH Pyproto] Release updates (#23349)
* update docs
* add todo
* Increment version for eventhub releases (#23420)
Increment package version after release of azure-eventhub
* AMQP websocket implementation (#23722)
* Initial implementation
* http proxy support
* change impl
* more changes
* working sol
* async impl
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* more changes
* sasl mixin
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/message.py
* refactor
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* oops
* comments
* comment
* Apply suggestions from code review
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* comments
* changes
* async test
* rasie
* lint
* changelog
* version
* comments
* move path to EH
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* Revert "AMQP websocket implementation (#23722)" (#24344)
This reverts commit 0123f4d.
* AMQP websocket implementation (#24345)
* Initial implementation
* http proxy support
* change impl
* more changes
* working sol
* async impl
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* more changes
* sasl mixin
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/message.py
* refactor
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_transport.py
* oops
* comments
* comment
* Apply suggestions from code review
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* comments
* changes
* async test
* rasie
* lint
* changelog
* version
* comments
* move path to EH
* Fix typo
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* remove extra SR related code
* update docs + type hints
* fixing failing tests
* [EventHubs] merge working websocket changes to feature branch (#24444)
* adam's working changes
* Adding back Rakshith's websocket changes (#24410)
* Adding back Rakshith's sync websocket changes
* fix async send and receive
* fix transport bugs
* add websocket to dev reqs + async fix hostname
* thank you kashif
* fix tests + turn on websocket tests
* update consumer test timing
* fix merge bugs + remove pyamqp specific tests
* update sleep time in test
* enable live test for ws receive
* fix to create Batch properly
* [eventhub] Websocket timeout error exception thrown (#24504)
* sync websocket timeout to operationtimeout, not changing other transport types just yet
* async websocket to operation timeout
* upstream
* default timeout 1.0
* default timeout 1
* throwing a socket timeout, operationtimeout was throwing out a real error
* replacing socket.timeout with TimeoutError, added into except statments as well
* timeout is inherited from oserror, dont need both
* test timeoutexception throwing errors
* deafult timeout to 1
* [eventhub] websocket default timeout fix (#24565)
* websocket timeout fix
* timeout interval for both ssl and webscoket
* [eventhub] Custom Endpoint (#24505)
* sync ce
* async ce
* add string ending
* only pass to transport
* running into same recieve issue with sync
* fixing async - needs to pass to sasl
* remove logger
* stopping here
* adding prefix to fix sample
* add in prefetch
* fixing transport remove print
* host being overriden
* removing trace
* fix to use url async
* aligning sync/async pattern
* removing uneeded hostname switch
* string formatting
* changelog
* adding docstrings for supported events
* pr comments refactoring sync
* mirroring on async
* pr comment docstring
* removing import
* missing _
* missing ssl
* if no port given, we use default set in config
* async of same ^
* add default port in connection stage if port is none
* adding in docstring to cliet/connection string constructor
* custom_endpoint_address in client base async to match sync
* fix import on websocket test
* fix import 2
* skipping tests
* removing import
* pytest.mark
* [EventHubs] pyproto - update release date + docs (#24723)
* add async doc rst file
* Increment version for eventhub releases (#24753)
Increment package version after release of azure-eventhub
* changes to update status (#25024)
* updating codeowners file in pyproto feature
* [Eventhub] pyamqp prefetch fix (#24890)
* prefetch fix
* adding async - sorry!
* async
* Use --no-cone in pipeline sparse checkout script (#25165) (#25208)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* [AMQP Python] Eventhub Pyamqp tests (#24895)
* starting tests
* updates to websocket sync
* moving around format - unittest and live test
* live test + unittests starting
* websocket async passing
* eol
* assert not return
* assert not return
* fixed assert
* auth tests
* auth unittest pyamqp
* replicating uamqp tests
* keep_alive_thread
* skip for now - no keep alive
* pickle/deepcopy, might not want to keep all
* stopping here for now - need tls on rabbitmq
* cleaning up tests - pickle
* removing and editing uneeded tests
* removing unused test
* added receive amqp tests
* exceptions with pytest.raises, not live
* moving around tests
* testing mgmt calls like _start_producer
* Use --no-cone in pipeline sparse checkout script (#25165)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* unused imports
Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* [Pyamqp] Fix network logging trace in client_base (#25218)
* pass in right kwarg for network tracing
* remove client changes. Another PR
* reverting link credit for now (#25310)
* [Pyamqp] Pyampq debug build Linkedin (#25296)
* enhanced logging for linkedin
* stuff
* fixes
* minor sample fix
* Changelog
* remove unused imports
* fix formatting changes
* change debug level
* update version info
* update changelog
* fix sample
* fix logging message for empty access token
* Increment version for eventhub releases (#25320)
Increment package version after release of azure-eventhub
* removing duplicate (#25321)
* add async unit tests (#25396)
* reset logging level (#25588)
* [Pyamqp] Remember Proxy Params (#25564)
* fix to keep proxy params
* async changes for proxy
* unit tests
* changes
* more changes
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* [Pyamqp] Intial TODOS Clean Up (#25630)
* set default SSL version
* dont need it for EH & SB
* wont impact us, check mtg notes
* decode error wont affect us
* address in SB PR
* change language on logger
* it does close socket
* no other closes needed
* keep for better tracking
* close w/ error when max frame size is invalid
* detach links on session end
* clean up links on session outgoing_end
* reject link by detaching
* uncomment logging for later review
* reject the link that was set in the try
* [Pyamqp] test fixes for pipeline (#25749)
* test fixes for pipeline
* connect to EHerror
* [PyAMQP] Stress testing reform (#25770)
* reform stress test
* reform stress test
* updating stress test format
* can specify azure_identity on producer
* epoch level sync
* removing commands
* aligning sync/async
* pyamqp logging
* uamqp to pyamqp
* import remove
* fixing deploy commands
* values file
* checking eng/ file resources
* remove version
* moving location of this folder
* imagepullpolicy, azure_identity for consumer
* [PyAMQP] Updating pyamqp with SB changes (#25804)
* pyamqp diff from anna's branch
* mgmt_request returns code, response, desc
* mgmt_request returns code, response, desc async
* handle error being thrown
* handle error being thrown 2
* handle error being thrown 3
* Fixed error path
* fix error path
Co-authored-by: Anna Tisch <antisch@microsoft.com>
Co-authored-by: Kashif Khan <361477+kashifkhan@users.noreply.github.com>
* removing buff producer to add back in later
* update cspell
* fix uppercase link in doc/dev/issues/resolve_issues_effectively
* [PyAMQP] Connections TODO (#26018)
* protocol errors
* fix formatting
* [PyAMQP] Kashif Client refactoring changes (#25451)
Linked to
link #22051
for a TODO about typing in docstrings
* need to use pyamqp (#25895)
* [EventHubs] add amqp switch support (#25965)
This PR is for adding switch support to the `feature/eventhub/pyproto` changes including the uamqp switch from current `main`.
fixes #21246
Addressing Anna's comments from uamqp switch PR (#25193) + main changes:
- [x] Moving pyamqp logic out to the PyamqpTransport
- [x] **Confirmed: The size of encoded pyamqp.Message is larger than uamqp.Message.** I thought otherwise b/c I was adding the header/property objects by default when building the outgoing uamqp message, even if all values inside those are None. I've fixed this.
- [ ] Make BatchMessage transport agnostic: #25494 (comment)
- Instead, updated EventDataBatch so that it takes an amqp_transport. If EventDataBatch is manually created and uses PyamqpTransport, inside `send_batch()`, if the producer client transport uses UamqpTransport, the BatchMessage corresponding to the client's amqp_transport will be built and sent.
- [X] Add `message` property to `EventData`/`EventDataBatch`, which return `LegacyMessage`/`LegacyMessageBatch` from `_pyamqp` for backcompat.
- [x] add `connection=None` parameter to `pyamqp.AMQPClient.open()` as per [this discussion](#25494 (comment))
- [x] Add an async SharedConnectionManager in pyamqp: #25494 (comment)
Issue created [[here](#25875)] to address the below TODOs in a separate PR:
- [x] add TODO in pyamqp that SenderClient should take msg_timeout: #25494 (comment)
- [x] add TODO in pyamqp that ReceiveClient should take timeout: #25494 (comment)
- [x] add both MAX_MESSAGE_LENGTH_BYTES and MAX_FRAME_SIZE_BYTES to pyamqp: #25494 (comment)
TODO:
- [ ] fix mypy/pylint issues
- made partial progress. fix rest in separate PR for issue: #25936
- [x] investigate how to remove `message` property from public API.
- mark as deprecated and log a deprecation warning if accessed
* prep release
* [Pyamqp] Exception Todos (#25893)
* throw proper exception and error condition on CBS
* error condition for ready
* close when open is on non-zero channel
* error condition is proper for timeout
* unattached handle
* max handles error condition
* fix strings
* fix
* end the session on unattached handle
* remove TODO
* conditions are proper
* raise link error on close or detach
* revert change
* remove TODO
* change to client error
* change to AmqpError
* change to AMQPError
* detach called
* detach the link, dont close the connection
* comments clean up + change condition
* handle invalid channel on end
* descriptive error message
* fix formatting
* detach err message on max handle
* [Pyamqp] Async WS implementation using a pure async library (#26234)
* init changes for aiohttp ws
* fixes + formatting
* fixes for context manager
* changes
* ssl options
* clean up
* move build opts in to mixin
* pass in proxy information
* attach port to proxy if given
* address comments
* remove self from proxy host
* [SB PyAMQP] Servicebus PyAMQP Working off of Anna's Branch (#24975)
* Added pyamqp
* Added message compatibility tests
* Start rewiring messages for pyamqp
* Added message backcompat layer
* Successful message send
* Started receiver
* Successful message receive
* Message settlement
* Fix other settlement outcomes
* Make tests live
* message partition_key if it can't be decoded - output value
* removing references to __future___ annotations for now - not supported in 3.6
* comparing name of transport - not the object
* passing in a dummy frame for new formatting of SBMessageReceived
* adding in fake frame for message in queue tests
* uamqp_mesage -> uamqp_message
* state should be auth_state
* switching this back - _message is Message
* Improved typing
* Revert "Improved typing"
This reverts commit aeffcb2.
* Fix TransportType enum
* Fix import statement
* Fix application property encoding
* Skip queue iterator tests
* Fix mgmt op timeout
* Fixes to mgmt link
* Fix frame decode tests
* More mgmt fixes
* Some message fixes
* Fix session filters
* Message tests
* Skip more iterator tests
* Update to retry policy
* adding in support for websockets is CE supported?
* fixing up pylint-still some issues
* some more pylint/TODOs
* pylint changes
* fixing pylint
* more pylint connection
* More test fixes
* Fix scheduling
* Fix retry test
* Fix error handling
* Sender refactor for timeout
* Fix link detach
* Fixed receiver control flow
* Update pyamqp async code
* Updated sb async
* Typing fix
* Some async fixes
* Skip async iter tests
* Workaround socket timeout
* Literal import
* More async test fixes
* Added keepalive
* Pylint cleanup
* fix mypy errors in _pyamqp
* fix mypy sb layer
* fix bug
* unused import
* lint
* fix failing tests
* ignore sb iterator receive samples
Co-authored-by: antisch <antisch@microsoft.com>
Co-authored-by: swathipil <swathip@microsoft.com>
* [Pyamqp] Pyamqp fix conn (#26568)
* remove unnecessary pops
* fix var name + remove unnecessary pop
* fix
* Bring in changes to fix bandit from failing
* [EventHubs&ServiceBus] merge sb and eh pyamqp (#26548)
* merge sb and eh pyamqp
* reenable pylint for EH
* turn on mypy for EH
* fix mypy errors eh layer
* fix EH mypy/pylint
* fix SB failing tests
* fix more tests/mypy
* import literal from typing extensions
* remove whitespace
* fix typing cast bug in EH
* lint
* fix port url in async transport
* kashifs comments
* mypy/lint/kashifs comments
* [Pyamqp] Fix Async Invalid Host Error Test (#26595)
* resert transport to original state
* add in missing continue in except
* Update sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/aio/_transport_async.py
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
Co-authored-by: swathipil <76007337+swathipil@users.noreply.github.com>
* fix merge conflict stress
* update cspell, ignore tables spelling
* skip connection verify tests sb for now
* fix logging formatting (#26682)
* copy over kashifs change to sb
* [ServiceBus&EventHubs] pyamqp - update EH/SB docs for release (#26741)
updating docs to prepare for release
* [ServiceBus&EventHubs] fix mypy/pylint (#26744)
* fix mypy/pylint
* bump SB version
* prep release alpha (#26755)
* Increment version for servicebus releases (#26762)
Increment package version after release of azure-servicebus
* Increment version for eventhub releases (#26766)
Increment package version after release of azure-eventhub
* [PyAMQP] Fix logging (#26785)
* make logging network trace a debug level log
* async changes
* await async sleep (#26853)
* fix for async socket (#26852)
* [Pyamqp] Possible solutions for network disruption using async websocket (#26856)
* heartbeat
* constant value for heartbeat
* address comments
* [Pyamqp] Fix to Improve Websocket Sync and Async Network Disruption Handling (#27006)
* changes
* fix lint
* remove prints
* remove unused import
* remove heartbeat from this PR
* [pyAMQP] Stress fixes for aiohttp and valueError (#27034)
* changes
* fix lint
* remove prints
* remove unused import
* remove heartbeat from this PR
* OS Error to catch client closed session error
* removing value error raise, change to logging -- causes error on network disconnect
* when looping on open clinet
* change to printf style
Co-authored-by: Kashif Khan <kashifkhan@microsoft.com>
* [Pyamqp] Changes for blocking exceptions (#27260)
* changes for blocking exceptions
* fix hang on unit test
* lint fixes
* fix bug for closes
* fixes
* lint issues
* Stress testing updates (#27456)
* can uncomment line 5 to run against git version of pyamqp
* can move line 6 into scripts/dev_requirement file
* test against newest version of pyamqp
* increase test time, get rid of unused tests
* removing test names
* change naming
* change naming
* raise logging level to catch only error level
* return logging to info
* adding resource requests
* message retention needs to last as long as the test
* websocket async test
* changing life of messages
* 32 partitions
* add uamqp flag - remove logging
* adding before trying matrix
* websocket dep
* helm ignore
* updating
* updating tests
* update consumer files
* remove log lines
* remove log lines
* remove commented
* update
* [EventHubs] kwargs/error testing (#27065)
* adding tests
* add auth/connection tests + fixes
* fix connection verify error handling
* revert consumer retry change
* call ws close in sync transport
* typo
* fix ws exc import
* fix async transport
* fix link detach vendor error exception parity
* add operationtimeouterror
* add more negative tests
* annas comments + lint
* lint + tests
* add ids for uamqp vs pyamqp tests
* update tests
* skip macos tests
* [EventHubs] check for any non-None values in amqp header/properties (#27444)
* add any method to amqp header/properties
* use count(None) to check non-None header/props vals
* [EventHubs] update sync receive client ready flow (#27411)
* update sync receive client ready flow
* lint + mypy
* fix reconnect test
* [pyamqp] os error add (#27351)
* os error add
* update except statement
* test mock of receive_bytes on pipeline
* client os error
* mock try 2
* pylint
* fix patch line
* fixed mock
* newline
* fixing spacing
* unused import
* fix error mssgs
* changes from perf run (#27703)
* Matrix Gen Stress Tests (#27754)
* stress matrix gen
* removing unused dockerfiles for now
* revert sb to main
* revert non-eh files
* restore samples/readmes to main
* restore ci/tests/shared reqs to main
* update to stable
* lint + fix tests for no uamqp import
* update test timeout
* re-organize changelog
Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>
Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Rakshith Bhyravabhotla <rakshith.bhyravabhotla@gmail.com>
Co-authored-by: Kashif Khan <kashifkhan@microsoft.com>
Co-authored-by: Libba Lawrence <llawrence@microsoft.com>
Co-authored-by: Kashif Khan <361477+kashifkhan@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Anna Tisch <antisch@microsoft.com>
fixes: #23472