-
Notifications
You must be signed in to change notification settings - Fork 16.4k
Add KubernetesPodOperatorAsync
#28523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
93cd391 to
d6e9eac
Compare
KubernetesPodOperatorAsync
af23ad3 to
5c88103
Compare
dimberman
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall this looks good to me. My only major question is whether we need to asynchronously load the configuration. otherwise great job!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this function really one that requires async? I imagine loading a kube config would be pretty quick?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you're referring to async def _load_config(self) right?
well it does do file operations which are technically blocking (even though they might be fast)
so you might as well handle them asyncly ... just cus... why not?
but now that you mention it one thing that is a bit curious is ... how are we able to await await config.load_kube_config becaus it look like that is not an async method... i must be missing something
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yup, @phanikumv this config should come from from kubernetes_asyncio import config instead of from kubernetes import config like it is doing right now.
tests/providers/cncf/kubernetes/operators/test_kubernetes_pod.py
Outdated
Show resolved
Hide resolved
tests/providers/cncf/kubernetes/triggers/test_wait_container.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May be place more descriptive comment here to describe what does actually in_cluster parameter mean?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you're referring to async def _load_config(self) right?
well it does do file operations which are technically blocking (even though they might be fast)
so you might as well handle them asyncly ... just cus... why not?
but now that you mention it one thing that is a bit curious is ... how are we able to await await config.load_kube_config becaus it look like that is not an async method... i must be missing something
375bae2 to
2c78f81
Compare
0ed246e to
bbd2254
Compare
|
#28230 was created first, lets get that one merged and than rebase this on top of it to add more features |
bbd2254 to
30f12a2
Compare
36d7187 to
04e81e2
Compare
04e81e2 to
373cb18
Compare
I am working to rebase it and add more features |
373cb18 to
7cf6740
Compare
7cf6740 to
5ac8b9d
Compare
|
Closing as the additional feature on periodic fetching of logs has been implemented in #27758 |
This PR donates the
KubernetesPodOperatorAsyncfrom astronomer-providers repo to Airflow.Astronomer and its customers have been running this in production since last 6 months
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.