Skip to content

Conversation

@paolo-moriello
Copy link
Contributor

@paolo-moriello paolo-moriello commented May 23, 2024

This is a follow-up on this pr: #39325.

The above PR addressed an issue for which KubernetesPodOperator properties were refreshed on auth-token expiration.

This PR covers also the get_logs=True case, which was not covered by the previous change, aiming at fixing this exception:

Traceback (most recent call last):
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/operators/pod.py", line 615, in execute_sync
    self.pod_manager.fetch_requested_container_logs(
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/utils/pod_manager.py", line 587, in fetch_requested_container_logs
    status = self.fetch_container_logs(pod=pod, container_name=c, follow=follow_logs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/utils/pod_manager.py", line 511, in fetch_container_logs
    if not self.container_is_running(pod, container_name=container_name):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/utils/pod_manager.py", line 645, in container_is_running
    remote_pod = self.read_pod(pod)
  [...]
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/kubernetes/client/api_client.py", line 180, in __call_api
    response_data = self.request(
                    ^^^^^^^^^^^^^
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/kubernetes/client/api_client.py", line 373, in request
    return self.rest_client.GET(url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/kubernetes/client/rest.py", line 244, in GET
    return self.request("GET", url,
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/kubernetes/client/rest.py", line 238, in request
    raise ApiException(http_resp=r)
kubernetes.client.exceptions.ApiException: (401)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg bot added area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues labels May 23, 2024
@paolo-moriello paolo-moriello force-pushed the k8spodoperator-refresh-token-logs branch from 02b3b51 to d7852f6 Compare May 24, 2024 06:18
@paolo-moriello paolo-moriello force-pushed the k8spodoperator-refresh-token-logs branch from d7852f6 to 1f1324b Compare May 25, 2024 09:04
@paolo-moriello paolo-moriello force-pushed the k8spodoperator-refresh-token-logs branch from 1f1324b to a825540 Compare May 28, 2024 15:48
@paolo-moriello
Copy link
Contributor Author

@amoghrajesh could you please have another look?

@paolo-moriello
Copy link
Contributor Author

hey, this is a bug in the 8.2.0 for the get_logs=True case, which is unexpected as it doesn't happen if logs is False. could we prioritize this so to fix this in the next release? 🙏

@amoghrajesh @romsharon98

@pankajastro pankajastro merged commit 032d276 into apache:main May 30, 2024
fdemiane pushed a commit to fdemiane/airflow that referenced this pull request Jun 6, 2024
…when logging (apache#39789)

* Refresh KubernetesPodOperator properties when credentials expire also when logs are enabled

* Linting

* Rename function

---------

Co-authored-by: pmoriello <paolo.moriello@zalando.ch>
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
…when logging (apache#39789)

* Refresh KubernetesPodOperator properties when credentials expire also when logs are enabled

* Linting

* Rename function

---------

Co-authored-by: pmoriello <paolo.moriello@zalando.ch>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants