KubernetesPodOperator duplicating logs when interrupted

[Nikita-Sobolev]

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.8.1

What happened?

The KubernetesPodOperator is duplicating tasks's logs two times when log read interrupted but container base still running they are interrupted. Happens randomly on different dags and different runs of the same dag. Assume it is somehow connected to the #35019

What you think should happen instead?

no logs duplicate

How to reproduce

KubernetesPodOperator on cloud AKS cluster

Operating System

Ubuntu 22.04

Versions of Apache Airflow Providers

apache-airflow==2.8.1
apache-airflow-providers-amazon==8.16.0
apache-airflow-providers-celery==3.5.1
apache-airflow-providers-cncf-kubernetes==7.13.0
apache-airflow-providers-common-io==1.2.0
apache-airflow-providers-common-sql==1.10.0
apache-airflow-providers-docker==3.9.1
apache-airflow-providers-elasticsearch==5.3.1
apache-airflow-providers-ftp==3.7.0
apache-airflow-providers-google==10.13.1
apache-airflow-providers-grpc==3.4.1
apache-airflow-providers-hashicorp==3.6.1
apache-airflow-providers-http==4.8.0
apache-airflow-providers-imap==3.5.0
apache-airflow-providers-microsoft-azure==8.5.1
apache-airflow-providers-mysql==5.5.1
apache-airflow-providers-odbc==4.4.0
apache-airflow-providers-openlineage==1.4.0
apache-airflow-providers-postgres==5.10.0
apache-airflow-providers-redis==3.6.0
apache-airflow-providers-sendgrid==3.4.0
apache-airflow-providers-sftp==4.8.1
apache-airflow-providers-slack==8.5.1
apache-airflow-providers-snowflake==5.2.1
apache-airflow-providers-sqlite==3.7.0
apache-airflow-providers-ssh==3.10.0
google-cloud-orchestration-airflow==1.10.0

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[raphaelauv]

could you try the latest version 8.1.1 of apache-airflow-providers-cncf-kubernetes

[tirkarthi]

Related #33498

[gbonazzoli]

@raphaelauv

with version 8.1.1 the problem is still present. It seems that now is allways getting "Pod docker-java-w2ade41b log read interrupted but container base still running"

Airflow's version:

airflow@airflow-test-worker-6cb8744f69-sw7xg:/opt/airflow$ airflow version
2.9.0

airflow@airflow-test-worker-6cb8744f69-sw7xg:/opt/airflow$ pip list | grep kub
apache-airflow-providers-cncf-kubernetes 8.1.1
kubernetes                               29.0.0
kubernetes_asyncio                       29.0.0

[eladkal]

Some work around it was done #33498
cc @fdemiane maybe you will have time to take a look?