Missing TI previous tries in Logs UI after upgrading to Airflow 2.10.1+

[ahidalgob]

Apache Airflow version

2.10.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

After #41483 we miss displaying buttons to see logs for previous tries that don't have TIHistory (because, for example, they ran in an older Airflow version).

Clicking see more on the right takes you to the other logs UI in which all tries are displayed correctly.

What you think should happen instead?

In case there are no TIHistory objects and there were more than 1 try, we should create buttons to see previous tries logs.

How to reproduce

Create multiple tries for a TI using an older version of Airflow (<2.10.0), and upgrade to Airflow 2.10.1 or higher.

Operating System

linux

Versions of Apache Airflow Providers

No response

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[sean-rose]

We're running into this problem as well after upgrading from Airflow 2.9.3 to 2.10.5.

More alarmingly, it seems to be happening for new DAG runs as well when tasks fail and retry, not just historical DAG runs from before upgrading to Airflow 2.10. Do task retries not generate TIHistory records?

[sean-rose]

More alarmingly, it seems to be happening for new DAG runs as well when tasks fail and retry, not just historical DAG runs from before upgrading to Airflow 2.10. Do task retries not generate TIHistory records?

Testing this using Airflow 2.10.5 in a local Docker Compose setup I'm seeing no records at all in the task_instance_history table after running a task that failed three times in row (retrying twice). 😬

Looks like this was fixed in #47223 on Feb 28 (after the most recent 2.10.5 release). @jedcunningham what are the prospects for getting this fix included in an upcoming Airflow 2 release?

[ashb]

@sean-rose #47223 is 100% not back-portable to Airflow 2, as that fix applies to the entirely new architecture for Airflow 3.

So the unless you or someone else from the community steps up and produces a fix for 2.x, the chances are 0 I'm afraid. All of the "core team"'s focus is on Airflow 3 right now, and generally the open source project only supports the latest version, and we are now in the release candidate phases of Airflow 3

[chloe-current]

We've run into this same issue due to the performing a database transaction inside on_retry_callback (in our case, accessing a connection through a hook). Our workaround was to start and join on_retry_callback in a different thread, which creates a new threadlocal database session