Better error handling for running ti heartbeats after task is cleared #53140
Description
Apache Airflow version
main (development)
If "Other Airflow 2 version" selected, which one?
No response
What happened?
When a running TI is cleared, it is immediately removed from the ti table and moved to TIH. This causes subsequent heartbeats from the running TI to receive a 404 (Not Found) from the API server.
As a result, the client (worker/task) logs an error like: "TaskInstance not found", while it could be running.
What you think should happen instead?
Instead of a 404, the API should handle this more gracefully by checking TIH when a TI is not found in TI.
How to reproduce
- Start a task
- Clear it while running
- Observe 404 errors from heartbeat API calls and the "TaskInstance not found" message.
Operating System
MacOS
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct