Skip to content

Conversation

@Shekharrajak
Copy link

Fix clearing mapped task instances in same group to respect depth-first execution (#40543)

Problem

When clearing a specific mapped task instance within a mapped task group (e.g., etl.e[1]), the system incorrectly cleared ALL downstream task instances (etl.t[] and etl.l[]) instead of only the relevant ones with the same map_index (etl.t[1] and etl.l[1]). This violated the depth-first execution model where each mapped instance should operate independently.

Solution

Added explicit handling in _get_relevant_map_indexes() to detect when two tasks are siblings in the same mapped task group. When ti_count == ancestor_ti_count, the function now returns the same map_index directly, ensuring depth-first execution is maintained.

Changes

  • Modified airflow-core/src/airflow/models/taskinstance.py:

    • Added special case check in _get_relevant_map_indexes() for same-group siblings
    • Returns map_index directly when ti_count == ancestor_ti_count
  • Added tests in airflow-core/tests/unit/models/test_taskinstance.py:

    • test_find_relevant_relatives_downstream_same_group_siblings: Parametrized test for map_indexes 0, 1, 2
    • test_find_relevant_relatives_upstream_same_group_siblings: Test for upstream direction

Testing

All 11 find_relevant_relatives tests pass, including:

  • 6 existing tests (still passing)
  • 4 new tests validating the fix

@shahar1 shahar1 self-requested a review January 10, 2026 10:29
Copy link
Contributor

@shahar1 shahar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Please fix static checks (it's recommended to do so with it with prek)
  2. If I remove the added condition and run the tests - they still pass. Try to think of how to simulate it more precisely in the tests, there might be a need for extra development in the testing tooling.

@shahar1 shahar1 self-requested a review January 10, 2026 10:48
@Shekharrajak
Copy link
Author

  1. Please fix static checks (it's recommended to do so with it with prek)

Thanks for review, committed the changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants