Support clearing and updating state of individual mapped task instances #22958

ephraimbuddy · 2022-04-12T19:34:45Z

Provide support for UI to clear/update state of mapped task instances

TODO:

Add test at UI side

airflow/api/common/mark_tasks.py

bbovenzi · 2022-04-13T19:30:42Z

Testing with the UI branch mapped-instance-actions. And everything appears to me working. 👍

airflow/api/common/mark_tasks.py

airflow/models/dag.py

airflow/api/common/mark_tasks.py

airflow/models/dag.py

airflow/www/views.py

github-actions · 2022-04-15T08:33:13Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

ephraimbuddy · 2022-04-15T11:38:20Z

Not sure how to fix some mypy errors here. @uranusjr can you help. Thanks

uranusjr · 2022-04-18T14:16:53Z

I pushed a commit for this. The entire set_task_instance_state, get_task_isntances and clear combination is a mess, so I hope this keeps everything right-ish. We’ll see…

Edit: Most things seem to work, except MSSQL does not seem to like tuple_ :(

bbovenzi · 2022-04-18T16:46:33Z

The latest changes got rid of map_indexes but we should still have an ability to pass multiple map indexes in a single request. Either comma separated like map_index=0%2C1 in the url or as multiple params like &map_index=0&map_index=1

uranusjr · 2022-04-19T01:33:57Z

I added a commit to support &map_index=0&map_index=1 This is translated to [(task_id, 0), (task_id, 1)] before being passed to set_state.

bbovenzi

/confirm, /failed and /success work. But /clear is not processing multiple map_index params chained together.

The UI branch at mapped-instance-actions is up-to-date if you want to manually test it.

uranusjr · 2022-04-20T02:35:31Z

Because I forgot to modify /clear to expect multiple map_index params 🤦‍♂️

bbovenzi

Everything is working as expected from the UI now

… the UI

…es from the UI

…instances from the UI

…ed taskinstances from the UI

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

This allows setting multiple instances of the same task to SUCCESS or FAILED in one request. This is translated to multiple task specifier tuples (task_id, map_index) when passed to set_state(). Also made some drive-through improvements adding types and clean some formatting up.

ephraimbuddy requested review from XD-DENG, ashb, bbovenzi, kaxil, mik-laj and ryanahamilton as code owners April 12, 2022 19:34

boring-cyborg bot added area:API Airflow's REST/HTTP API area:webserver Webserver related Issues labels Apr 12, 2022

uranusjr reviewed Apr 12, 2022

View reviewed changes

airflow/api/common/mark_tasks.py Outdated Show resolved Hide resolved

bbovenzi changed the title ~~Support clearing and updating state of mapped task instances~~ Support clearing and updating state of individual mapped task instances Apr 13, 2022

ephraimbuddy force-pushed the clearing-and-marking-mapped-task branch from 95a0a2e to 916ae06 Compare April 13, 2022 20:08

jedcunningham reviewed Apr 13, 2022

View reviewed changes

airflow/api/common/mark_tasks.py Outdated Show resolved Hide resolved

airflow/models/dag.py Outdated Show resolved Hide resolved

uranusjr reviewed Apr 14, 2022

View reviewed changes

airflow/api/common/mark_tasks.py Outdated Show resolved Hide resolved

airflow/models/dag.py Outdated Show resolved Hide resolved

airflow/models/dag.py Outdated Show resolved Hide resolved

airflow/www/views.py Outdated Show resolved Hide resolved

uranusjr approved these changes Apr 15, 2022

View reviewed changes

github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Apr 15, 2022

ephraimbuddy force-pushed the clearing-and-marking-mapped-task branch from 79ee7de to 98abe5f Compare April 15, 2022 11:35

This comment was marked as outdated.

Sign in to view

uranusjr force-pushed the clearing-and-marking-mapped-task branch 2 times, most recently from 02f3f6b to 45cb2d6 Compare April 18, 2022 15:30

uranusjr force-pushed the clearing-and-marking-mapped-task branch 2 times, most recently from 22581b4 to 0a52609 Compare April 19, 2022 10:04

bbovenzi requested changes Apr 19, 2022

View reviewed changes

uranusjr force-pushed the clearing-and-marking-mapped-task branch from 0a52609 to a890173 Compare April 20, 2022 02:40

uranusjr force-pushed the clearing-and-marking-mapped-task branch from a890173 to b1b2fd1 Compare April 20, 2022 13:53

bbovenzi approved these changes Apr 20, 2022

View reviewed changes

ephraimbuddy and others added 13 commits April 20, 2022 17:41

Allow marking/clearing mapped taskinstances from the UI

44d58f4

fixup! Allow marking/clearing mapped taskinstances from the UI

7c63b5c

fixup! fixup! Allow marking/clearing mapped taskinstances from the UI

c149958

fixup! fixup! fixup! Allow marking/clearing mapped taskinstances from…

cc099fa

… the UI

add tests

d4cb610

fixup! fixup! fixup! fixup! Allow marking/clearing mapped taskinstanc…

d0d0a41

…es from the UI

fixup! fixup! fixup! fixup! fixup! Allow marking/clearing mapped task…

0c30cee

…instances from the UI

fixup! fixup! fixup! fixup! fixup! fixup! Allow marking/clearing mapp…

14ad8f7

…ed taskinstances from the UI

Apply suggestions from code review

6c3b250

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

fixup! Apply suggestions from code review

78f91bc

Refactor to straighten up types

4b4f1dc

Introduce tuple_().in_() shim for MSSQL compat

19ba72a

ashb force-pushed the clearing-and-marking-mapped-task branch from b1b2fd1 to 19ba72a Compare April 20, 2022 16:41

ashb merged commit 4fa718e into apache:main Apr 20, 2022

jedcunningham added changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) area:dynamic-task-mapping AIP-42 labels Apr 25, 2022

jedcunningham added this to the Airflow 2.3.0 milestone Apr 26, 2022

ashb deleted the clearing-and-marking-mapped-task branch July 4, 2025 13:21

kevinhongzl mentioned this pull request Oct 4, 2025

fix: enable api to clear ti instances by specifying map indexes #56346

Merged

Support clearing and updating state of individual mapped task instances #22958

Support clearing and updating state of individual mapped task instances #22958

Uh oh!

Conversation

ephraimbuddy commented Apr 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

bbovenzi commented Apr 13, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 15, 2022

Uh oh!

ephraimbuddy commented Apr 15, 2022

Uh oh!

This comment was marked as outdated.

uranusjr commented Apr 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bbovenzi commented Apr 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

uranusjr commented Apr 19, 2022

Uh oh!

bbovenzi left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

uranusjr commented Apr 20, 2022

Uh oh!

bbovenzi left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ephraimbuddy commented Apr 12, 2022 •

edited

Loading

uranusjr commented Apr 18, 2022 •

edited

Loading

bbovenzi commented Apr 18, 2022 •

edited

Loading

bbovenzi left a comment •

edited

Loading