map_index_template not compute when task marked success manually #39092
Description
Apache Airflow version
2.9.0
What happened?
If i mark a task manually at success while it's running only already succeed mapped task have a map_index_template
What you think should happen instead?
map_index_template
should be compute in all cases
How to reproduce
from airflow.operators.python import PythonOperator
from pendulum import today
from airflow import DAG
dag_1 = DAG(
dag_id="dag_1",
schedule_interval=None,
max_active_tasks=2,
start_date=today("UTC").add(days=-1)
)
with dag_1:
def generate():
return [{'date': '2024-01-01'}, {'date': '2024-01-02'}, {'date': '2024-01-03'}, {'date': '2024-01-04'},
{'date': '2024-01-05'}, {'date': '2024-01-06'}, {'date': '2024-01-07'}]
def toto(date, arg_2=None):
import time
print(date)
time.sleep(10)
a = PythonOperator(
task_id="a",
python_callable=generate)
b = PythonOperator.partial(
task_id="b",
map_index_template="{{ task.op_kwargs['date'] }}",
python_callable=toto).expand(op_kwargs=a.output)
a >> btrigger a run , wait for a mapped task to succes , then mark state of the task b at success
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct