Inconsistent behavior of EmptyOperator between start and end tasks #29974
Description
Apache Airflow version
Other Airflow 2 version (please specify below)
What happened
We are using Airflow 2.4.3.
When looking at the documentation for the EmptyOperator, it says explicitly that it is never processed by the executor.
However what I notice is that in our cases it differs between start and end EmptyOperators. The start tasks are not processed by the executor but for some reason the end tasks are for some reason.
This results in unexpected behavior and is inefficient as it creates a pod on kubernetes in our case for no reason. Additionally, it causes some weird behavior in our lineage graphs.
For the start task we see no logs:
*** Log file does not exist: /opt/airflow/logs/dag_id=dbt-datahub/run_id=scheduled__2023-03-07T00:00:00+00:00/task_id=initial_task_start/attempt=1.log
*** Fetching from: http://:8793/log/dag_id=dbt-datahub/run_id=scheduled__2023-03-07T00:00:00+00:00/task_id=initial_task_start/attempt=1.log
*** Failed to fetch log file from worker. Request URL is missing an 'http://' or 'https://' protocol.
dbtdatahubend-dc6d51700abc41e0974b46caafd857ac
*** Reading local file: /opt/airflow/logs/dag_id=dbt-datahub/run_id=manual__2023-03-07T16:56:07.937548+00:00/task_id=end/attempt=1.log
[2023-03-07, 16:56:31 UTC] {taskinstance.py:1165} INFO - Dependencies all met for <TaskInstance: dbt-datahub.end manual__2023-03-07T16:56:07.937548+00:00 [queued]>
[2023-03-07, 16:56:31 UTC] {taskinstance.py:1165} INFO - Dependencies all met for <TaskInstance: dbt-datahub.end manual__2023-03-07T16:56:07.937548+00:00 [queued]>
[2023-03-07, 16:56:31 UTC] {taskinstance.py:1362} INFO -
--------------------------------------------------------------------------------
[2023-03-07, 16:56:31 UTC] {taskinstance.py:1363} INFO - Starting attempt 1 of 1
[2023-03-07, 16:56:31 UTC] {taskinstance.py:1364} INFO -
--------------------------------------------------------------------------------
[2023-03-07, 16:56:31 UTC] {taskinstance.py:1383} INFO - Executing <Task(EmptyOperator): end> on 2023-03-07 16:56:07.937548+00:00
[2023-03-07, 16:56:31 UTC] {standard_task_runner.py:55} INFO - Started process 19 to run task
[2023-03-07, 16:56:31 UTC] {standard_task_runner.py:82} INFO - Running: ['airflow', 'tasks', 'run', 'dbt-datahub', 'end', 'manual__2023-03-07T16:56:07.937548+00:00', '--job-id', '24', '--raw', '--subdir', 'DAGS_FOLDER/dbt-datahub/dbt-datahub.py', '--cfg-path', '/tmp/tmpdr42kl3k']
[2023-03-07, 16:56:31 UTC] {standard_task_runner.py:83} INFO - Job 24: Subtask end
[2023-03-07, 16:56:31 UTC] {task_command.py:376} INFO - Running <TaskInstance: dbt-datahub.end manual__2023-03-07T16:56:07.937548+00:00 [running]> on host dbtdatahubend-dc6d51700abc41e0974b46caafd857ac
[2023-03-07, 16:56:31 UTC] {taskinstance.py:1590} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=Conveyor
AIRFLOW_CTX_DAG_ID=dbt-datahub
AIRFLOW_CTX_TASK_ID=end
AIRFLOW_CTX_EXECUTION_DATE=2023-03-07T16:56:07.937548+00:00
AIRFLOW_CTX_TRY_NUMBER=1
AIRFLOW_CTX_DAG_RUN_ID=manual__2023-03-07T16:56:07.937548+00:00
[2023-03-07, 16:56:31 UTC] {taskinstance.py:1401} INFO - Marking task as SUCCESS. dag_id=dbt-datahub, task_id=end, execution_date=20230307T165607, start_date=20230307T165631, end_date=20230307T165631
[2023-03-07, 16:56:31 UTC] {base.py:71} INFO - Using connection ID 'datahub_rest_default' for task execution.
[2023-03-07, 16:56:31 UTC] {base.py:71} INFO - Using connection ID 'datahub_rest_default' for task execution.
[2023-03-07, 16:56:31 UTC] {_plugin.py:147} INFO - Emitting Datahub Dataflow: DataFlow(urn=<datahub.utilities.urns.data_flow_urn.DataFlowUrn object at 0x7fb9ced397c0>, id='dbt-datahub', orchestrator='airflow', cluster='prod', name=None, description='None\n\n', properties={'_access_control': 'None', '_default_view': "'grid'", 'catchup': 'True', 'fileloc': "'/opt/airflow/dags/dbt-datahub/dbt-datahub.py'", 'is_paused_upon_creation': 'None', 'start_date': 'None', 'tags': '[]', 'timezone': "Timezone('UTC')"}, url='https://app.dev.datafy.cloud/environments/datahubtest/airflow/tree?dag_id=dbt-datahub', tags=set(), owners={'Conveyor'})
[2023-03-07, 16:56:31 UTC] {_plugin.py:165} INFO - Emitting Datahub Datajob: DataJob(id='end', urn=<datahub.utilities.urns.data_job_urn.DataJobUrn object at 0x7fb9cecbbfa0>, flow_urn=<datahub.utilities.urns.data_flow_urn.DataFlowUrn object at 0x7fb9cecbf910>, name=None, description=None, properties={'depends_on_past': 'False', 'email': '[]', 'label': "'end'", 'execution_timeout': 'None', 'sla': 'None', 'task_id': "'end'", 'trigger_rule': "<TriggerRule.ALL_SUCCESS: 'all_success'>", 'wait_for_downstream': 'False', 'downstream_task_ids': 'set()', 'inlets': '[]', 'outlets': '[]'}, url='https://app.dev.datafy.cloud/environments/datahubtest/airflow/taskinstance/list/?flt1_dag_id_equals=dbt-datahub&_flt_3_task_id=end', tags=set(), owners={'Conveyor'}, group_owners=set(), inlets=[], outlets=[], upstream_urns=[<datahub.utilities.urns.data_job_urn.DataJobUrn object at 0x7fb9cecbbc10>])
[2023-03-07, 16:56:31 UTC] {_plugin.py:179} INFO - Emitted Start Datahub Dataprocess Instance: DataProcessInstance(id='dbt-datahub_end_manual__2023-03-07T16:56:07.937548+00:00', urn=<datahub.utilities.urns.data_process_instance_urn.DataProcessInstanceUrn object at 0x7fb9cecbb040>, orchestrator='airflow', cluster='prod', type='BATCH_AD_HOC', template_urn=<datahub.utilities.urns.data_job_urn.DataJobUrn object at 0x7fb9cecbbfa0>, parent_instance=None, properties={'run_id': 'manual__2023-03-07T16:56:07.937548+00:00', 'duration': '0.163779', 'start_date': '2023-03-07 16:56:31.157871+00:00', 'end_date': '2023-03-07 16:56:31.321650+00:00', 'execution_date': '2023-03-07 16:56:07.937548+00:00', 'try_number': '1', 'hostname': 'dbtdatahubend-dc6d51700abc41e0974b46caafd857ac', 'max_tries': '0', 'external_executor_id': 'None', 'pid': '19', 'state': 'success', 'operator': 'EmptyOperator', 'priority_weight': '1', 'unixname': 'airflow', 'log_url': 'https://app.dev.datafy.cloud/environments/datahubtest/airflow/log?execution_date=2023-03-07T16%3A56%3A07.937548%2B00%3A00&task_id=end&dag_id=dbt-datahub&map_index=-1'}, url='https://app.dev.datafy.cloud/environments/datahubtest/airflow/log?execution_date=2023-03-07T16%3A56%3A07.937548%2B00%3A00&task_id=end&dag_id=dbt-datahub&map_index=-1', inlets=[], outlets=[], upstream_urns=[])
[2023-03-07, 16:56:31 UTC] {_plugin.py:191} INFO - Emitted Completed Data Process Instance: DataProcessInstance(id='dbt-datahub_end_manual__2023-03-07T16:56:07.937548+00:00', urn=<datahub.utilities.urns.data_process_instance_urn.DataProcessInstanceUrn object at 0x7fb9ced39700>, orchestrator='airflow', cluster='prod', type='BATCH_SCHEDULED', template_urn=<datahub.utilities.urns.data_job_urn.DataJobUrn object at 0x7fb9cecbbfa0>, parent_instance=None, properties={}, url=None, inlets=[], outlets=[], upstream_urns=[])
[2023-03-07, 16:56:31 UTC] {local_task_job.py:159} INFO - Task exited with return code 0
[2023-03-07, 16:56:31 UTC] {taskinstance.py:2623} INFO - 0 downstream tasks scheduled from follow-on schedule check
Airflow scheduler logs for the dag:
[2023-03-08 13:25:28,870] {scheduler_job.py:346} INFO - 1 tasks up for execution:
<TaskInstance: dbt-datahub3.dbt-run manual__2023-03-08T13:25:26.874182+00:00 [scheduled]>
[2023-03-08 13:25:28,870] {scheduler_job.py:411} INFO - DAG dbt-datahub3 has 0/32 running and queued tasks
[2023-03-08 13:25:28,870] {scheduler_job.py:497} INFO - Setting the following tasks to queued state:
<TaskInstance: dbt-datahub3.dbt-run manual__2023-03-08T13:25:26.874182+00:00 [scheduled]>
[2023-03-08 13:25:28,873] {scheduler_job.py:536} INFO - Sending TaskInstanceKey(dag_id='dbt-datahub3', task_id='dbt-run', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) to executor with priority 2 and queue default
[2023-03-08 13:25:28,873] {base_executor.py:95} INFO - Adding to queue: ['airflow', 'tasks', 'run', 'dbt-datahub3', 'dbt-run', 'manual__2023-03-08T13:25:26.874182+00:00', '--local', '--subdir', 'DAGS_FOLDER/dbt-datahub/dbt-datahub.py']
[2023-03-08 13:25:28,875] {kubernetes_executor.py:551} INFO - Add task TaskInstanceKey(dag_id='dbt-datahub3', task_id='dbt-run', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) with command ['airflow', 'tasks', 'run', 'dbt-datahub3', 'dbt-run', 'manual__2023-03-08T13:25:26.874182+00:00', '--local', '--subdir', 'DAGS_FOLDER/dbt-datahub/dbt-datahub.py'] with executor_config {}
[2023-03-08 13:25:28,876] {kubernetes_executor.py:305} INFO - Kubernetes job is TaskInstanceKey(dag_id='dbt-datahub3', task_id='dbt-run', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1)
[2023-03-08 13:25:28,972] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type ADDED
[2023-03-08 13:25:28,972] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 Pending
[2023-03-08 13:25:28,976] {scheduler_job.py:588} INFO - Executor reports execution of dbt-datahub3.dbt-run run_id=manual__2023-03-08T13:25:26.874182+00:00 exited with status queued for try_number 1
[2023-03-08 13:25:28,981] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type MODIFIED
[2023-03-08 13:25:28,981] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 Pending
[2023-03-08 13:25:28,985] {scheduler_job.py:621} INFO - Setting external_id for <TaskInstance: dbt-datahub3.dbt-run manual__2023-03-08T13:25:26.874182+00:00 [queued]> to 42
[2023-03-08 13:25:29,002] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type MODIFIED
[2023-03-08 13:25:29,002] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 Pending
[2023-03-08 13:25:29,707] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type MODIFIED
[2023-03-08 13:25:29,707] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 Pending
[2023-03-08 13:25:30,721] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type MODIFIED
[2023-03-08 13:25:30,721] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 Pending
[2023-03-08 13:25:31,721] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type MODIFIED
[2023-03-08 13:25:31,722] {kubernetes_executor.py:219} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 is Running
[2023-03-08 13:25:44,671] {scheduler_job.py:346} INFO - 1 tasks up for execution:
<TaskInstance: dbt-datahub3.end_task manual__2023-03-08T13:25:26.874182+00:00 [scheduled]>
[2023-03-08 13:25:44,671] {scheduler_job.py:411} INFO - DAG dbt-datahub3 has 0/32 running and queued tasks
[2023-03-08 13:25:44,671] {scheduler_job.py:497} INFO - Setting the following tasks to queued state:
<TaskInstance: dbt-datahub3.end_task manual__2023-03-08T13:25:26.874182+00:00 [scheduled]>
[2023-03-08 13:25:44,673] {scheduler_job.py:536} INFO - Sending TaskInstanceKey(dag_id='dbt-datahub3', task_id='end_task', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) to executor with priority 1 and queue default
[2023-03-08 13:25:44,674] {base_executor.py:95} INFO - Adding to queue: ['airflow', 'tasks', 'run', 'dbt-datahub3', 'end_task', 'manual__2023-03-08T13:25:26.874182+00:00', '--local', '--subdir', 'DAGS_FOLDER/dbt-datahub/dbt-datahub.py']
[2023-03-08 13:25:44,676] {kubernetes_executor.py:551} INFO - Add task TaskInstanceKey(dag_id='dbt-datahub3', task_id='end_task', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) with command ['airflow', 'tasks', 'run', 'dbt-datahub3', 'end_task', 'manual__2023-03-08T13:25:26.874182+00:00', '--local', '--subdir', 'DAGS_FOLDER/dbt-datahub/dbt-datahub.py'] with executor_config {}
[2023-03-08 13:25:44,676] {kubernetes_executor.py:305} INFO - Kubernetes job is TaskInstanceKey(dag_id='dbt-datahub3', task_id='end_task', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1)
[2023-03-08 13:25:44,749] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type MODIFIED
[2023-03-08 13:25:44,749] {kubernetes_executor.py:219} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 is Running
[2023-03-08 13:25:44,756] {scheduler_job.py:588} INFO - Executor reports execution of dbt-datahub3.end_task run_id=manual__2023-03-08T13:25:26.874182+00:00 exited with status queued for try_number 1
[2023-03-08 13:25:44,759] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type ADDED
[2023-03-08 13:25:44,759] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 Pending
[2023-03-08 13:25:44,763] {scheduler_job.py:621} INFO - Setting external_id for <TaskInstance: dbt-datahub3.end_task manual__2023-03-08T13:25:26.874182+00:00 [queued]> to 42
[2023-03-08 13:25:44,765] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type MODIFIED
[2023-03-08 13:25:44,765] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 Pending
[2023-03-08 13:25:44,774] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type MODIFIED
[2023-03-08 13:25:44,774] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 Pending
[2023-03-08 13:25:45,748] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type MODIFIED
[2023-03-08 13:25:45,748] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 Pending
[2023-03-08 13:25:46,763] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type MODIFIED
[2023-03-08 13:25:46,763] {kubernetes_executor.py:207} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 Pending
[2023-03-08 13:25:46,775] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type MODIFIED
[2023-03-08 13:25:46,775] {kubernetes_executor.py:212} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 Succeeded
[2023-03-08 13:25:46,962] {kubernetes_executor.py:383} INFO - Attempting to finish pod; pod_id: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1; state: None; annotations: {'dag_id': 'dbt-datahub3', 'task_id': 'dbt-run', 'execution_date': None, 'run_id': 'manual__2023-03-08T13:25:26.874182+00:00', 'try_number': '1'}
[2023-03-08 13:25:46,963] {kubernetes_executor.py:598} INFO - Changing state of (TaskInstanceKey(dag_id='dbt-datahub3', task_id='dbt-run', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1), None, 'dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1', 'datahubtest', '184454484') to None
[2023-03-08 13:25:46,988] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type MODIFIED
[2023-03-08 13:25:46,988] {kubernetes_executor.py:212} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 Succeeded
[2023-03-08 13:25:46,997] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 had an event of type DELETED
[2023-03-08 13:25:46,997] {kubernetes_executor.py:212} INFO - Event: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1 Succeeded
[2023-03-08 13:25:47,001] {kubernetes_executor.py:696} INFO - Deleted pod: TaskInstanceKey(dag_id='dbt-datahub3', task_id='dbt-run', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) in namespace datahubtest
[2023-03-08 13:25:47,001] {scheduler_job.py:588} INFO - Executor reports execution of dbt-datahub3.dbt-run run_id=manual__2023-03-08T13:25:26.874182+00:00 exited with status None for try_number 1
[2023-03-08 13:25:47,078] {kubernetes_executor.py:383} INFO - Attempting to finish pod; pod_id: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1; state: None; annotations: {'dag_id': 'dbt-datahub3', 'task_id': 'dbt-run', 'execution_date': None, 'run_id': 'manual__2023-03-08T13:25:26.874182+00:00', 'try_number': '1'}
[2023-03-08 13:25:47,079] {kubernetes_executor.py:383} INFO - Attempting to finish pod; pod_id: dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1; state: None; annotations: {'dag_id': 'dbt-datahub3', 'task_id': 'dbt-run', 'execution_date': None, 'run_id': 'manual__2023-03-08T13:25:26.874182+00:00', 'try_number': '1'}
[2023-03-08 13:25:47,079] {kubernetes_executor.py:598} INFO - Changing state of (TaskInstanceKey(dag_id='dbt-datahub3', task_id='dbt-run', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1), None, 'dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1', 'datahubtest', '184454492') to None
[2023-03-08 13:25:47,085] {kubernetes_executor.py:696} INFO - Deleted pod: TaskInstanceKey(dag_id='dbt-datahub3', task_id='dbt-run', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) in namespace datahubtest
[2023-03-08 13:25:47,085] {kubernetes_executor.py:598} INFO - Changing state of (TaskInstanceKey(dag_id='dbt-datahub3', task_id='dbt-run', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1), None, 'dbtdatahub3dbtrun-43aa890b165342d09555ed1555b5f7c1', 'datahubtest', '184454493') to None
[2023-03-08 13:25:47,090] {kubernetes_executor.py:696} INFO - Deleted pod: TaskInstanceKey(dag_id='dbt-datahub3', task_id='dbt-run', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) in namespace datahubtest
[2023-03-08 13:25:47,090] {scheduler_job.py:588} INFO - Executor reports execution of dbt-datahub3.dbt-run run_id=manual__2023-03-08T13:25:26.874182+00:00 exited with status None for try_number 1
[2023-03-08 13:25:47,757] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type MODIFIED
[2023-03-08 13:25:47,757] {kubernetes_executor.py:219} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 is Running
[2023-03-08 13:25:52,768] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type MODIFIED
[2023-03-08 13:25:52,768] {kubernetes_executor.py:219} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 is Running
[2023-03-08 13:25:53,077] {dagrun.py:597} INFO - Marking run <DagRun dbt-datahub3 @ 2023-03-08 13:25:26.874182+00:00: manual__2023-03-08T13:25:26.874182+00:00, state:running, queued_at: 2023-03-08 13:25:26.882341+00:00. externally triggered: True> successful
[2023-03-08 13:25:53,078] {dagrun.py:644} INFO - DagRun Finished: dag_id=dbt-datahub3, execution_date=2023-03-08 13:25:26.874182+00:00, run_id=manual__2023-03-08T13:25:26.874182+00:00, run_start_date=2023-03-08 13:25:27.768180+00:00, run_end_date=2023-03-08 13:25:53.078112+00:00, run_duration=25.309932, state=success, external_trigger=True, run_type=manual, data_interval_start=2023-03-07 00:00:00+00:00, data_interval_end=2023-03-08 00:00:00+00:00, dag_hash=2e078fcb467b387d8c788854319f9b3a
[2023-03-08 13:25:53,083] {dag.py:3336} INFO - Setting next_dagrun for dbt-datahub3 to 2023-03-08T00:00:00+00:00, run_after=2023-03-09T00:00:00+00:00
[2023-03-08 13:25:54,777] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type MODIFIED
[2023-03-08 13:25:54,777] {kubernetes_executor.py:212} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 Succeeded
[2023-03-08 13:25:54,824] {kubernetes_executor.py:383} INFO - Attempting to finish pod; pod_id: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07; state: None; annotations: {'dag_id': 'dbt-datahub3', 'task_id': 'end_task', 'execution_date': None, 'run_id': 'manual__2023-03-08T13:25:26.874182+00:00', 'try_number': '1'}
[2023-03-08 13:25:54,824] {kubernetes_executor.py:598} INFO - Changing state of (TaskInstanceKey(dag_id='dbt-datahub3', task_id='end_task', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1), None, 'dbtdatahub3endtask-da871afe935944a8b6f344d991242e07', 'datahubtest', '184454541') to None
[2023-03-08 13:25:54,846] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type MODIFIED
[2023-03-08 13:25:54,846] {kubernetes_executor.py:212} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 Succeeded
[2023-03-08 13:25:54,853] {kubernetes_executor.py:696} INFO - Deleted pod: TaskInstanceKey(dag_id='dbt-datahub3', task_id='end_task', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) in namespace datahubtest
[2023-03-08 13:25:54,854] {scheduler_job.py:588} INFO - Executor reports execution of dbt-datahub3.end_task run_id=manual__2023-03-08T13:25:26.874182+00:00 exited with status None for try_number 1
[2023-03-08 13:25:54,855] {kubernetes_executor.py:150} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 had an event of type DELETED
[2023-03-08 13:25:54,855] {kubernetes_executor.py:212} INFO - Event: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07 Succeeded
[2023-03-08 13:25:54,905] {kubernetes_executor.py:383} INFO - Attempting to finish pod; pod_id: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07; state: None; annotations: {'dag_id': 'dbt-datahub3', 'task_id': 'end_task', 'execution_date': None, 'run_id': 'manual__2023-03-08T13:25:26.874182+00:00', 'try_number': '1'}
[2023-03-08 13:25:54,905] {kubernetes_executor.py:383} INFO - Attempting to finish pod; pod_id: dbtdatahub3endtask-da871afe935944a8b6f344d991242e07; state: None; annotations: {'dag_id': 'dbt-datahub3', 'task_id': 'end_task', 'execution_date': None, 'run_id': 'manual__2023-03-08T13:25:26.874182+00:00', 'try_number': '1'}
[2023-03-08 13:25:54,906] {kubernetes_executor.py:598} INFO - Changing state of (TaskInstanceKey(dag_id='dbt-datahub3', task_id='end_task', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1), None, 'dbtdatahub3endtask-da871afe935944a8b6f344d991242e07', 'datahubtest', '184454542') to None
[2023-03-08 13:25:54,910] {kubernetes_executor.py:696} INFO - Deleted pod: TaskInstanceKey(dag_id='dbt-datahub3', task_id='end_task', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) in namespace datahubtest
[2023-03-08 13:25:54,910] {kubernetes_executor.py:598} INFO - Changing state of (TaskInstanceKey(dag_id='dbt-datahub3', task_id='end_task', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1), None, 'dbtdatahub3endtask-da871afe935944a8b6f344d991242e07', 'datahubtest', '184454543') to None
[2023-03-08 13:25:54,915] {kubernetes_executor.py:696} INFO - Deleted pod: TaskInstanceKey(dag_id='dbt-datahub3', task_id='end_task', run_id='manual__2023-03-08T13:25:26.874182+00:00', try_number=1, map_index=-1) in namespace datahubtest
[2023-03-08 13:25:54,915] {scheduler_job.py:588} INFO - Executor reports execution of dbt-datahub3.end_task run_id=manual__2023-03-08T13:25:26.874182+00:00 exited with status None for try_number 1
Dag code used:
default_args = {
"owner": "someone",
"depends_on_past": False,
"start_date": datetime(year=2023, month=3, day=6),
"email": [],
"email_on_failure": False,
"email_on_retry": False,
"retries": 0,
"retry_delay": timedelta(minutes=5),
}
dag = DAG(
"dbt-datahub3", default_args=default_args, schedule_interval="@daily", max_active_runs=1
)
dummyStart = DummyOperator(
dag=dag,
task_id="start_task",
)
job = ConveyorContainerOperatorV2(
dag=dag,
task_id="dbt-run",
arguments=["build", "--target", "datahubtest"],
)
dummyEnd = DummyOperator(
dag=dag,
task_id="end_task",
)
dummyStart >> job >> dummyEnd
What you think should happen instead
I expect it to be consistent and that no matter whether the EmptyOperator is in your dag, the same behavior is observed (it is never processed by the executor-.
How to reproduce
Create 1 dag containing:
- a start emptyOperator task
- a random task (in our case a simple containerTask)
- an end emptyOperator task
Operating System
kubernetes
Versions of Apache Airflow Providers
apache-airflow-providers-amazon==6.0.0
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.0.2
apache-airflow-providers-common-sql==1.3.3
apache-airflow-providers-docker==3.2.0
apache-airflow-providers-elasticsearch==4.2.1
apache-airflow-providers-ftp==3.3.0
apache-airflow-providers-google==8.4.0
apache-airflow-providers-grpc==3.0.0
apache-airflow-providers-hashicorp==3.1.0
apache-airflow-providers-http==4.1.1
apache-airflow-providers-imap==3.1.1
apache-airflow-providers-microsoft-azure==4.3.0
apache-airflow-providers-mysql==3.2.1
apache-airflow-providers-odbc==3.1.2
apache-airflow-providers-opsgenie==3.1.0
apache-airflow-providers-postgres==5.2.2
apache-airflow-providers-redis==3.0.0
apache-airflow-providers-sendgrid==3.0.0
apache-airflow-providers-sftp==4.1.0
apache-airflow-providers-slack==4.2.3
apache-airflow-providers-sqlite==3.3.1
apache-airflow-providers-ssh==3.2.0
Deployment
Other Docker-based deployment
Deployment details
/
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct