Skip to content

Conversation

@tirkarthi
Copy link
Contributor

@tirkarthi tirkarthi commented Nov 30, 2023

closes: #35288
related: #35288

When a task is deferred then the same attempt is used to queue and resume causing the queued_dttm to be greater than start_date for an execution. This causes queued duration to be displayed with incorrect value taking queued_dttm - start_date which is not correct since start_date should always be greater than queued time. Also this causes start offset and to have a fixed queued duration bar width of 5. start offset and queued duration min width are shown only when queued_dttm is valid by being greater than start_date.

Before patch : (Incorrect queued duration of 23 hours and start of the bar is incorrect)

gantt_before_patch

After patch : (queued duration is 0 since queued_dttm is greater than start_date. start_date position is also correct in gantt as per start_date)

gantt_after

Sample dag :

from __future__ import annotations

from datetime import datetime

from airflow import DAG
from airflow.models.baseoperator import BaseOperator
from airflow.triggers.file import FileTrigger


class FileCheckOperator(BaseOperator):
    def __init__(self, filepath, **kwargs):
        self.filepath = filepath
        super().__init__(**kwargs)

    def execute(self, context):
        self.defer(
            trigger=FileTrigger(filepath=self.filepath),
            method_name="execute_complete",
        )

    def execute_complete(self, context, event=None):
        pass


with DAG(
    dag_id="file_trigger",
    start_date=datetime(2021, 1, 1),
    catchup=False,
    schedule_interval=None,
) as dag:
    t1 = FileCheckOperator(task_id="t1", filepath="/tmp/a")
    t2 = FileCheckOperator(task_id="t2", filepath="/tmp/b")

    t1
    t2

@boring-cyborg boring-cyborg bot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Nov 30, 2023
@bbovenzi bbovenzi added this to the Airflow 2.8.1 milestone Dec 5, 2023
@bbovenzi bbovenzi merged commit 0376e93 into apache:main Dec 5, 2023
@ephraimbuddy ephraimbuddy added the type:bug-fix Changelog: Bug Fixes label Dec 6, 2023
ephraimbuddy pushed a commit that referenced this pull request Dec 6, 2023
…t_date for deferred tasks. (#35984)

* Fix gantt chart queued duration when queued_dttm is greater than start_date for deferred tasks.

* Merge hasQueuedDttm and validQueuedDttm.

(cherry picked from commit 0376e93)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect queued duration for deferred tasks in gantt view

3 participants