Option to make UI centered around execution date rather than the start_date of interval

[dennis-weyland-by]

Description

Consider the following example:
The current datetime is 2022-02-10 09:45 CET and airflow executed two runs:

• daily_run scheduled to run every day at 09:43 CET
• maintenance triggered manually at 2022-02-10 09:41 CET

In the UI it looks like this:

From someone new to airflow there are some unintuitive things for display:

• Although both dags ran on the same day, they display different dates in the last run
• The DateTime in the Next run column for daily_run is in the past

On a technical level, this is documented. Airflow runs its schedule after the schedule interval is completed and what is displayed is the start_date of the interval similar to the start_date parameter of a DAG. As a developer, which works with Airflow, I have to read the documentation and act accordingly. So far this is fine.

My point is, that not every consumer of the UI is familiar with Airflow and to some, this is very confusing and counterintuitive.
As a developer, I would like to have the option to switch the displayed days to the execution/run after date. This information is also already there when using the info tooltip:

Summary
Introduce a config flag for the web-server that on the front page the actual time of the future/past execution of a scheduled dag is shown. Since this is a behavior inconsistent with internal airflow behavior, the flag should be disabled by default.

Optional: Extending this to DAg Details page
The Frontpage is not the only place the behavior is shown. So it might make sense that the flag would also change the following examples:
Tree View:

• Upper right: next Run
• Calendar Filter
• Column Name (and ordering) in Tree View

Calendar:

Are you willing to submit a PR?
My lack of javascript logic prevents me from doing a pr myself, but I would be willing to help on this issue if needed.

Use case/motivation

I use airflow to schedule/orchestrate ML pipelines like model training, predicting, and so on. I want to use the Airflow UI to further enable those pipelines to an audience, which is not accustomed to airflow. It turned out that the current behavior of how dates are displayed (especially with a mix of scheduled and manually triggered tasks) is very confusing for this use case and makes it hard to onboard people as WebUI-only users.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[bbovenzi]

@uranusjr I'd be curious to hear your thoughts after all of the recent schedule_interval changes. I'm happy to update the UI to work with whatever is most useful for users.

[dennis-weyland-by]

@uranusjr @bbovenzi any updates?

[uranusjr]

If I understand what you’re trying to say correctly, the situaion is actually entirely the other way around. It evolves around execution date. All the UI confusions you have (runs at the same date displayed as different dates) happen because those dates are execution dates, not data intervals. Yes, a manual run’s execution date points to its data interval’s end. So to solve your problem, the UI needs to be modified to evolve around the data interval instead (and I think it actually may make more sense to make it evolve around the end of the data interval than the start), not the execution date.

[dennis-weyland-by]

Yes, thank you. I wasn't aware that the execution date is aligned with the start of the interval and not the end. But yes I agree, that the UI should display the end of the interval then. Do you think that this should be default behavior or would you rather prefer a feature flag and stay with the current behavior as default?

[uranusjr]

Not sure (honestly I kind of semi-delibrately avoided this when we first added data intervals to the hover dialog box); personally I’d want to just flip everything over, but @bbovenzi probably has a better idea how backward compatibility needs to be dealt with in the web UI.

[dennis-weyland-by]

Yes totally agree on this point. From a personal point, I really find the current UI behavior confusing but it's established. And it seems like the focus on interval start date times runs deep. Seems like even dag identifier for scheduled runs relies on that. @bbovenzi can you comment your thoughts?

[bbovenzi]

Yes, the data interval end date lines up better. It shouldn't be hard to update all the next run areas of the UI to use that instead.
Also, I think we show too many dates in our tooltips which also causes confusion. I'm playing around with a better details view, to place all of these other dates so that we can simplify our at-a-glance tooltips.

[dennis-weyland-by]

After giving it some thought: While it's a valid discussion, if interval_start or interval_end date is the more intuitive one, the bottom line here was that the confusion was since my use case doesn't need a distinction between interval_start and interval_end date. For such cases Airflow has Timetables to reduce the interval to a single point in time and then the UI is not confusing anymore.

To me, it feels like relying on timetables for such a feature is much more consistent as switching defaults when airflow is very much focused on interval_start_dates.