Add details drawer to Grid View #22123

bbovenzi · 2022-03-09T16:03:34Z

Add a details drawer to the right side of the grid view to quickly see details and actions for the DAG / Dag Run / Task Instance. This replaces the modal when clicking on a dag run or task instance. (Modals still exist for graph and gantt views)

Benefits:

Simplify tooltips to only show basic at-a-glance info, and rely on the drawer for more advanced details
Show details in context of what run or task you are looking at
Dag Run and Task Instance actions in context of what Task/Run you are changing
Run and Task actions is a seamless experience without multiple page redirects. And one can immediately see changes take place.

Next Up:

The side drawer for a mapped task will include a table of all of its mapped instances with their respective actions
A better DAG details side drawer. In this PR, it is a table of some basic details, but I would like for it to be an overview of your DAG and the runs/tasks that are currently being displayed in the grid view

Closes #19938
Closes #21428
Related #20382
Closes #19674

Other notes:
Python isn't my strong suit so any help with the changes I had to make to the webserver (views.py and utils.py) would be greatly appreciated.

Dag Details:

Run Details:

Task Instance Details:

Navigating between details:

Scrolling:

Marking a Task Failed:

Clearing a DAG Run:

Changing Timezone:

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

ashb · 2022-03-10T12:24:08Z

airflow/www/static/js/tree/InstanceTooltip.jsx

+        Duration:
        {' '}
-        {endDate && formatDateTime(endDate)}
+        {formatDuration(duration || getDuration(startDate, endDate))}


What does this show when the task is still in progress?

Do we not show end date anymore?

endDate is in the details but not in the tooltip.
When in progress it will use Date.now() instead of enddate

bbovenzi · 2022-03-17T18:20:44Z

airflow/www/static/js/dag.js

  }
 }

-export function callModalDag(dag) {


We only called this in the grid view, so we can fully replace it

bbovenzi · 2022-03-17T18:53:13Z

airflow/www/templates/airflow/dag.html

      </div>
    </div>
  </div>
-  <!-- Modal for DAG run -->


This is now all in React instead.

eladkal · 2022-03-17T20:10:57Z

Looks really good!

2 comments:

for me the arrow is associated with the browser arrows (back / forward). In this case the action is close so wouldn't X icon be more suitable?
I think in the confirm pop-ups we should default the confirm button

ashb · 2022-03-17T20:57:59Z

I think in the confirm pop-ups we should default the confirm button

What did the current confirm page default to?

bbovenzi · 2022-03-17T20:59:12Z

I think in the confirm pop-ups we should default the confirm button

What did the current confirm page default to?

It highlighted "Cancel" by default.

eladkal · 2022-03-20T20:21:02Z

Played with it a bit in local env this is really really good!

Have a few suggestions/bugs all are minor:

[Suggestion] Can we have the ability to easily copy to clipboard?

To give example of what i had in mind:
[Tweak] Need to fix the name to Grid also in:
[Bug] When changing Timezone there is a weird "jump" of values :

Also the timezone changes only in start date but not in Timezone (last row in the table) - why?
Is this the timezone the DAG is defined with in the code? If so maybe better to call it DAG defined timezone or something like that?
[Bug] When opening the panel of a fresh task (state None) the panel isn't refreshed while the task progresses. So when the task finishes the started/ended values are empty (you must refresh the screen manually to solve this)
The values in the panel should be refreshed when with the auto-refresh property.
(Edit: noticed also the status left as no-status though that is Success and probably duration is also not updated.. so I guess the whole panel isn't being updated with the auto refresh functionality)
[Suggestion] I got a bit lost when clicked on Filter upstream. It was impossible to return where I was before I had to enter the Grid view and repeat my steps. Is there a way to make a better experience with it?
[Suggestion] There are two "More details" one that leads to DAG details and one that leads to Task Details. Can we simply rename it to be "DAG details"/"Task Details"?
The same goes for "Show Details Panel" It shows different panel for Task/DAG but the button has the same name.
[Bug] I'm not really sure how I managed to do that but I get a tooltip of Show when the button says hide and vise versia. The tool tip is redundant any anyway isn't it?

bbovenzi · 2022-03-21T17:25:20Z

Thanks for all the testing @eladkal!

I can do that in a later PR
Already fixed in a different PR
The whole DAG details page will change pretty soon, so I won't fix for now
I think this should be a later PR. It should totally happen, but to do correctly I think we need to redo how tree_data works in views.py. I will absolutely do it before 2.3.0, but I think making that change to this PR will make it even harder than it already is to review all of the code changes. (Edit: I figured out a workable solution for this PR.)
Added a "Reset Root" button if you filter upstream
Updated all the "More Details" to specifiy which details we are linking to
That title was backwards and now redundant. Removed.

eladkal · 2022-03-21T18:56:37Z

Great. Since I'm not familiar with the UI side of the code I can't do a proper code review but I can confirm that functionality wise it work as expected.

airflow/www/static/js/tree/StatusBox.jsx

airflow/www/static/js/tree/details/Header.jsx

airflow/www/static/js/tree/details/content/Dag.jsx

airflow/www/static/js/tree/details/content/dagRun/ClearRun.jsx

airflow/www/static/js/tree/details/content/dagRun/MarkFailedRun.jsx

airflow/www/static/js/tree/details/content/dagRun/index.jsx

airflow/www/static/js/tree/details/content/dagRun/QueueRun.jsx

airflow/www/static/js/tree/details/content/taskInstance/ExtraLinks.jsx

airflow/www/static/js/tree/details/content/taskInstance/taskActions/Clear.jsx

airflow/www/views.py

uranusjr · 2022-03-24T04:11:47Z

Python logic looks good to me, but we can probably do better sharing logic between HTML and JSON requests.

airflow/www/views.py

uranusjr

For the Python part

github-actions · 2022-03-25T15:24:39Z

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

uranusjr · 2022-03-28T11:09:00Z

I added a commit to fix the KeyError. There are still a couple of failures relating to argument count that need fixing, I’ll leave those to you.

- Remove details panel title - Add button to reset root - Make "More Details" buttons more specific - Specify timezone as DAG timezone

None of these changes were relevant to this PR. Better to be done separately.

- useState vs useDisclosure - Remove extraneous elements - Copy changes - Wire up params for runTask - Breadcrumb padding

…Data hooks

- Pass 'Accept' headers for json returns - Consolidate more endpoints to return json or redirect

- add readme - rename context providers to avoid confusion with Airflow Providers

boring-cyborg bot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Mar 9, 2022

ashb reviewed Mar 10, 2022

View reviewed changes

bbovenzi force-pushed the mapped-task-drawer branch from 74d739c to 1a9e6f3 Compare March 15, 2022 00:50

bbovenzi marked this pull request as ready for review March 17, 2022 17:10

bbovenzi requested a review from ryanahamilton as a code owner March 17, 2022 17:10

bbovenzi commented Mar 17, 2022

View reviewed changes

ashb added the area:dynamic-task-mapping AIP-42 label Mar 17, 2022

bbovenzi added the AIP-38 Modern Web Application label Mar 17, 2022

bbovenzi added this to the Airflow 2.3.0 milestone Mar 17, 2022

bbovenzi force-pushed the mapped-task-drawer branch from 74fada2 to 60445c4 Compare March 21, 2022 17:19

This was referenced Mar 21, 2022

Streamline task action UX #20382

Closed

See individual mapped tasks in grid and graph views #22409

Closed

ryanahamilton reviewed Mar 22, 2022

View reviewed changes

bbovenzi requested a review from uranusjr March 23, 2022 16:57

uranusjr reviewed Mar 24, 2022

View reviewed changes

airflow/www/views.py Outdated Show resolved Hide resolved

uranusjr reviewed Mar 24, 2022

View reviewed changes

airflow/www/views.py Outdated Show resolved Hide resolved

uranusjr reviewed Mar 25, 2022

View reviewed changes

airflow/www/views.py Outdated Show resolved Hide resolved

uranusjr approved these changes Mar 25, 2022

View reviewed changes

github-actions bot added the okay to merge It's ok to merge this PR as it does not require more tests label Mar 25, 2022

bbovenzi and others added 24 commits March 31, 2022 16:57

Code cleanup and fix task clear

6c91288

Fix task/run label, dialog focus, dag details overflow, panel open/close

b6d3b2f

Add timezone provider

3b7c025

Fix TimezoneEvent import

cd2c5f1

Improve button UX

ec930f7

- Remove details panel title - Add button to reset root - Make "More Details" buttons more specific - Specify timezone as DAG timezone

autorefresh dag run details

a47ed4f

auto-refresh task instance details

912b733

revert useTreeData changes

50a8e44

None of these changes were relevant to this PR. Better to be done separately.

Address PR feedback

20aebb2

- useState vs useDisclosure - Remove extraneous elements - Copy changes - Wire up params for runTask - Breadcrumb padding

Handle task/run action sideeffects by separating autorefresh and tree…

a25704d

…Data hooks

Clean up views.py endpoints

5c60ab0

- Pass 'Accept' headers for json returns - Consolidate more endpoints to return json or redirect

pass request as arg

a3f70a9

remove request as arg

ea1ed4d

Anticipate when the 'Accept' header is not present

6229ace

Fix argument count errors

7ff5c72

Replace hard coded urls

8c36a1c

Replace hard coded urls in react components

39e732b

Update filter upstream link

4c29813

Split TaskInstance details component

09ff24f

Fix undefined variables in tests

566eca6

init_api_connexion in tests

aef7325

- add readme - rename context providers to avoid confusion with Airflow Providers

Fix url params, hide last item breadcrumb links

5824b12

Update task run failed copy

a3a01dd

Fix taskinstance/list buttons

ea3551d

bbovenzi force-pushed the mapped-task-drawer branch from bc06078 to ea3551d Compare March 31, 2022 20:57

bbovenzi merged commit 2bb26a3 into main Mar 31, 2022

bbovenzi deleted the mapped-task-drawer branch March 31, 2022 21:53

ephraimbuddy added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Apr 8, 2022

mjkonarski-b mentioned this pull request Jun 26, 2023

After upgrade from Airflow 2.2.4, grid disappears for some DAGs #23588

Closed

2 tasks

Usiel mentioned this pull request Sep 6, 2023

Ensure details panel is shown when any tab is selected #34136

Merged

Add details drawer to Grid View #22123

Add details drawer to Grid View #22123

Uh oh!

Conversation

bbovenzi commented Mar 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ashb Mar 10, 2022

Choose a reason for hiding this comment

Uh oh!

bbovenzi Mar 17, 2022

Choose a reason for hiding this comment

Uh oh!

bbovenzi Mar 17, 2022

Choose a reason for hiding this comment

Uh oh!

bbovenzi Mar 17, 2022

Choose a reason for hiding this comment

Uh oh!

eladkal commented Mar 17, 2022

Uh oh!

ashb commented Mar 17, 2022

Uh oh!

bbovenzi commented Mar 17, 2022

Uh oh!

eladkal commented Mar 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bbovenzi commented Mar 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eladkal commented Mar 21, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

uranusjr commented Mar 24, 2022

Uh oh!

Uh oh!

uranusjr left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 25, 2022

Uh oh!

uranusjr commented Mar 28, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

bbovenzi commented Mar 9, 2022 •

edited

Loading

eladkal commented Mar 20, 2022 •

edited

Loading

bbovenzi commented Mar 21, 2022 •

edited

Loading