Add a new graph inside of the grid view

[bbovenzi]

Embed the graph view inside of the grid view so a user can see history and dependencies at the same time.

Closes: #19939

Task Groups:

A complex DAG:

Dynamic task groups:

Edge labels:

Allows users to quickly clear a task group. Closes: #29380

The current graph is built off of dagre-d3 which has been deprecated for quite some time now. So in one way or another we had to update the graph.

Follow up PRs:

• Once we are confident this new graph is feature complete, delete the old graph and redirect to this one
• Update Dataset Dependencies to use the same components
• Move Task Actions out of the details tab, turn it into a modal, and have it accessible within the graph and logs tabs (and including filtering both up and down streams)
• Animate task group expand/collapse

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[notatallshaw-gts]

Looks great! I'm sure you've already thought of these use cases but just in case two things I'd be worried about with this style of straight lines is:

1. How it looks when a task has a large number of downstream tasks (e.g. 20+)?
2. Is it possible to visually distinguish lines when they have to cross each other?

The current curve lines means even though it's messy the angle of lines means it's almost always possible to unambiguously follow them, e.g.:

import datetime
from airflow.models import DAG
from airflow.operators.empty import EmptyOperator


with DAG(
    dag_id="test",
    schedule_interval="@once",
    start_date=datetime.datetime(2000, 1, 1),
) as test:
    upstream_tasks = [EmptyOperator(task_id=f"upstream_{i}") for i in range(4)]
    downstream_tasks = [EmptyOperator(task_id=f"downstream_{i}") for i in range(4)]
    for upstream_task in upstream_tasks:
        upstream_task >> downstream_tasks

    other_tasks = [EmptyOperator(task_id=f"other_{i}") for i in range(16)]
    upstream_tasks[0] >> other_tasks

[bbovenzi]

Here's the two versions of that DAG:

With task selection working across grid/graph. You can highlight the upstream/downstream even when not hovering or explicitly filtering the whole graph:

We could also experiment with concentrating edges, but there's always going to be trade-offs between different graphing strategies and various DAG structures.

[bbovenzi]

Just added the ability to zoom to a selected node to help navigate large graphs:

[eladkal]

Just added the ability to zoom to a selected node to help navigate large graphs

This is super cool!
BTW a really useful feature would be to select 2 tasks in the list then showing graph view only for these two tasks (all nodes that are between the two) it would be extremely useful for zooming in on a specific flow. Basically what you show with the blue lines but without showing all the rest of the nodes.

[potiuk]

❤️ ❤️ ❤️ ❤️ ❤️ ❤️

[notatallshaw-gts]

We could also experiment with concentrating edges, but there's always going to be trade-offs between different graphing strategies and various DAG structures.

Yeah I didn't want to kick of a design discussion in this PR and slow work down, I'm sure once this lands people will have lots of suggestions.

I just wanted to check this didn't introduce regressions in being able to follow edges in complicated graphs, and from that perspective it seems good!

[pierrejeambrun]

Tested locally and looks great! 🎉

I have a small warning complaining for width/height of parent container:

Also it looks like the webpack css loader is not working when being reloaded. (I mean hot realoding):

• start breeze in dev mode
• tail -f .build/www/asset_compile_dev_mode.out
• notice that asset compilation is successful
• update the code and observe asset rebuild, I have a css loading error on reactflow/dist/style.css. Let me know if you also experience this.

[blag]

So looking forward to this. This looks great!

[talnagar]

this feature looks amazing!

[bbovenzi]

This allows us to import the css file for react-flow inside of react

[bbovenzi]

We already have tests that cover task_group_to_dict() and dag_edges()

[ashb]

(partial review only)

[pierrejeambrun]

Very nice! 🎉

[bbovenzi]

Failing check is not because of this PR. Merging.

[jedcunningham]

(#protm)