Skip to content

Conversation

@eumiro
Copy link
Contributor

@eumiro eumiro commented Aug 19, 2023

In all these cases, complete lists are created although only their length or the first element are needed. This PR introduces shortcuts.

@boring-cyborg boring-cyborg bot added area:CLI area:dev-tools area:providers area:webserver Webserver related Issues provider:cncf-kubernetes Kubernetes (k8s) provider related issues provider:databricks provider:google Google (including GCP) related issues labels Aug 19, 2023
@eumiro eumiro force-pushed the fewer-lists branch 2 times, most recently from 4b3d738 to 70abde4 Compare August 19, 2023 17:40
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this test seems wrong actually. The current logic checks there’s at least one task with no deps set, but the comment suggests it should instead test that no deps are set at all.

@ephraimbuddy (author of this) can you confirm which is the original intention here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really wrong but can be improved. We should assert that non of the tasks have dependencies probably by checking the total tasks without dependencies:
assert len([x for x in dag.tasks if not x.downstream_list])==6

@potiuk potiuk merged commit 4154cc0 into apache:main Aug 24, 2023
@eumiro eumiro deleted the fewer-lists branch August 24, 2023 16:16
@ephraimbuddy ephraimbuddy added the type:misc/internal Changelog: Misc changes that should appear in change log label Oct 2, 2023
@ephraimbuddy ephraimbuddy added this to the Airflow 2.7.2 milestone Oct 2, 2023
ephraimbuddy pushed a commit that referenced this pull request Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:CLI area:dev-tools area:providers area:webserver Webserver related Issues provider:cncf-kubernetes Kubernetes (k8s) provider related issues provider:databricks provider:google Google (including GCP) related issues type:misc/internal Changelog: Misc changes that should appear in change log

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants