Fix: DAGs are not marked as stale if the dags folder change #41433

utkarsharma2 · 2024-08-13T13:59:01Z

Closes: #41432

…hanges

airflow/dag_processing/manager.py

uranusjr · 2024-08-22T03:18:26Z

Is a test possible?

utkarsharma2 · 2024-08-22T05:14:49Z

Is a test possible?

Yes, I'll add them.

utkarsharma2 · 2024-08-26T12:34:01Z

Is a test possible?

@uranusjr, @Lee-W I have added a test, PTAL.

tests/dag_processing/test_job_runner.py

airflow/dag_processing/manager.py

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

ephraimbuddy · 2024-08-28T09:36:21Z

@utkarsharma2 , can you backport this to v2-10-test

utkarsharma2 · 2024-08-28T10:03:46Z

Thanks @ephraimbuddy I'll do that.

…1433) * Fix: DAGs are not marked as stale if the AIRFLOW__CORE__DAGS_FOLDER changes * Update airflow/dag_processing/manager.py * Add testcase * Add code comment * Update code comment * Update the logic for checking the current dag_directory * Update testcases * Remove unwanted code * Uncomment code * Add processor_subdir when creating processor_subdir * Fix test_retry_still_in_executor test * Remove config from test * Update airflow/dag_processing/manager.py Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> * Update if condition for readability --------- Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> (cherry picked from commit 9f30a41)

utkarsharma2 · 2024-08-28T11:15:43Z

Cherry picked - #41829

…41829) * Fix: DAGs are not marked as stale if the AIRFLOW__CORE__DAGS_FOLDER changes * Update airflow/dag_processing/manager.py * Add testcase * Add code comment * Update code comment * Update the logic for checking the current dag_directory * Update testcases * Remove unwanted code * Uncomment code * Add processor_subdir when creating processor_subdir * Fix test_retry_still_in_executor test * Remove config from test * Update airflow/dag_processing/manager.py Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> * Update if condition for readability --------- Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> (cherry picked from commit 9f30a41)

…pache#41433)" This reverts commit 9f30a41.

…41829) * Fix: DAGs are not marked as stale if the AIRFLOW__CORE__DAGS_FOLDER changes * Update airflow/dag_processing/manager.py * Add testcase * Add code comment * Update code comment * Update the logic for checking the current dag_directory * Update testcases * Remove unwanted code * Uncomment code * Add processor_subdir when creating processor_subdir * Fix test_retry_still_in_executor test * Remove config from test * Update airflow/dag_processing/manager.py Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> * Update if condition for readability --------- Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> (cherry picked from commit 9f30a41)

…pache#41433)" This reverts commit 9f30a41.

…41433)" (#42197) This reverts commit 9f30a41.

…pache#41433) (apache#41829)" This reverts commit 996af78.

…(41433) (#41829) (#41893) 41829 (#42220) * Revert "Skip test_scan_stale_dags_when_dag_folder_change in DB isolation mode (#41893)" This reverts commit 07af14a. * Revert "Fix: DAGs are not marked as stale if the dags folder change (#41433) (#41829)" This reverts commit 996af78.

…pache#41433)" (apache#42197) This reverts commit 9f30a41.

Fix: DAGs are not marked as stale if the AIRFLOW__CORE__DAGS_FOLDER c…

74ce043

…hanges

utkarsharma2 requested review from ephraimbuddy and jedcunningham as code owners August 13, 2024 13:59

boring-cyborg bot added the area:Scheduler including HA (high availability) scheduler label Aug 13, 2024

Merge branch 'main' into FixDagParsing

b881678

utkarsharma2 commented Aug 13, 2024

View reviewed changes

airflow/dag_processing/manager.py Outdated Show resolved Hide resolved

utkarsharma2 added 3 commits August 13, 2024 19:31

Update airflow/dag_processing/manager.py

a41d411

Merge branch 'main' into FixDagParsing

00f2b0a

Merge branch 'main' into FixDagParsing

40c6cb0

Lee-W reviewed Aug 21, 2024

View reviewed changes

airflow/dag_processing/manager.py Outdated Show resolved Hide resolved

utkarsharma2 added 3 commits August 26, 2024 18:01

Add testcase

5e11d81

Add code comment

252e473

Update code comment

df185ba

utkarsharma2 requested review from Lee-W and uranusjr August 26, 2024 12:34

utkarsharma2 added 2 commits August 26, 2024 18:04

Merge branch 'main' into FixDagParsing

9898b39

Update the logic for checking the current dag_directory

75c9d93

ephraimbuddy reviewed Aug 26, 2024

View reviewed changes

tests/dag_processing/test_job_runner.py Outdated Show resolved Hide resolved

tests/dag_processing/test_job_runner.py Outdated Show resolved Hide resolved

tests/dag_processing/test_job_runner.py Outdated Show resolved Hide resolved

utkarsharma2 added 4 commits August 26, 2024 20:24

Update testcases

4b745b8

Remove unwanted code

79cd22e

Merge branch 'main' into FixDagParsing

da5ae39

Uncomment code

055192a

utkarsharma2 force-pushed the FixDagParsing branch from bb61051 to 055192a Compare August 26, 2024 15:08

Merge branch 'main' into FixDagParsing

245bef8

utkarsharma2 requested a review from ephraimbuddy August 26, 2024 15:10

ephraimbuddy reviewed Aug 26, 2024

View reviewed changes

tests/dag_processing/test_job_runner.py Show resolved Hide resolved

utkarsharma2 requested a review from ephraimbuddy August 26, 2024 16:15

ephraimbuddy approved these changes Aug 27, 2024

View reviewed changes

jedcunningham reviewed Aug 28, 2024

View reviewed changes

airflow/dag_processing/manager.py Outdated Show resolved Hide resolved

airflow/dag_processing/manager.py Outdated Show resolved Hide resolved

utkarsharma2 and others added 3 commits August 28, 2024 11:54

Remove config from test

3545a7a

Update airflow/dag_processing/manager.py

3917819

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

Merge branch 'main' into FixDagParsing

da42bab

utkarsharma2 requested a review from jedcunningham August 28, 2024 06:36

Update if condition for readability

fa17bc6

utkarsharma2 merged commit 9f30a41 into apache:main Aug 28, 2024

utkarsharma2 deleted the FixDagParsing branch August 28, 2024 09:07

ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Aug 29, 2024

Revert "Fix: DAGs are not marked as stale if the dags folder change (a…

60c2fa2

…pache#41433)" This reverts commit 9f30a41.

utkarsharma2 added the type:bug-fix Changelog: Bug Fixes label Aug 30, 2024

utkarsharma2 added this to the Airflow 2.10.1 milestone Aug 30, 2024

jcrobak mentioned this pull request Sep 9, 2024

DAGs are missing and deleted in Airflow 2.10.1 #42111

Closed

2 tasks

utkarsharma2 added a commit to astronomer/airflow that referenced this pull request Sep 12, 2024

Revert "Fix: DAGs are not marked as stale if the dags folder change (a…

3d432dd

…pache#41433)" This reverts commit 9f30a41.

utkarsharma2 mentioned this pull request Sep 12, 2024

Revert "Fix: DAGs are not marked as stale if the dags folder change #42197

Merged

ephraimbuddy pushed a commit that referenced this pull request Sep 12, 2024

Revert "Fix: DAGs are not marked as stale if the dags folder change (#…

3bc2d3a

…41433)" (#42197) This reverts commit 9f30a41.

ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Sep 13, 2024

Revert "Fix: DAGs are not marked as stale if the dags folder change (a…

2bda0a2

…pache#41433) (apache#41829)" This reverts commit 996af78.

joaopamaral pushed a commit to joaopamaral/airflow that referenced this pull request Oct 21, 2024

Revert "Fix: DAGs are not marked as stale if the dags folder change (a…

c942342

…pache#41433)" (apache#42197) This reverts commit 9f30a41.

jedcunningham mentioned this pull request Feb 18, 2025

[v2-10-test] Don't resolve path for DAGs folder #46877

Merged

kaxil mentioned this pull request May 15, 2025

Status of testing of Apache Airflow 2.11.0rc1 #50672

Closed

39 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix: DAGs are not marked as stale if the dags folder change #41433

Fix: DAGs are not marked as stale if the dags folder change #41433

Uh oh!

utkarsharma2 commented Aug 13, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

uranusjr commented Aug 22, 2024

Uh oh!

utkarsharma2 commented Aug 22, 2024

Uh oh!

utkarsharma2 commented Aug 26, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ephraimbuddy commented Aug 28, 2024

Uh oh!

utkarsharma2 commented Aug 28, 2024

Uh oh!

utkarsharma2 commented Aug 28, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Fix: DAGs are not marked as stale if the dags folder change #41433

Fix: DAGs are not marked as stale if the dags folder change #41433

Uh oh!

Conversation

utkarsharma2 commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

uranusjr commented Aug 22, 2024

Uh oh!

utkarsharma2 commented Aug 22, 2024

Uh oh!

utkarsharma2 commented Aug 26, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ephraimbuddy commented Aug 28, 2024

Uh oh!

utkarsharma2 commented Aug 28, 2024

Uh oh!

utkarsharma2 commented Aug 28, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

utkarsharma2 commented Aug 13, 2024 •

edited

Loading