Skip to content

Conversation

@vatsrahul1001
Copy link
Contributor

When the DAG bundle name is changed the dag_version.bundle_name field still points to the old, inactive bundle.
This causes a ValueError: Requested bundle is not configured error when accessing affected DAGs.

closes: #49861

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@TJaniF
Copy link
Contributor

TJaniF commented Apr 28, 2025

Tested this with and it fixes the issue reported here: #49861 for me.

Also tested with 2 local dag bundles:

started with this:

AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[
    {
        "name": "dags-folder1",
        "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
        "kwargs": {
            "path": "/files/dags/"
        }
    },
    {
        "name": "dags-folder2",
        "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
        "kwargs": {
            "path": "/files/dags2/"
        }
    }
]'

Ran both dags, added a task to both dags, ran both dags again so both have a v2 version
then restart with

AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[
    {
        "name": "dags-folder1-RENAMED",
        "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
        "kwargs": {
            "path": "/files/dags/"
        }
    },
    {
        "name": "dags-folder2",
        "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
        "kwargs": {
            "path": "/files/dags2/"
        }
    }
]'

All runs correctly

Removing one of the bundles also works as expected: the dag history is still shown but running it is not possible.

No API server errors :)

@vatsrahul1001 vatsrahul1001 requested a review from dstandish April 28, 2025 13:54
Copy link
Contributor

@phanikumv phanikumv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a test covering this

@vatsrahul1001 vatsrahul1001 requested a review from phanikumv April 29, 2025 14:18
@vatsrahul1001
Copy link
Contributor Author

Please add a test covering this

Done

@vatsrahul1001 vatsrahul1001 added this to the Airflow 3.0.1 milestone Apr 30, 2025
@vatsrahul1001 vatsrahul1001 merged commit c675d74 into apache:main Apr 30, 2025
51 checks passed
@vatsrahul1001 vatsrahul1001 deleted the dag-version-bundle-mismatch-fix branch April 30, 2025 09:21
kaxil pushed a commit that referenced this pull request Apr 30, 2025
…uration changes (#49886)

Auto-correct stale bundle_name in DAG versions when DAG bundle configuration changes (#49886)

(cherry picked from commit c675d74)
@ephraimbuddy
Copy link
Contributor

While this worked, it doesn't seem like the right solution. A bundle was chosen randomly and versions updated.

ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Jun 18, 2025
If there's a bundle name change without the dag structure changing,
we should update the dag version to reflect the change.

This issue was fixed in apache#49886 but the fix randomly assigned versions
to the new bundle.
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Jun 18, 2025
If there's a bundle name change without the dag structure changing,
we should update the dag version to reflect the change.

This issue was fixed in apache#49886 but the fix randomly assigned versions
to the new bundle.
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Jun 19, 2025
If there's a bundle name change without the dag structure changing,
we should update the dag version to reflect the change.

This issue was fixed in apache#49886 but the fix randomly assigned versions
to the new bundle.
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Jun 19, 2025
If there's a bundle name change without the dag structure changing,
we should update the dag version to reflect the change.

This issue was fixed in apache#49886 but the fix randomly assigned versions
to the new bundle.
ephraimbuddy added a commit that referenced this pull request Jun 19, 2025
* Update dag version when bundle name changes

If there's a bundle name change without the dag structure changing,
we should update the dag version to reflect the change.

This issue was fixed in #49886 but the fix randomly assigned versions
to the new bundle.

* fixup! Update dag version when bundle name changes

* fixup! fixup! Update dag version when bundle name changes

* fixup! fixup! fixup! Update dag version when bundle name changes

* Change this to create new version when the bundle name changes
github-actions bot pushed a commit that referenced this pull request Jun 19, 2025
* Update dag version when bundle name changes

If there's a bundle name change without the dag structure changing,
we should update the dag version to reflect the change.

This issue was fixed in #49886 but the fix randomly assigned versions
to the new bundle.

* fixup! Update dag version when bundle name changes

* fixup! fixup! Update dag version when bundle name changes

* fixup! fixup! fixup! Update dag version when bundle name changes

* Change this to create new version when the bundle name changes
(cherry picked from commit cc7abdc)

Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>
RoyLee1224 pushed a commit to RoyLee1224/airflow that referenced this pull request Jun 21, 2025
* Update dag version when bundle name changes

If there's a bundle name change without the dag structure changing,
we should update the dag version to reflect the change.

This issue was fixed in apache#49886 but the fix randomly assigned versions
to the new bundle.

* fixup! Update dag version when bundle name changes

* fixup! fixup! Update dag version when bundle name changes

* fixup! fixup! fixup! Update dag version when bundle name changes

* Change this to create new version when the bundle name changes
kaxil pushed a commit that referenced this pull request Jul 2, 2025
)

If there's a bundle name change without the dag structure changing,
we should update the dag version to reflect the change.

This issue was fixed in #49886 but the fix randomly assigned versions
to the new bundle.

* fixup! Update dag version when bundle name changes

* fixup! fixup! Update dag version when bundle name changes

* fixup! fixup! fixup! Update dag version when bundle name changes

* Change this to create new version when the bundle name changes
(cherry picked from commit cc7abdc)

Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Removing a dag bundle with previous history from Airflow causes a ValueError: Requested bundle is not configured error

4 participants