Stop reserializing DAGs during db migration #45362
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
At first, this may seem strange, but we already have to deal with older versions of serialized DAGs - so the necessity of reserializing during migrations isn't there any longer. The DAG processor can simply do it once Airflow has started back up.
1 task
dstandish
approved these changes
Jan 2, 2025
potiuk
approved these changes
Jan 2, 2025
Member
Author
|
Failure is unrelated (see #45329). |
HariGS-DB
pushed a commit
to HariGS-DB/airflow
that referenced
this pull request
Jan 16, 2025
At first, this may seem strange, but we already have to deal with older versions of serialized DAGs - so the necessity of reserializing during migrations isn't there any longer. The DAG processor can simply do it once Airflow has started back up.
got686-yandex
pushed a commit
to got686-yandex/airflow
that referenced
this pull request
Jan 30, 2025
At first, this may seem strange, but we already have to deal with older versions of serialized DAGs - so the necessity of reserializing during migrations isn't there any longer. The DAG processor can simply do it once Airflow has started back up.
This was referenced Apr 6, 2025
kuyperse
added a commit
to aws/amazon-mwaa-docker-images
that referenced
this pull request
Aug 25, 2025
*Issue #, if available:* n/a *Description of changes:* Disables dag serialization during Airflow db migrations for the migrate-db command. The command is intended to be used without custom plugins/requirements/startup script that could interfere with db operations. This means that it is unable to serialize dags which require dependencies which have not been installed. So we need to disable dag serialization during the migration. This is ok because the dag processor is responsible for dag serialization and will handle it after the db migration/initialization. The flag was removed from Airflow as of Airflow 2.11.0 because the dag processor makes it redundant: apache/airflow#45362. For testing I deployed a control environment and test environment to verify the dag serialization errors are resolved by the fix. The dags functioned as expected with the flag disabled. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
jedcunningham
added a commit
to astronomer/airflow
that referenced
this pull request
Nov 14, 2025
The `reserialize_dags` parameter was left in the `upgradedb` function signature after the reserialization code was removed in PR apache#45362. This parameter is not used anywhere in the codebase and can be safely removed.
jedcunningham
added a commit
that referenced
this pull request
Nov 21, 2025
The `reserialize_dags` parameter was left in the `upgradedb` function signature after the reserialization code was removed in PR #45362. This parameter is not used anywhere in the codebase and can be safely removed.
Copilot AI
pushed a commit
to jason810496/airflow
that referenced
this pull request
Dec 5, 2025
The `reserialize_dags` parameter was left in the `upgradedb` function signature after the reserialization code was removed in PR apache#45362. This parameter is not used anywhere in the codebase and can be safely removed.
itayweb
pushed a commit
to itayweb/airflow
that referenced
this pull request
Dec 6, 2025
The `reserialize_dags` parameter was left in the `upgradedb` function signature after the reserialization code was removed in PR apache#45362. This parameter is not used anywhere in the codebase and can be safely removed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
At first, this may seem strange, but we already have to deal with older versions of serialized DAGs - so the necessity of reserializing during migrations isn't there any longer. The DAG processor can simply do it once Airflow has started back up.p
I've opened #45361 as a check point, once the dust has settled, to see if we need to add it back.