-
Notifications
You must be signed in to change notification settings - Fork 16.4k
[AIRFLOW-6954] Use DagRunType instead of ID_PREFIX in run_id #7583
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AIRFLOW-6954] Use DagRunType instead of ID_PREFIX in run_id #7583
Conversation
airflow/models/dagrun.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need this method?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, it seems to be used only in tests. Should I remove it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. Yes. Fstrings are a better idea. It's clearer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this method create this template by itself? It seems to me that this template does not change and it will always be {DagRunType}__{execution_date}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nearly always, backfill has only one underscore
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to standardize it if we want to introduce a new column in the table later. I would be happy if this PR cleaned up all the weirdnesses about DagRunType. In the next PR, we will update the database model and queries.
Cleaning up this mess is crucial for performance.
select * from test_runs where run_type=‘{}’
(0.0284, 0.037, 0.025)
select * from test_runs where run_id like ‘{}%’
(0.0426, 0.0531, 0.0377)
type / like [-33.35%, -28.89%, -32.99%]
(avg, min, max)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mik-laj I agree so I used the double underscore
7c6c434 to
3f5c979
Compare
4d098eb to
f407053
Compare
f4b8837 to
745f40f
Compare
fixup! [AIRFLOW-6954] Use DagRunType instead of ID_PREFIX in run_id
745f40f to
70a0bb7
Compare
Codecov Report
@@ Coverage Diff @@
## master #7583 +/- ##
==========================================
- Coverage 86.98% 86.47% -0.52%
==========================================
Files 904 904
Lines 43728 43722 -6
==========================================
- Hits 38037 37808 -229
- Misses 5691 5914 +223
Continue to review full report at Codecov.
|

Now we have the
DagRunTypethere's no more need to keepID_PREFIXandID_FORMAT_PREFIXused for run_id.Issue link: AIRFLOW-6954
Make sure to mark the boxes below before creating PR: [x]
[AIRFLOW-NNNN]. AIRFLOW-NNNN = JIRA ID** For document-only changes commit message can start with
[AIRFLOW-XXXX].In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.