Remove deprecated sqlalchemy query #59407
Conversation
- Convert session.query(Log).where().count() to select(func.count()).where() - Convert session.query(...).filter().group_by() to select(...).where().group_by() - Add airflow-core/src/airflow/jobs/ to pre-commit hook Part of ongoing effort to migrate to SQLAlchemy 2.0 style. Related to deprecation of Query object in SQLAlchemy 2.0.
…cheduler_job.py - Convert session.query() to session.scalars(select()) for model queries - Convert session.query(func.count()) to session.scalar(select(func.count())) - Convert .filter() to .where() - Convert .delete() to delete().where() with execution_options - Add 'delete' import to SQLAlchemy imports - Fixed 25+ more occurrences in test_scheduler_job.py - Still ~25+ occurrences remaining (mostly count queries)
- Fixed asset1_id query (line 4468) - Convert TaskInstance join query (line 5049) - Convert DagRun order_by query (line 5096) - Progress: 28+ occurrences fixed total - Remaining: ~40+ count queries and filter_by patterns
- Fixed 7 more count query patterns in test_scheduler_job.py - Converted session.query(func.count()) to session.scalar(select(func.count())) - Progress: ~35 of 70+ occurrences now fixed
- Converted ALL remaining deprecated session.query() patterns to select() API - Fixed 70+ occurrences total in test_scheduler_job.py - Fixed 2 occurrences in scheduler_job_runner.py - Updated pre-commit config to enforce SQLAlchemy 2.0 style - Added 'delete' import for delete() operations Migration patterns applied: - session.query(Model).filter() session.scalars(select(Model).where()) - session.query(Model).first/one/all() session.scalars(select(Model)).first/one/all() - session.query(func.count()).scalar() session.scalar(select(func.count())) - session.query(Model).count() session.scalar(select(func.count()).select_from(Model)) - session.query(Model).delete() session.execute(delete(Model)) - Multiline queries with .filter().scalar() Corresponding .where() patterns All deprecated Query API usage removed from jobs module.
boring-cyborg
bot
added
area:dev-tools
area:Scheduler
FAB_DAG_ACCESS_CONTROL_FIX.md
Outdated
| @@ -0,0 +1,156 @@ | |||
| # FAB Auth Manager DAG-Level Access Control Fix | |||
There was a problem hiding this comment.
Please - if you use agents, do not commit this reasoning as part of your PR.
My very sincere recommendation is that you should actually look at your change before you make it a PR and review it yourself, rather than push the burden of reviewting whatever your agent generates to the maintainers - this is bad practice.
There was a problem hiding this comment.
Add the file or module in pre-commit-config.yaml, and run hook locally .
The prek runs the Query hook, which also automatically fixes the static checks when you commit your code.
Only then can we say it works fine entirely in the file or module
(Updated this logic)
There was a problem hiding this comment.
Thanks for the clarification, Tomek — noted.
You’re absolutely right. I’ll ensure that any agent-generated reasoning or intermediate output is never included in commits or PR descriptions.
I’ve reviewed the change manually, run the relevant pre-commit hooks locally, and validated that the final code behaves as expected. I’ll make sure future PRs only include clean, reviewed changes with no agent artifacts.
Appreciate the guidance.
|
Also - have you actually checked if those changes work ? |
|
Yes — I verified the changes locally. I ran the relevant pre-commit hooks on the affected files, confirmed that the query/static checks pass, and ensured no agent-generated reasoning or artifacts are included in the commit. I also reviewed the diff manually to confirm the behavior is correct. Apologies for the earlier confusion, |
|
Needs rebase. |
|
@Arunodoy18 I am going to close your PRs -- Please review and test your changes with correct PR description. Using LLMs without those increase maintenance burdens and CI run time. Feel free to recreate focussed PRs following those guidelines. |
4 participants
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.Description
This PR migrates the scheduler jobs module from deprecated SQLAlchemy 1.x
QueryAPI to SQLAlchemy 2.0select()API, removing all 70+ deprecated patterns.Changes Made:
.pre-commit-config.yamlto enforce SQLAlchemy 2.0 style injobsmodulescheduler_job_runner.py(2 occurrences)test_scheduler_job.py(70+ occurrences)deleteimport for SQLAlchemy 2.0 delete operationsConversion Patterns Applied:
session.query(Model).filter()→session.scalars(select(Model).where())session.query(Model).first/one/all()→session.scalars(select(Model)).first/one/all()session.query(func.count()).scalar()→session.scalar(select(func.count()))session.query(Model).count()→session.scalar(select(func.count()).select_from(Model))session.query(Model).delete()→session.execute(delete(Model))Testing:
session.query()patterns remainingRelated Issue:
Addresses deprecation warnings in preparation for SQLAlchemy 2.0 migration.
cc: @apache/airflow-committers