Skip to content

Conversation

@Taragolis
Copy link
Contributor

Refers to:

This PR show what we should change before we could merge it.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@Taragolis Taragolis added the full tests needed We need to run full set of tests for this PR to merge label Sep 25, 2023
@Taragolis Taragolis force-pushed the ban-lazy-imports-in-codebase branch from 217a3f0 to d404932 Compare September 25, 2023 11:28
@Taragolis Taragolis changed the title Ban airflow in airflow.models lazy imports in airflow codebase Ban lazy imports from airflow and airflow.models in codebase Sep 25, 2023
@Taragolis Taragolis force-pushed the ban-lazy-imports-in-codebase branch from d404932 to 1aa5631 Compare September 25, 2023 11:32
@uranusjr
Copy link
Member

I think importing from airflow.models is mostly fine actually, since every one of those models eventually need to pull in most of the models anyway due to how SQLAlchemy’s ORM works. But if the check is simple why not, I guess.

@ephraimbuddy
Copy link
Contributor

I think where we have issue is in importing from models when we are inside the models but every other imports in other places should work but of course, it’s better to ban it everywhere in the codebase, so this is good

Copy link
Member

@hussein-awala hussein-awala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1000

@Taragolis
Copy link
Contributor Author

There are 500+ occurrence lazy imports in airflow codebase, without from airflow import __version__

In general every import should be excluded by up to 3 PRs:

  1. Exclude from core + core tests + example dags
  2. Exclude from providers + providers tests
  3. Ban import in [tool.ruff.flake8-tidy-imports.banned-api]

What I have done with airflow.AirflowException locally:

  1. Create in branch and in first commit was banned in[tool.ruff.flake8-tidy-imports.banned-api]
  2. pre-commit run ruff --all-files
  3. Resolve all cases in core, create second commit
  4. pre-commit run ruff --all-files
  5. Resolve all cases in providers, create third commit(s)
  6. Check again that there is no any static checks failed
  7. Create 3 branches from main and cherry pick changes
  8. Create 3 PRs

Copy link
Contributor

@vincbeck vincbeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Nov 10, 2023
@github-actions github-actions bot closed this Nov 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full tests needed We need to run full set of tests for this PR to merge stale Stale PRs per the .github/workflows/stale.yml policy file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants