Skip to content

Fix dependencies of standard provider #43553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
potiuk merged 1 commit into from
Oct 31, 2024
Merged

Fix dependencies of standard provider #43553

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion generated/provider_dependencies.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1286,7 +1286,8 @@
"standard": {
"deps": [
"apache-airflow-providers-common-sql>=1.18.0",
"apache-airflow>=2.10.0"
"apache-airflow>=2.8.0",
"virtualenv>=20.26.0"
],
"devel-deps": [],
"plugins": [],
Expand Down
3 changes: 2 additions & 1 deletion providers/src/airflow/providers/standard/provider.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,9 @@ versions:
- 1.0.0

dependencies:
- apache-airflow>=2.10.0
- apache-airflow>=2.8.0
- apache-airflow-providers-common-sql>=1.18.0
- virtualenv>=20.26.0
Copy link
Contributor

@jscheffl jscheffl Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhm, I am thinking about this as dependency... so in new Airflow Standard provider will be a standard install dependency, this has a standard dependency to virtualenv...
So far ths was always optional, even the examples excluded parts if virtualenv was not available as dependency. Do we really want to make this a pre-requisite?

Personally I think it is not harming but we change the default dependencies here. (Compared to 2.x line where no dependecy was modelled)

Copy link
Member Author

@potiuk potiuk Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Thought about it too. No harm I think. It's been always installed anyway in most cases. And the problem is that if we add it as provider's optional dependency - then almost never will actually install it, while PythonVirtualEnv operator is one that is pretty used.

So I think we should eventually remove the virtualenv extra from airflow and have it always installed.

Another option is - we migh change PythonVirtualenv to use the built-in venv.

Which actually might be a very good idea. WDYT?

Copy link
Contributor

@jscheffl jscheffl Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COmpared to the authors of virtualenv I don't see a real benefir over the included venv. Might be a clever small change.

Otherwise - much cooler would be if UV is supported. We had some discussions in slack... not that pre-commit gets uv support on main-line bewfore we have Airflow 3 out :-D

Copy link
Member Author

@potiuk potiuk Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be for both - venv (only) and uv (optional) support in the new PythonVirtualenv in provider.

Copy link
Member Author

@potiuk potiuk Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First part fixed in #43568


integrations:
- integration-name: Standard
Expand Down