Skip to content

Conversation

@potiuk
Copy link
Member

@potiuk potiuk commented Jun 27, 2022

The setup.py had ALWAUS contained provider dependencies,
but this is really a remnant of Airlfow 1.10 where providers
were not separated out to subfolders of "providers".

This change moves all the provider-specific dependencies
to provider.yaml where they are kept together with all other
provider meta-data.

Later, when we move providers out, we can move them to
provider specific setup.py files (or let provider-specific
setup.py files read them from provider.yaml) but this is
not something we want to do it now.

The dependencies.json is now renamed to provider_dependencies.json
and moved to "airflow" so tha it can be kept as part of the
sources needed for sdist package to provide extras. Pre-commit still
generates the file as needed and it contains now both:

  • cross-provider-deps information which providers depend on each
    other
  • deps - information what regular dependencies are needed for each
    provider

On top of preparing to splitting providers it has the advantage,
that there will be no more case where adding a dependency change
for provider will not run tests for that provider.


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragement file, named {pr_number}.significant.rst, in newsfragments.

The ``setup.py`` had ALWAUS contained provider dependencies,
but this is really a remnant of Airlfow 1.10 where providers
were not separated out to subfolders of "providers".

This change moves all the provider-specific dependencies
to provider.yaml where they are kept together with all other
provider meta-data.

Later, when we move providers out, we can move them to
provider specific setup.py files (or let provider-specific
setup.py files read them from provider.yaml) but this is
not something we want to do it now.

The dependencies.json is now renamed to provider_dependencies.json
and moved to "airflow" so tha it can be kept as part of the
sources needed for sdist package to provide extras. Pre-commit still
generates the file as needed and it contains now both:

* cross-provider-deps information which providers depend on each
  other
* deps - information what regular dependencies are needed for each
  provider

On top of preparing to splitting providers it has the advantage,
that there will be no more case where adding a dependency change
for provider will not run tests for that provider.
@potiuk potiuk force-pushed the move-setup-dependencies-to-providers branch from 99ddc72 to 83d1b99 Compare June 27, 2022 00:57
@potiuk
Copy link
Member Author

potiuk commented Jun 27, 2022

Closing this one i favour of #24672 - this one is run from "apache" repo, so we will be able to see production build succeeding (from fork it will fail until it is merged).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant