Skip to content

Conversation

@potiuk
Copy link
Member

@potiuk potiuk commented Dec 5, 2023

Provider's compatibility checks have been run for one Python version
and one Airflow version. However provider compatibility is a bit more
nuanced:

  • some providers are 2.5.0+, some are 2.6.0+, some are 2.7.0+
  • some proivders do not work on Python 3.8 (cohere for importlib
    incompatibility reasons) and we need to run the checks on Python 3.9

This PR adds the following capabilities:

  • adds option to install airflow from github URL (finally not used
    in this PR but good to leave it in)
  • the non-default Python checks are only run in case we already build CI
    imagese for those Python versions are already built
  • builds Airflow vith dev0 version to account for pip comparision
    not allowing for selective -pre flags

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk potiuk force-pushed the test-installing-provider-matrix branch 7 times, most recently from 5ad0516 to fe96cee Compare December 5, 2023 10:31
@potiuk potiuk changed the title Installing providers using matrix Run matrix of compatibility checks for providers Dec 5, 2023
@potiuk potiuk requested review from eladkal and vincbeck December 5, 2023 10:32
@potiuk potiuk marked this pull request as ready for review December 5, 2023 10:33
@potiuk potiuk force-pushed the test-installing-provider-matrix branch from fe96cee to 4e3b56d Compare December 5, 2023 19:27
@potiuk
Copy link
Member Author

potiuk commented Dec 5, 2023

cc: @eladkal - I thinkk that one has a chance to go green and once merged, you should be able to bump min version easily (I also included more detailed description on how bumping should look like as it will require to remove the old compatibility check from 2.5.0). It turns out that 2.7.0 had a bug fixed in 2.7.1 (apparently) about thread._Rlock pickling during importing (so I have to base all checks for 2.7 on 2.7.1) -but that's fine.

Provider's compatibility checks have been run for one Python version
and one Airflow version. However provider compatibility is a bit more
nuanced:

* some providers are 2.5.0+, some are 2.6.0+, some are 2.7.0+
* some proivders do not work on Python 3.8 (cohere for importlib
  incompatibility reasons) and we need to run the checks on Python 3.9

This PR adds the following capabilities:

* adds option to install airflow from github URL (finally not used
  in this PR but good to leave it in)
* the non-default Python checks are only run in case we already build CI
  imagese for those Python versions are already built
* builds Airflow vith dev0 version to account for `pip` comparision
  not allowing for selective `-pre` flags
@potiuk potiuk force-pushed the test-installing-provider-matrix branch from 4e3b56d to d8bd041 Compare December 5, 2023 19:58
@potiuk potiuk merged commit ace97c0 into apache:main Dec 5, 2023
@potiuk potiuk deleted the test-installing-provider-matrix branch December 5, 2023 21:36
potiuk added a commit that referenced this pull request Dec 15, 2023
Provider's compatibility checks have been run for one Python version
and one Airflow version. However provider compatibility is a bit more
nuanced:

* some providers are 2.5.0+, some are 2.6.0+, some are 2.7.0+
* some proivders do not work on Python 3.8 (cohere for importlib
  incompatibility reasons) and we need to run the checks on Python 3.9

This PR adds the following capabilities:

* adds option to install airflow from github URL (finally not used
  in this PR but good to leave it in)
* the non-default Python checks are only run in case we already build CI
  imagese for those Python versions are already built
* builds Airflow vith dev0 version to account for `pip` comparision
  not allowing for selective `-pre` flags

(cherry picked from commit ace97c0)
@ephraimbuddy ephraimbuddy added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants