Change constraints to use pipcheck

[potiuk]

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

[potiuk]

@turbaszek - this is a change that uses the deps synced so that the pip check is successful. Probably some tests will fail, but now rather than fixing deps, wee will have to fix the tests. Pls. base all the further changes on this one for #12508

[potiuk]

Once we fix all the tests here, we can fix them for master!

[potiuk]

manually

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[potiuk]

We are trying to have a separate branch and check where we can tests latest constraints upgraded and run tests there. before we share thee constraints with thee master branch.

[potiuk]

And runnning pip check in CI pipeline there.

[kaxil]

We are trying to have a separate branch and check where we can tests latest constraints upgraded and run tests there. before we share thee constraints with thee master branch.

Isn't the constraints-push CI job exactly do that, constraints are eagerly updated and only if they pass the tests are the constraints pushed to constraints-master?

[potiuk]

Isn't the constraints-push CI job exactly do that, constraints are eagerly updated and only if they pass the tests are the constraints pushed to constraints-master?

No. It's more than that. Pip always (for a very long time) had conflicts between different versions of our dependencies. PIP does not always solve them. What we did with constraints we simply captured what pip install does with the current setup.py files. The passing tests, do not mean that there is no conflict between dependencies.

The constraints mechanism did not solve those conflict - this something we are trying to do right now with Tomek. It simply captured the set of constraints that was the result of pip install.

Pip check simply fails if there are version conflicts and shows them.

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[turbaszek]

if they pass the tests

Passing test is not the same as passing pip check. Tests works but we have "broken" dependencies. Once we have this we will be able to run pip check as part of CI. Then we will be sure that a) tests pass b) dependencies are compatible

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[potiuk]

OK @turbaszek -> this PR has now all the constraints updated and pip check should not fail, however we have some failures resulting from the azure storage Blob missing. I think it's a missing implicit dependency and will try to fix it - but in the meantime you can rebase your Pip-check change on top of mine (maybe just add it here)

[github-actions]

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

[kaxil]

Got it, would be a good change with the new PIP resolver, can you add the summary of this to PR descriptions for future readers too, please

[potiuk]

Got it, would be a good change with the new PIP resolver, can you add the summary of this to PR descriptions for future readers too, please

This PR is mostly with trying it out and running tests. we will create separate PR from that later on when we actually figure out all that needs to be done

[potiuk]

I mark it as Draft for now :)

[potiuk]

@turbaszek -> Seems it worked and we can get a working pip-check set of deps :). At least for 3.6. I am going to mark it as full-test-needed and re-push.

[potiuk]

Deleting it and the "constraints-pipcheck" branch as we have a good "regular" solution with "upgrade to latest dependencies" label in #12635