FIX: extends typing-extensions to be installed with python 3.8+ #21566

[frankcash]

Uses built in util to handle import of literal until python 3.8 is phased out.

Changes setup.cfg to install typing-extensions will all versions of python, not just <3.8

closes: #21566

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

[uranusjr]

Why not just add typing_extension as a dependency?

[github-actions]

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

[frankcash]

Looks like the Github Actions are "failed" because one was cancelled? Error: The operation was canceled.

[potiuk]

Closed/reopened to check if this was a flaky fail.

[potiuk]

It could be that the typing extension actually causes some too long running constraint generation.

[frankcash]

It could be that the typing extension actually causes some too long running constraint generation.

looks like it timed out again. Let me know how I can help resolve this / if I need to change the code

[frankcash]

@potiuk / @uranusjr I've re-run this a few times but it continues to time out at the step

I am confused on what I need to change / do to get this to resolve -- would someone point me in the right direction?

[potiuk]

I am confused on what I need to change / do to get this to resolve -- would someone point me in the right direction?

Yeah. I was afraid it is not "accident". I will have to take a look at that more closely, but I think the problem is that it creates unsolvable conflict with one of the relased providers (we prepare few variants of the constraints and apparently this one gets triggered.

What happens under the hood pip gets into backtracking and tries to find a solution to impossible conflict resolution and it takes awfully lot of time to resolve it. I am afraid this one will need to wait until we take a bit closer look what's going on in detail. I should find time for it this week.

[frankcash]

Sounds good, thank you so much for your time

[frankcash]

@potiuk / @uranusjr lgtm if one of ya'll want to merge it

[frankcash]

Keeps timing out at Build Images / Build CI image 3.7 (pull_request_target) In progress — This check has started... 🙃

[frankcash]

it looks like a lot of other builds are failing at the build images / build ci image {python_version} step..

[potiuk]

it looks like a lot of other builds are failing at the build images / build ci image {python_version} step..

Not all - only those that change "setup.py". This is pip getting into backtracking loop.

[potiuk]

@frankcash -> I would really love if you could upvote and comment here: pypa/pip#10258 (comment).

The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that pip is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help pip.

I proposed to add optional timeout on pip install execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes pip fail installing airflow from the scratch and we need to help pip to determine the right set of dependencies because it is unable to figure it out on its own).

[frankcash]

Thanks for linking that @potiuk , I will provide a comment on it.

[potiuk]

@frankcash I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.

[frankcash]

🤞

[frankcash]

@potiuk it worked yay