[Bug][CI] Python 3.6 used in TVM CI is reaching EOL

[leandron]

Python 3.6 is widely used in our CI jobs (as it is default for Ubuntu 18.04, that we use), and it is going to be in EOL (end of life) status by 31st December 2021, which means no security fixes to be released after that date. See https://www.python.org/dev/peps/pep-0494/#lifespan.

This issue is a placeholder for us to discuss on moving away from Python 3.6 and decide:

1. Which version to use? Suggestion: Python 3.8 (EOL October/2024), installed from the deadsnakes repository, so that we can move Python soon and in future move Ubuntu version as well. See Python 3.8 release information https://www.python.org/dev/peps/pep-0569/
2. How to get there
3. Expected side-effects and changes, e.g. tlcpack

cc @tqchen @areusch @u99127 @Mousius @jroesch @driazati @mshawcroft for comments

[u99127]

I'd suggest python 3.8 as that seems to be the best bet for reasons as you suggest. Keeping the CI in a version that is reliably going to work for the next few years with security updates is probably sensible. If we have static checkers that allow us to keep to only 3.6 based python language features, that would be good if people want to still deploy with older python versions.

[driazati]

1. 1. What's the advantage of deadsnakes vs the default system repos for python3.8 or a more complete Python packager like miniconda?
   2. I would suggest we stick to the lowest supported version of Python available (meaning we should use Python 3.7 in this case), even if that means we have to upgrade more often. If we go to Python 3.8 it would be pretty easy for someone to accidentally break 3.7 support (e.g. using a := operator) without anyone noticing until a user reports an error
2. What else needs to be done other than update the docker images + fix any issues that come up with new dependency versions pip might pick up?
3. tlcpack in general looks pretty anemic, maybe now would be a good time to increase support for more python/system versions (e.g. conda only has 3.6-3.8, no pip wheels are available for Linux), though I don't have any of the historic context on these things

[mshawcroft]

python3.6 aside, the CI images are all based on ubuntu 18.04 LTS which goes out of maintenance in 3 months time. I don't think the fact that python.org have dropped support on 3.6 is an immediate problem, the ubuntu images in use, which ship 3.6 remain in support short term. The more urgent issue is perhaps that of moving from 18.04 to 20.04 before 18.04 falls out of support. That will naturally bring in python3.8 as the base python version. It is probably also worth mentioning here that tensorflow already dropped 3.6 support, which means at least one frontend to tvm no longer supports 3.6.

On the tlcpack topic, I agree with the post above, from a user perspective at least, tlcpack looks pretty anemic:

• There are still packages on pypi
• There are no packages for the recent 0.8.0 release of tvm.
• There are no packages for most of the python versions in common use.

[tqchen]

Agree about dropping python 3.6 support. Thanks @leandron for bringing it up.

[leandron]

1. What's the advantage of deadsnakes vs the default system repos for python3.8 or a more complete Python packager like miniconda?

I'm not very familiar with *conda distributions. I feel that deadsnakes is very quick into adding support for new versions of Python in a Ubuntu-friendly packaging, and also simple to install and to adapt into existing Docker images we have. That is why I suggested that.

2. I would suggest we stick to the lowest supported version of Python available (meaning we should use Python 3.7 in this case), even if that means we have to upgrade more often. If we go to Python 3.8 it would be pretty easy for someone to accidentally break 3.7 support (e.g. using a := operator) without anyone noticing until a user reports an error

I'm OK with Python 3.7 as minimum (that is the minimum for e.g. TensorFlow from now). My suggestions was mostly trying to avoid going with moving python again in months time.

• What else needs to be done other than update the docker images + fix any issues that come up with new dependency versions pip might pick up?

I think that summarises all that needs doing: adjust the docker images, and in case the python version to be used is not the default version (the one called when invoking python3) in the system, we probably need to have a way to point to what the python binary file is, e.g. python3.7 for example.

3. • tlcpack in general looks pretty anemic, maybe now would be a good time to increase support for more python/system versions (e.g. conda only has 3.6-3.8, no pip wheels are available for Linux), though I don't have any of the historic context on these things

In tlcpack, things are pretty much implement on an as-needed basis. Certainly needs expanding to cover more platforms.

As @mshawcroft mentioned, perhaps this discussion is more about moving Ubuntu versions, rather than Python versions, but I didn't do any investigation in that direction.

[tqchen]

how about we start with py3.7(as that is non-controversial) then have a discussion about requiring 3.8?