Debian 10 Buster using Python 3.7

[nzlosh]

Despite Ubuntu's popularity, Debian still remains present in production environments. It would be useful to support this platform to expand the potential StackStorm user base by having native packaging and installation process for this distribution.

The current stable version is Debian 10 Buster which provides Python3.7 only, no Python 3.6 packages are provided by Debian. Since Debian and Ubuntu are very similar I was able to build docker images and build system packages for StackStorm. See the links below for details.

Build package docker images:
StackStorm/st2packaging-dockerfiles#75

Build system packages:
StackStorm/st2-packages#628

For the current experience of running StackStorm 3.2dev on Debian Buster, a large portion of the code base functions out of the box. There is an issue with rule processing which prevents StackStorm from being using in production on Buster.

This feature request is to officially integrate Debian 10 Buster support into the packaging process and test against Python 2.7, 3.6 and 3.7 as part of the commit process to StackStorm.

[punkrokk]

@nzlosh Hey Carlos - what do you think would be the added challenges with support the bleeding edge - aka Fedora and Debian? I realize Ubuntu is generally faster shipping upstream improvements than CentOS, but while I am personally all for this effort, what new responsibilities would surface with keeping this working?

Things like:

• Frequency of Debian updates?
• Frequency of intra-release breaking changes?
• What is the level of effort to support python 3.7?
• Could python 3.6 be installed directly as a way to get the same outcome? It's not out of the question to even ship a st2-python36 deb package.

I HTH moving the conversation forward.

[nzlosh]

Thanks for your comments.

• Debian moves Testing to Stable every ~2years.
• Stable is called stable because the promise is there are no breaking changes. Obviously there will be mistakes along the way, but they wouldn't be intentionally introduced by Debian maintainers.
• I'm not qualified to determine the effort required to support Python3.7, that's something the StackStorm devs would have a better idea of.
• Python 3.6 does not exist as native packages and I wasn't able to find any PPA (community packages) It would not be a viable production solution to attempt to backport Python 3.6 in my opinion.

[punkrokk]

@nzlosh Can you expand why you think installing python 3.6 from source is a bad idea? I wouldn't remove python 3.7 for the OS. It would just make python 3.6 available for ST2 (which I think could be resolved by the st2rpm/deb build.)

[nzlosh]

I'm talking about the maintenance required to package it from source and test it and host the package so production machine reinstallations would be reliably reproducible. In my opinion the effort to package Python 3.6 just for Debian is wasted time and effort better spent on testing and stablising StackStorm on Python 3.7. Ultimately I think packaging Python 3.6 is out of scope for StackStorm.

[nzlosh]

To have Debian 10 support, Mongo 4.2 will need to be used because this is the lowest version packaged by Mongo for Debian 10. There are tar.gz files for Mongo 4.0, but this would either require the end user or StackStorm to handle creating a package and maintain it for any security or bug fixes.

[funilrys]

AFAIK, nowadays Debian 11 is the stable version.

AFAICS, Stacktorm uses the upstream repos for Mango... Is there any other valid reason for the "exclusion" of Debian 11 nowdays - besides the Python version?

Btw, Debian 11 provides python 3.9.

Am I missing something? After looking at the ansible repository, I see that Stackstorm uses upstream repositories - especially when a particular version is needed. So there shouldn't be a problem to provide Debian packages - as long as the upstream repos of the dependencies provide Debian packages.

AFAICS, all upstream repos that are being used provide Debian packages:

• Mango
• NGINX
• NodeJS
• RabbitMQ

Please let me know if I'm wrong but assuming that packages within the stackstorm repo are automatically built, this shouldn't be a hassle to implement.

It seems, that right now, with Debian 11 it's the other way around. Debian provides Python 3.9 and the CI/CD uses Python 3.6 up to 3.8.

Can we reconsider this and build/provide Debian 11 packages?

Thanks in advance for your time.

cc: @nzlosh @cognifloyd

[nzlosh]

Python3.9 isn't tested and therefore not supported.

There's work underway to improve the build process by migrating the existing one to Pants (it's a very large task). Once that work is complete, we will evaluate the complexity of supporting a broader range of Python versions.

At the moment, we can only guarantee support for Python 3.6 and 3.8. You're free to try building StackStorm from source https://docs.stackstorm.com/development/sources.html and make any adjusts to the instructions to fit with Debian 11 if you must run it on Debian.

[funilrys]

@nzlosh Can't we just provide the Debian package in the unstable branch/repos ?
That may opens the door for those willing to test and give feedback.

I have nothing against building from source, but I have to ask ... 😄

[nzlosh]

The package build process requires unit tests in the StackStorm core and system tests in the st2-package repository to pass in order to publish Debian packages to any environment. I know this isn't what you'd like to hear, but the path of least resistance to at least attempting to get StackStorm running on Debian 11 will be to install from source.

[nzlosh]

Closing this issue. If you like, you can create a new issue requesting Debian 11 with Python 3.9 be officially supported.