Breaking change introduced in the legacy Poetry installer

[adzeeman]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name:
• Poetry version:
• Link of a Gist with the contents of your pyproject.toml file:

Issue

Hi Poetry team. With Poetry 1.2.0, the legacy installer no longer works the same way it had.

We use Poetry in production as part of our CI/CD process, and the breaking change resulted in multiple pipelines failing.

On 31 August, the legacy installer stopped working. The fix was to append the --version 1.1.15 to the installer. In our own time, we will migrate over to 1.2.0 but will need to evaluate that release before doing so.

On 1 September, the installer stopped working again, even with the fix that worked the day before. GET_POETRY_IGNORE_DEPRECATION=1 is now required. We had to update all our pipelines again.

The bug is created as a request that breaking changes are not introduced in the future when attempting to phase out legacy procedures. Instead, please use alternative measures, such as deprecation warnings.

I created this bug since we plan to continue using Poetry but wanted to raise this concern with the development team.

[neersighted]

Hi @adzeeman, sorry about the headache. To give you a little background, get-poetry.py was deprecated with a warning message for over a year (and the replacement was available for 6 months before the deprecation) -- however usage of the script in production with no pinned version was much more common than we thought.

Thus, when 1.2.0 was released, the old installer would break as it would select the latest stable version for install, and then realize that 1.2.0 wasn't installable. As a compromise it was suggested that the installer fall back to 1.1.x with loud warnings in that situation -- we did so, but the with addition of a random failure percentage to try and force investigation and migration, since clearly warning messages, blog posts, and release notes had not made those downloading the script from our repo sufficiently aware.

I am open to the idea of considering an explicit --version to be an opt-out of the failure chance, but I hesitate because the eventual (total) removal of this script in several months will likely cause similar heartache. If we get it out of the way now, that may be for the best, even if it causes more pain and confusion now.

This was not meant to happen at the same time as 1.2.0 -- honestly the plan was to get rid of get-poetry.py before the 1.2.0 release, but we simply didn't have the bandwidth and the issue forced itself with the new stable release.

Since the cat is already out of the bag, do you think allowing for --version to silently continue is really any better since we still need to remove this installer? Note that the new installer has long been able to install Poetry 1.1 -- the release branches and installers have no intrinsic ties, beyond 1.2 lacking the build machinery to produce get-poetry.py-compatible archives.

Finally, sorry again about the pain here -- deprecating the get-poetry.py script was thought to have been a solved problem a year ago, though it is clear that the message the script printed for the last 12 months could have been more clear on the nature of the upcoming breakage. Future deprecations of similar tools (e.g. install-poetry.py in this repo) will be informed by this experience.

Would it be possible to publish a new version 1.3 which is same as latest version of 1.1 and port the recent changes of 1.2 changes to v2?

[neersighted]

Would it be possible to publish a new version 1.3 which is same as latest version of 1.1 and port the recent changes of 1.2 changes to v2?

What? If I understand correctly, you want to re-release 1.1 as 1.3? The 1.1 branch is no longer maintained and has been in a state of benign neglect for some time, mostly due to how far 1.2 drifted and how much technical debt had accumulated there. If you need 1.1, just pin to 1.1 -- the new installer and supported install methods (pipx, manual) support 1.1 just fine, if you still need it.

If you need 1.1, just pin to 1.1 -- the new installer and supported install methods (pipx, manual) support 1.1 just fine, if you still need it.

did exactly that.

Since 1.2 stable is released yesterday. it is just a suggestion so that it won't break for wide range of users.

[neersighted]

I'm still unclear as to what you're suggesting then -- never release new versions? We're getting off-topic for this thread -- I would suggest migrating to Discord or a discussion. This issue should remain strictly for discussing get-poetry.py, which is not directly related to the release of 1.2.0 stable.

[steve-mavens]

I think the suggestion was that incompatibility with get-poetry.py constitutes a breaking change in 1.2, and therefore it should have had a major version bump to 2.0. But you're going to say that incompatibility with get-poetry is not a breaking change since get-poetry has been deprecated for over a year! And even so it's not clear to me whether renumbering 1.2 as 2.0 would help at all with get-poetry breaking, since can't it still select "2.0" as the latest version?

Our team only got around to making a general switch from get-poetry to install.python-poetry.org last week, and we still have older code that cannot be installed by poetry 1.1.* (because it doesn't support local dependencies so well), and therefore stuck on poetry 1.0.10, and therefore stuck on get-poetry.py. I think the python ecosystem in general, doesn't support neglect: if you don't actively maintain all your code then it will tend to break for some reason or other, even with pinned dependencies. And once you start making those changes, you need your build to still work, which means more maintenance. poetry is one part of this, but by no means unusual in the context of Python tooling. We don't like it, but we don't have much choice but to do it.

Presumably when get-poetry disappears, to get poetry 1.0.10 we'd have to either fork the installer, use an older version from some other URL, or else pip install poetry?

[jonapich]

I still believe that my original solution was the proper fix for the "planet":

#6287

I have also spent some hours yesterday morning pinning some missing --version 1.1.15 to the legacy script in order to extinguish the fires. This morning, it started failing again even though I did pin the version (that 5% thing). So the "fix" that was introduced basically made our workaround useless in favor of a new workaround.

It would be better if the script would install the last compatible version (1.1.15) and issue a warning that to obtain 1.2+ you need to use the new installer, and even crash if you pass in --version 1.2.0 saying it's just not the way to do it. That would be the best thing to do, because now, the worst that will happen is that people are going to bypass your deprecation with an env var, which has a good potential to create additional support cases about plugins not working. The worst that should happen, is that you successfully install and use an old version you're used to 🤷🏻‍♂️

Train of thoughts:

• Deprecations and forced migrations are sometimes necessary, but it's not in this case since the new installer is a new file with a shiny new url. There's nothing to do to stay "backward compatible", except preventing 1.2+ from being installed.
• The legacy installer could live at its documented URL forever and never be removed from the repository. You can probably even protect the file against changes. This is what we would expect from a "curl installable" script. There's no rush to have everyone move to 1.2, since the 1.1.15 version works amazingly well and will most likely continue to do so for years. Feeling the urge to delete this file feels a bit OCD considering the harm it would cause.
• The 1.2 announcement mentioned "years of work" to come up to this. This sort of wording makes me think that this should have been released as 2.0. A major bump like that absolutely opens the door for people to have to migrate to the new installer.
• User experience is always the driving factor / priority. Some people have very complex build setups that takes hours, and also logging isn't always easy to retrieve. It's not very courteous to force a migration when the old version is still working just fine.

[finswimmer]

Hello @jonapich,

I have also spent some hours yesterday morning pinning some missing --version 1.1.15 to the legacy script in order to extinguish the fires.

may I asked you, why you didn't take the chance to replace the old installer by the new one? You can use the new installer to install 1.1.15 as well.

It would be better if the script would install the last compatible version (1.1.15) and issue a warning that to obtain 1.2+ you need to use the new installer, and even crash if you pass in --version 1.2.0 saying it's just not the way to do it.

As @neersighted wrote we are open for this idea:

I am open to the idea of considering an explicit --version to be an opt-out of the failure chance, but I hesitate because the eventual (total) removal of this script in several months will likely cause similar heartache. If we get it out of the way now, that may be for the best, even if it causes more pain and confusion now.

This may save you some time now, but if we drop the older installer script from the repo in some month, there will be other people, telling us we should have informed them earlier.

The legacy installer could live at its documented URL forever and never be removed from the repository.

We don't want to blow up our code base for legacy reasons over the time. So it's necessary to make a clean up from time to time. You can always use a permalink to the file in a specific commit, e.g. https://github.com/python-poetry/poetry/blob/a613347bf0a9fd7b7ec782cd45222b3a0f0e7401/get-poetry.py , or even put this file in a local repo if you heavily rely on it.

There's no rush to have everyone move to 1.2, since the 1.1.15 version works amazingly well and will most likely continue to do so for years.

As said above, you can use the new installer to install 1.1.15.

The 1.2 announcement mentioned "years of work" to come up to this. This sort of wording makes me think that this should have been released as 2.0.

The version number based on SemVer doesn't tell you anything about the amount of code changes, it is telling you something about the changes in the API. At Poetry we consider CLI as the API and nothing more. The installer script is a stand alone project.

It's not very courteous to force a migration when the old version is still working just fine.

Please let me repeat it once more: You are not forced to a new version of Poetry. You are forced to a new version of the installer only.

fin swimmer

[remram44]

This is a bit useless after the fact, but I would like to point out that the way you advertised the deprecation of the old installer was not ideal. This is the warning that would be shown up the the day you made it fail on purpose:

This installer is deprecated, and cannot install Poetry 1.2.0a1 or newer.

What I read there is that "this installer is deprecated, what that means is it can't install Poetry 1.2.0". What you meant is that "this installer is deprecated and going away any minute, also incidentally it cannot install Poetry 1.2.0".

Probably a lot of people, like me, were planning to upgrade to the new installer at the same time they upgraded to 1.2.0. All those people now have their pipelines broken because there was not a single day between the release of 1.2.0 and the day the "deprecated" installer was disabled (even if using it to install an older version with the --version flag).

Your "brownout" strategy is not tripping up stragglers who haven't updated the installer despite many warnings. I am a very excited Poetry user who was going to update as fast as possible following 1.2.0's release and was still trip up by this. Please consider revising your deprecation strategy (or having a public strategy...) for future breaking changes, it would be greatly appreciated.

[ssnover]

may I asked you, why you didn't take the chance to replace the old installer by the new one? You can use the new installer to install 1.1.15 as well.

I can't speak for others in this thread, but the looming problem on my team is that the install script is used in a docker container build pipeline. We maintain branches for old release versions so that we can build them if we need a patch. All of those branches contain a Docker build which will be invalidated by the removal of the installer. So this move doesn't just force updating our current pipeline, it requires us to update the installer for every single release branch we maintain.

[dimbleby]

I offer no view on whether the poetry project could have handled this better, but I want to encourage those of you who are blaming poetry for your CI problems to take at least a glance in the mirror.

Many of us have learned this the hard way so I have some sympathy, but the fact is: if you write your pipelines in such a way that strangers on the internet can break them - then sooner or later that will happen.

If you were relying on get-poetry.py never changing - you should have downloaded it and used your own copy. Count youselves lucky that no-one went rogue and uploaded a version that steals all your secrets and deletes all your code.

(But you shouldn't use a local copy of the script either: because you're getting all your dependencies from a repository that you own and not from public PyPI, right? So the script is useless to you anyway.)

Here endeth the lesson, please resume the complaining...

[remram44]

There is a world between "you can't expect to be up forever" and "you can't criticize the people taking it down". This doesn't even make sense.

I am not saying that my company or my system is dying because of this. But I don't think that nothing deserves criticism until something or someone actually dies either.

Let's find a way to lighten the impact and make sure future changes are handled better, please.