Discussion: Testing against multiple versions of PostgreSQL

[austindrenski]

Moved from #709 (comment)

@austindrenski wrote:

Just a note, but should we at some point start testing against multiple release versions of PostgreSQL?

Might be something to consider in a few weeks, after 2.2 and the move to Azure Pipelines.

@roji wrote:

A long, long time ago it actually worked this way. The problems were a much longer build time (multiplied by the number of versions supported), and significantly more complexity in maintenance of the CI platform (this may be gone with Azure, or not).

As a general rule, I'm not sure we've seen many version-related issues that weren't reported and fixed rather quickly. So while testing all versions is theoretically better, I'm simply not sure the added value is worth the downsides... Am definitely open to discussing this though - maybe open a new issue for discussing?

[roji]

Just happened to see this, does anyone think there's actual value in this? Should we maybe close and revisit if we start seeing version-related issues?

[austindrenski]

Yeah, let's close for now. I still like the idea, but as noted in the past, many of the benefits would be theoretical (e.g. for peace of mind), whereas many of the complexity costs are real (e.g. yesterday's PostGIS issue multiplied by the number of versions we'd be testing).

[roji]

Yeah, that's exactly the kind of headache I'm very happy to avoid...