Upgrade Helm chart dependency to bitnami/postgresql 13.x.x and PostgreSQL 16

[dnskr]

Official Helm Chart version

1.11.0 (latest released)

Apache Airflow version

2.7.1

Kubernetes Version

1.27

Helm Chart configuration

No response

Docker Image customizations

No response

What happened

Airflow Helm chart 1.11.0 depends on bitnami/postgresql:12.10.0 subchart and deploys PostgreSQL 11 by default.

Apart from good practice of keeping dependencies up to date, there is an end-of-life for PostgreSQL 11 coming in November 9, 2023 as described in PostgreSQL Versioning Policy.

What you think should happen instead

Next Airflow Helm chart release should use latest bitnami/postgresql:13.x.x and PostgreSQL 16 by default.

How to reproduce

Deploy Airflow Helm chart 1.11.0 with default configuration.

Anything else

This issue is created to highlight the problem and discuss how it should be fixed with minimal impact on end users.

PostgreSQL was intentionally downgraded from 15 to 11 after bitnami/postgresql upgrade from 10.5.3 to 12.1.9.
Related PRs:

• Upgrade bitnami/postgresql helm depenency from 10.5.3 to 12.1.9 #29071
• Fix PgBouncer after postgres subchart upgrade #29207

The PR #29207 also contains the discussion of future version upgrade.

Also related PRs with minor changes and fixes:

• Fix bitnami/postgresql subchart upgrade #29210
• Fix bitnami/postgresql default username and password #29478
• Chart: Update postgres subchart #33747

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[Taragolis]

and PostgreSQL 16 by default.

This would unlikely happen, Airflow CI do not have regular runs on Postgres 16 yet (#34543)

After EOL Postgres 11 I guess default would be Postgres 12, at least this happen for other components: default python is 3.8, default db backend the oldest one.

[dnskr]

Airflow Helm chart has been released recently, so taking it into considiration, there is a good chance that Airflow can support Postgres 16 officially before next Helm chart release.
I'm fine with upgrading PostgreSQL to the newest possible version, either it's 15, 14, 13 or even 12. However, I'm not sure that upgrading to PostgreSQL 12 is a pragmatic solution, because end-of-life for PostgreSQL 12 coming next year (November 14, 2024).

Main challenge with the upgrade is that there is no auto-upgrade options for major versions of PostgreSQL as far as I know, i.e. major version upgrade flow requires users to manage it manually with some database downtime. See https://www.postgresql.org/docs/current/pgupgrade.html for more details.

[eladkal]

We added Postgres 16 support to main branch #34543

[dnskr]

Looks like there is no automated solution for the problem at the moment.
Related issue bitnami/charts#14926

[potiuk]

We can continue upgrading it manually as we do now. And it's fine to upgrade to postgres 16 immediately. Postgres in our chart is not intended for production use and you SHOULD use external postgres. So we do not have to maintain any compatibility or upgrade path.