Skip to content

Conversation

@evanasse
Copy link
Contributor

@evanasse evanasse commented Feb 27, 2023

This PR brings the pgbouncer and pgbouncer-exporter images to more recent versions.

pgbouncer: 1.16.1
pgbouncer: 0.14.0

It's not the most recent pgbouncer version, though.
pgbouncer is limited to 1.16.1 since pgbouncer-exporter does not support 1.18.0.

I was not able to build from a more recent version of alpine than 3.14.
It returns the following error when building with alpine >= 3.15:

docker build TLS error output
In file included from lib/usual/tls/tls.c:18:
lib/usual/tls/tls_compat.h:33: warning: "NID_kx_ecdhe" redefined
   33 | #define NID_kx_ecdhe (-90)
      |
In file included from /usr/include/openssl/objects.h:65,
                 from /usr/include/openssl/evp.h:86,
                 from /usr/include/openssl/hmac.h:67,
                 from /usr/include/openssl/ssl.h:150,
                 from lib/usual/tls/tls_compat.h:14:
/usr/include/openssl/obj_mac.h:4465: note: this is the location of the previous definition
 4465 | #define NID_kx_ecdhe            960
      |
lib/usual/tls/tls_compat.h:34: warning: "NID_kx_dhe" redefined
   34 | #define NID_kx_dhe (-91)
      |
/usr/include/openssl/obj_mac.h:4469: note: this is the location of the previous definition
 4469 | #define NID_kx_dhe              961
      |
lib/usual/tls/tls_compat.h:47:28: error: static declaration of 'X509_OBJECT_new' follows non-static declaration
   47 | static inline X509_OBJECT *X509_OBJECT_new(void)
      |                            ^~~~~~~~~~~~~~~
In file included from /usr/include/openssl/x509.h:389,
                 from /usr/include/openssl/pem.h:71,
                 from /usr/include/openssl/ssl.h:151:
/usr/include/openssl/x509_vfy.h:273:14: note: previous declaration of 'X509_OBJECT_new' with type 'X509_OBJECT *(void)' {aka 'struct x509_object_st *(void)'}
  273 | X509_OBJECT *X509_OBJECT_new(void);
      |              ^~~~~~~~~~~~~~~
In file included from /usr/include/openssl/bio.h:69,
                 from /usr/include/openssl/evp.h:67:
lib/usual/tls/tls_compat.h: In function 'X509_OBJECT_new':
lib/usual/tls/tls_compat.h:49:49: error: invalid application of 'sizeof' to incomplete type 'X509_OBJECT' {aka 'struct x509_object_st'}
   49 |         X509_OBJECT *obj = OPENSSL_malloc(sizeof(*obj));
      |                                                 ^
lib/usual/tls/tls_compat.h:51:38: error: invalid application of 'sizeof' to incomplete type 'X509_OBJECT' {aka 'struct x509_object_st'}
   51 |                 memset(obj, 0, sizeof(*obj));
      |                                      ^
lib/usual/tls/tls_compat.h: At top level:
lib/usual/tls/tls_compat.h:58:20: error: static declaration of 'X509_OBJECT_free' follows non-static declaration
   58 | static inline void X509_OBJECT_free(X509_OBJECT *obj)
      |                    ^~~~~~~~~~~~~~~~
/usr/include/openssl/x509_vfy.h:274:6: note: previous declaration of 'X509_OBJECT_free' with type 'void(X509_OBJECT *)' {aka 'void(struct x509_object_st *)'}
  274 | void X509_OBJECT_free(X509_OBJECT *a);
      |      ^~~~~~~~~~~~~~~~
lib/usual/tls/tls_compat.h: In function 'X509_OBJECT_free':
lib/usual/tls/tls_compat.h:61:24: error: invalid use of incomplete typedef 'X509_OBJECT' {aka 'struct x509_object_st'}
   61 |                 if (obj->type == X509_LU_X509) {
      |                        ^~
lib/usual/tls/tls_compat.h:62:38: error: invalid use of incomplete typedef 'X509_OBJECT' {aka 'struct x509_object_st'}
   62 |                         X509_free(obj->data.x509);
      |                                      ^~
lib/usual/tls/tls_compat.h:63:31: error: invalid use of incomplete typedef 'X509_OBJECT' {aka 'struct x509_object_st'}
   63 |                 } else if (obj->type == X509_LU_CRL) {
      |                               ^~
lib/usual/tls/tls_compat.h:64:42: error: invalid use of incomplete typedef 'X509_OBJECT' {aka 'struct x509_object_st'}
   64 |                         X509_CRL_free(obj->data.crl);
      |                                          ^~
lib/usual/tls/tls_compat.h: At top level:
lib/usual/tls/tls_compat.h:70:28: error: conflicting types for 'X509_STORE_CTX_get_obj_by_subject'; have 'X509_OBJECT *(X509_STORE_CTX *, int,  X509_NAME *)' {aka 'struct x509_object_st *(struct x509_store_ctx_st *, int,  struct X509_name_st *)'}
   70 | static inline X509_OBJECT *X509_STORE_CTX_get_obj_by_subject(X509_STORE_CTX *ctx, int lookup, X509_NAME *name)
      |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/openssl/x509_vfy.h:341:14: note: previous declaration of 'X509_STORE_CTX_get_obj_by_subject' with type 'X509_OBJECT *(X509_STORE_CTX *, X509_LOOKUP_TYPE,  X509_NAME *)' {aka 'struct x509_object_st *(struct x509_store_ctx_st *, X509_LOOKUP_TYPE,  struct X509_name_st *)'}
  341 | X509_OBJECT *X509_STORE_CTX_get_obj_by_subject(X509_STORE_CTX *vs,
      |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/usual/tls/tls.c: In function 'tls_configure_ssl':
lib/usual/tls/tls_compat.h:38:47: error: invalid use of incomplete typedef 'SSL_CTX' {aka 'struct ssl_ctx_st'}
   38 | #define SSL_CTX_get0_param(ssl_ctx) ((ssl_ctx)->param)
      |                                               ^~
lib/usual/tls/tls.c:442:42: note: in expansion of macro 'SSL_CTX_get0_param'
  442 |                 X509_VERIFY_PARAM *vfp = SSL_CTX_get0_param(ctx->ssl_ctx);
      |                                          ^~~~~~~~~~~~~~~~~~
make: *** [/pgbouncer-1.16.1/lib/mk/antimake.mk:1230: .objs/pgbouncer/lib/usual/tls/tls.o] Error 1

It builds successfully when using openssl instead of libressl, but the container was crashing soon after launch with a seg fault (SIGSEGV, exit code 139).
With libressl the container runs fine.

I added user "nobody" to prevent the image to be root by default.

The openssl addition in pgbouncer-exporter Dockerfile is to fix a vulnerability.

These changes work fine for my use case (which was purely aimed at removing vulnerabilities). Let me know if somethings needs to change to match others.

Related GitHub discussion: #29110

@boring-cyborg boring-cyborg bot added area:helm-chart Airflow Helm Chart area:production-image Production image improvements and fixes labels Feb 27, 2023
@boring-cyborg
Copy link

boring-cyborg bot commented Feb 27, 2023

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@potiuk
Copy link
Member

potiuk commented Mar 4, 2023

Thanks. Images built and pushed. I will add a change to the chart to use them.

@potiuk potiuk merged commit 9700691 into apache:main Mar 4, 2023
@boring-cyborg
Copy link

boring-cyborg bot commented Mar 4, 2023

Awesome work, congrats on your first merged pull request!

potiuk added a commit to potiuk/airflow that referenced this pull request Mar 4, 2023
The images for pgbouncer and pgbouncer exporter have been refreshed
with latest working for pgbounced alpine images and pgbpuncer
versions. This PR switches to those images by default.

Follow up after apache#29792
@eladkal eladkal added this to the Airflow Helm Chart 1.9.0 milestone Mar 4, 2023
potiuk added a commit that referenced this pull request Mar 4, 2023
…#29919)

The images for pgbouncer and pgbouncer exporter have been refreshed
with latest working for pgbounced alpine images and pgbpuncer
versions. This PR switches to those images by default.

Follow up after #29792
@andrew-stein-sp
Copy link
Contributor

@potiuk It appears only an arm64 image was built and pushed. Can you push an amd64 image please?

@potiuk
Copy link
Member

potiuk commented Mar 9, 2023

Yes - we have an issue to make multi-platform images #29967 but I pushed the AMD images just now. Thanks for notifying it @andrew-stein-sp

@evanasse evanasse deleted the update-exporters branch April 14, 2023 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:helm-chart Airflow Helm Chart area:production-image Production image improvements and fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants