Status of testing of Apache Airflow Helm Chart 1.8.0rc1

[jedcunningham]

We have a kind request for all the contributors to the latest Apache Airflow Helm Chart 1.8.0rc1.

Could you please help us to test the RC versions of the Apache Airflow Helm Chart?

Please let us know in the comment if the issue is addressed in the latest RC.

• Add nodePort declaration to chart/values.schema.json (#26945): @joseph-max-coalfire
  Linked issues:
  • Add NodePort Option to the values schema (#26812)
• Make custom env vars optional for job templates (#27148): @richardbadman @tseruga
  Linked issues:
  • [Helm Chart] Decouple custom_airflow_environment from all pods and jobs (#26045)
• Add workeer log-groomer-sidecar enable option in helm chart (#27178): @BobDu @jedcunningham
  Linked issues:
  • Adds support for custom command and args (#16153)
• Chart: Default airflow version to 2.4.2 (#27230): @ephraimbuddy
• Add webserverConfigConfigMapName (#27419): @JulesTriomphe
• ExtraVolumeMounts in sidecars and initContainers (#27420): @JulesTriomphe
• allow pgbouncer replicas to be configurable (#27439): @pgvishnuram
• [BUMP] helm chart - redis 7 (#27443): @raphaelauv
• Chart: add hostaliases to pod template file (#27544): @Aakcht
• Chart: Decrease default gitSync wait (#27625): @jedcunningham
• Chart: Update default airflow version to 2.4.3 (#27668): @ephraimbuddy
• Correct repo example for cloning via ssh - comments in helm chart (#27671): @schudoku @morkot
  Linked issues:
  • helm chart - mounting-dags-from-a-private-github-repo-using-git-sync-sidecar (#21970)
• Replace helm chart icon (#27704): @ronahk
• Chart: add airflow_local_settings to all airflow containers (#27779): @rauljmz @Aakcht @venkatnpedada
  Linked issues:
  • Not able to use custom logging with airflow helm chart (#27498)
  • Cannot reference airflow_local_settings in config settings (#25637)
• Add global volume & volumeMounts to the chart (#27781): @csp33
  Linked issues:
  • Allow global volume & volumeMounts in Helm chart (#27687)
• git-sync 3.6.3 (#27848): @danielhoherd
• Enhance chart to allow over-riding command-line args to statsd exporter (#28041): @rob-1126
• Chart: Default airflow version to 2.5.0 (#28074): @ephraimbuddy
• Fix helm chart NOTES.txt to show correct URL (#28264): @okue
• Remove checks for 1.19 api checks (#28461): @potiuk @Jmdaming
  Linked issues:
  • Wrong apiVersion Detected During Ingress Creation (#22657)
• Add missing annotations on deployments [#28604] (#28688): @sudeepgupta90
  Linked issues:
  • Missing annotations on Deployments (#28604)
• Add worker service account for LocalKubernetesExecutor (#28813): @Chen-Oliver
• Fix helm chart create scheduler service to serve task logs for LocalKubernetesExecutor (#28828): @snjypl
• Fix incorrect comments in worker-kedaautoscaler.yaml (#28858): @EricGao888
• Chart: add doc note about podtemplate images (#29032): @Aakcht
• Upgrade bitnami/postgresql helm depenency from 10.5.3 to 12.1.9 (#29071): @snjypl
• Update Airflow version to 2.5.1 (#29074): @pierrejeambrun
• Fix PgBouncer after postgres subchart upgrade (#29207): @snjypl @jedcunningham
  Linked issues:
  • Upgrade bitnami/postgresql helm depenency from 10.5.3 to 12.1.9 (#29071)

Thanks to all who contributed to the release (probably not a complete list!):
@richardbadman @ronahk @pgvishnuram @sudeepgupta90 @Jmdaming @joseph-max-coalfire @Aakcht @tseruga @JulesTriomphe @morkot @rauljmz @pierrejeambrun @csp33 @raphaelauv @rob-1126 @jedcunningham @okue @venkatnpedada @Chen-Oliver @schudoku @EricGao888 @danielhoherd @BobDu @snjypl @potiuk @ephraimbuddy

[danielhoherd]

I have validated #27848 which was easy to validate. It was just a version bump.

[EricGao888]

I have validated #28858 which fixed some comments in the source code.

[Aakcht]

Checked #27544 , #27779 and #29032 - all good.

[sudeepgupta90]

Checked #28604 <#28604> - all good

…

On Wed, Feb 1, 2023 at 10:46 AM Aakcht ***@***.***> wrote: Checked #27544 <#27544> , #27779 <#27779> and #29032 <#29032> - all good. — Reply to this email directly, view it on GitHub <#29273 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5XLD44466PUGRJMP5D6RLWVHWTJANCNFSM6AAAAAAUM3WRA4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[csp33]

Checked #27781
Works as expected

[pgvishnuram]

Checked #27439 works as expected

[JulesTriomphe]

I have spotted multiple issues:

• There is a missing prefixing - here to cleanly render the volume mounts for the migrate-database-job, same here for the dag-processor, here, here and here for the scheduler, here and here for the triggerer, here and here for the webserver, here, here, here and here for the workers
• Globally defined volumes are missing in the pod-template-file which will make pod spawning fail on clusters where readOnlyRootFilesystem is True and a volume for airflow home has been set globally. Workaround would be to set that volume in each system's extraVolumes and extraVolumeMounts but that would greatly reduce the functionality of global volumes and volume mounts.
• There is a missing trailing - here to cleanly render the "airflow_dags_mount"
• There is a missing prefixing - here to cleanly render webserver service ports, and I didn't manage to clean the rendering of the fields themselves (name aligned with the dash). The current state looks like this:

ports:

    - 
      name: airflow-ui
      port: 8080

I will try adding the missing globally defined volumes to the pod template file, deploy and then open a PR to correct this and another for the rendering issues.

[richardbadman]

Checked #27148, works as expected

[ronahk]

Checked #27704 <#27704> works as expected.

…

On Tue, Jan 31, 2023 at 10:09 PM Jed Cunningham ***@***.***> wrote: We have a kind request for all the contributors to the latest Apache Airflow Helm Chart 1.8.0rc1 <https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.8.0rc1>. Could you please help us to test the RC versions of the Apache Airflow Helm Chart? Please let us know in the comment if the issue is addressed in the latest RC. - Add nodePort declaration to chart/values.schema.json (#26945) <#26945>: @joseph-max-coalfire <https://github.com/joseph-max-coalfire> Linked issues: - Add NodePort Option to the values schema (#26812) <#26812> - Make custom env vars optional for job templates (#27148) <#27148>: @richardbadman <https://github.com/richardbadman> @tseruga <https://github.com/tseruga> Linked issues: - [Helm Chart] Decouple custom_airflow_environment from all pods and jobs (#26045) <#26045> - Add workeer log-groomer-sidecar enable option in helm chart (#27178) <#27178>: @BobDu <https://github.com/BobDu> @jedcunningham <https://github.com/jedcunningham> Linked issues: - Adds support for custom command and args (#16153) <#16153> - Chart: Default airflow version to 2.4.2 (#27230) <#27230>: @ephraimbuddy <https://github.com/ephraimbuddy> - Add webserverConfigConfigMapName (#27419) <#27419>: @JulesTriomphe <https://github.com/JulesTriomphe> - ExtraVolumeMounts in sidecars and initContainers (#27420) <#27420>: @JulesTriomphe <https://github.com/JulesTriomphe> - allow pgbouncer replicas to be configurable (#27439) <#27439>: @pgvishnuram <https://github.com/pgvishnuram> - [BUMP] helm chart - redis 7 (#27443) <#27443>: @raphaelauv <https://github.com/raphaelauv> - Chart: add hostaliases to pod template file (#27544) <#27544>: @Aakcht <https://github.com/Aakcht> - Chart: Decrease default gitSync wait (#27625) <#27625>: @jedcunningham <https://github.com/jedcunningham> - Chart: Update default airflow version to 2.4.3 (#27668) <#27668>: @ephraimbuddy <https://github.com/ephraimbuddy> - Correct repo example for cloning via ssh - comments in helm chart (#27671) <#27671>: @schudoku <https://github.com/schudoku> @morkot <https://github.com/morkot> Linked issues: - helm chart - mounting-dags-from-a-private-github-repo-using-git-sync-sidecar (#21970) <#21970> - Replace helm chart icon (#27704) <#27704>: @ronahk <https://github.com/ronahk> - Chart: add airflow_local_settings to all airflow containers (#27779) <#27779>: @rauljmz <https://github.com/rauljmz> @Aakcht <https://github.com/Aakcht> @venkatnpedada <https://github.com/venkatnpedada> Linked issues: - Not able to use custom logging with airflow helm chart (#27498) <#27498> - Cannot reference airflow_local_settings in config settings (#25637) <#25637> - Add global volume & volumeMounts to the chart (#27781) <#27781>: @csp33 <https://github.com/csp33> Linked issues: - Allow global volume & volumeMounts in Helm chart (#27687) <#27687> - git-sync 3.6.3 (#27848) <#27848>: @danielhoherd <https://github.com/danielhoherd> - Enhance chart to allow over-riding command-line args to statsd exporter (#28041) <#28041>: @rob-1126 <https://github.com/rob-1126> - Chart: Default airflow version to 2.5.0 (#28074) <#28074>: @ephraimbuddy <https://github.com/ephraimbuddy> - Fix helm chart NOTES.txt to show correct URL (#28264) <#28264>: @okue <https://github.com/okue> - Remove checks for 1.19 api checks (#28461) <#28461>: @potiuk <https://github.com/potiuk> @Jmdaming <https://github.com/Jmdaming> Linked issues: - Wrong apiVersion Detected During Ingress Creation (#22657) <#22657> - Add missing annotations on deployments [#28604] (#28688) <#28688>: @sudeepgupta90 <https://github.com/sudeepgupta90> Linked issues: - Missing annotations on Deployments (#28604) <#28604> - Add worker service account for LocalKubernetesExecutor (#28813) <#28813>: @Chen-Oliver <https://github.com/Chen-Oliver> - Fix helm chart create scheduler service to serve task logs for LocalKubernetesExecutor (#28828) <#28828>: @snjypl <https://github.com/snjypl> - Fix incorrect comments in worker-kedaautoscaler.yaml (#28858) <#28858>: @EricGao888 <https://github.com/EricGao888> - Chart: add doc note about podtemplate images (#29032) <#29032>: @Aakcht <https://github.com/Aakcht> - Upgrade bitnami/postgresql helm depenency from 10.5.3 to 12.1.9 (#29071) <#29071>: @snjypl <https://github.com/snjypl> - Update Airflow version to 2.5.1 (#29074) <#29074>: @pierrejeambrun <https://github.com/pierrejeambrun> - Fix PgBouncer after postgres subchart upgrade (#29207) <#29207>: @snjypl <https://github.com/snjypl> @jedcunningham <https://github.com/jedcunningham> Linked issues: - Upgrade bitnami/postgresql helm depenency from 10.5.3 to 12.1.9 (#29071) <#29071> Thanks to all who contributed to the release (probably not a complete list!): @richardbadman <https://github.com/richardbadman> @ronahk <https://github.com/ronahk> @pgvishnuram <https://github.com/pgvishnuram> @sudeepgupta90 <https://github.com/sudeepgupta90> @Jmdaming <https://github.com/Jmdaming> @joseph-max-coalfire <https://github.com/joseph-max-coalfire> @Aakcht <https://github.com/Aakcht> @tseruga <https://github.com/tseruga> @JulesTriomphe <https://github.com/JulesTriomphe> @morkot <https://github.com/morkot> @rauljmz <https://github.com/rauljmz> @pierrejeambrun <https://github.com/pierrejeambrun> @csp33 <https://github.com/csp33> @raphaelauv <https://github.com/raphaelauv> @rob-1126 <https://github.com/rob-1126> @jedcunningham <https://github.com/jedcunningham> @okue <https://github.com/okue> @venkatnpedada <https://github.com/venkatnpedada> @Chen-Oliver <https://github.com/Chen-Oliver> @schudoku <https://github.com/schudoku> @EricGao888 <https://github.com/EricGao888> @danielhoherd <https://github.com/danielhoherd> @BobDu <https://github.com/BobDu> @snjypl <https://github.com/snjypl> @potiuk <https://github.com/potiuk> @ephraimbuddy <https://github.com/ephraimbuddy> — Reply to this email directly, view it on GitHub <#29273>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AR36W4FYUKOGRX2WB5DSY3DWVFWPJANCNFSM6AAAAAAUM3WRA4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[rob-1126]

Checked #28041 - works as expected.

[jedcunningham]

@JulesTriomphe, thanks for testing this out! Since the issue you found isn't a regression from 1.7.0, it'll end up going in the next helm chart release instead of blocking this one.

[joseph-max-coalfire]

Tested #26945 -- looks good, working as intended 💯