fix(chart): CeleryExecutor namespace depends on Airflow version

[mikaeld]

CeleryExecutor is being decoupled from Airflow in version 2.7.0. This work was done in PR #32526.
However, the changed Celery Worker liveness probe command breaks backward compatibility.

This PR adds a semverCompare to use the right import namespace depending on the Airflow version defined in the chart.

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[potiuk]

Right. Indeed Airflow version is the "cut-off". Even if you install the new provider for <2.7.0 still the old package name and old celery classes will be used, so this is fine.

[potiuk]

Thanks for finding and fixing it @mikaeld ❤️

[potiuk]

Right. Indeed Airflow version is the "cut-off". Even if you install the new provider for <2.7.0 still the old package name and old celery classes will be used, so this is fine.

Actually there is an edge case :) . But I am not sure if we want to handle it @jedcunningham ....

You could potentially install the new provider for Airflow 2.6. or 2.5 and set "airflow.providers.celery.executors.celery_executor.CeleryExecutor" as your executor. Then the liveness probe would not work. But I think this case is not even properly handled by our Helm Chart yet.

[jedcunningham]

Those users can set workers.livenessProbe.command in that case, probably the best answer. But yeah, the chart won't handle it nicely either way. Way too much is relying on executor: CeleryExecutor.