KubernetesPodOperator failure email alert with actual error log from command executed

[karunpoudel]

Description

When a command executed using KubernetesPodOperator fails, the alert email only says:
Exception: Pod Launching failed: Pod pod_name_xyz returned a failure
along with other parameters supplied to the operator but doesn't contain actual error message thrown by the command.

I am thinking similar to how xcom works with KubernetesPodOperator, if the command could write the error log in sidecar container in /airflow/log/error.log and airflow picks that up, then it could be included in the alert email (probably at the top). It can use same sidecar as for xcom (if that is easier to maintain) but write in different folder.

Looks like kubernetes has a way to send termination message.
https://kubernetes.io/docs/tasks/debug-application-cluster/determine-reason-pod-failure/
Just need to pull that from container status message and include it in failure message at the top.

Use case/motivation

Similar to how email alert for most other operator includes key error message right there without having to login to airflow to see the logs, i am expecting similar functionality from KubernetesPodOperator too.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[potiuk]

Maybe you would like to contribute that feature ?

[karunpoudel]

i haven't looked into code base so not sure about the complexity. i saw the tag "good first issue". might be able to contribute with some guidance

[potiuk]

Sure. It's not very comples :). Happy to help .

[karunpoudel]

Looks like kubernetes has a way to send termination message.
https://kubernetes.io/docs/tasks/debug-application-cluster/determine-reason-pod-failure/
Just need to pull that from container status message and include it in failure message at the top.