airflow/utils/email.py def send_mime_email doesn't take all values from connection

Apache Airflow version

2.7.1

What happened

When defining an SMTP connection only the user and password are taken from the connection. The host, port, starttls, start_ssl, from_email, timeout and retry are all taken from the configuration and not from the connection. Although the smtp provider (https://airflow.apache.org/docs/apache-airflow-providers-smtp/1.0.0/index.html) provides all these fields.

What you think should happen instead

The following lines should be included in airflow/utils/email.py around line 246:

if conn_id is not None:
    try:
        from airflow.hooks.base import BaseHook

        airflow_conn = BaseHook.get_connection(conn_id)
        smtp_user = airflow_conn.login
        smtp_password = airflow_conn.password
        smtp_host = airflow_conn.host
        smtp_port = airflow_conn.port

        extra = json.loads(airflow_conn.extra)

        smtp_starttls = not extra["disable_tls"]
        smtp_timeout = int(extra["timeout"])
        smtp_retry_limit = int(extra["retry_limit"])
        smtp_ssl = not extra["disable_ssl"]

The from_email should be retrieved from the connection earlier in the call stack.

How to reproduce

Install the SMTP provider:
pip install apache-airflow-providers-smtp

Create an SMTP connection in the UI
Set the AIRFLOW__EMAIL__EMAIL_CONN_ID environment variable to the connection id of the SMTP connection

Use the EmailOperator (from airflow.operators.email import EmailOperator) in a DAG.

Check the values that are used in send_mime_email.

Operating System

Ubuntu

Versions of Apache Airflow Providers

apache-airflow-providers-smtp==1.3.2

Deployment

Docker-Compose

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[eladkal]

@hussein-awala is working on the smtp migration from core to smtp provider

[github-actions]

This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.

[scottlimmer]

What's the status of this issue? I've just wasted too much time as a result of this and I'd like to see it fixed.

Is it actively being worked on? Will a PR be considered? Is there a reason why https://github.com/apache/airflow/blob/main/airflow/utils/email.py#L247,L255 doesn't include other SMTP params?

[potiuk]

What's the status of this issue? I've just wasted too much time as a result of this and I'd like to see it fixed.

Is it actively being worked on? Will a PR be considered? Is there a reason why https://github.com/apache/airflow/blob/main/airflow/utils/email.py#L247,L255 doesn't include other SMTP params?

The status is that it waits for someone to fix. As usual. And if you want to volunteer to fix it - we can assign it to you and you might lead it to completion @scottlimmer - otherwise it will have to wait for one of the > 3000 contributors to volunteer and implement it. That's how open-source development works, in case you were not aware.

Generally contributing it back and fixing is a great way to give back for the absolutely free softwre that you can use and get without any guarantees if you read the licence. And contributing a fix back might be great way to help others people like you who might have struggled with it - or might struggle with it in the future. So I heartily recommend it. It's also great way to feel good that you don't just "get" from the community, but you also "give".

Second best - if you do not feel skilled enough to contribute back - is to find someone who you encourage (or sometimes pay if you have a company or just want to help the community in monetary, rather than engineering time ways. This happens a lot - many people who contribute here are paid individually to contribute features or fixes that are needed by those who decide to pay them.

Eventually, you might simply wait until someone picks an interest and fixes it. This might be helped by marking the issue as "good first issue" - which I just did. Then you contribute nothing, but also your expectations here "it happens when it happens".

So .. pick your favourite way how you might (or might choose not) want to contribute to fix the issue.

[potiuk]

Is there a reason why https://github.com/apache/airflow/blob/main/airflow/utils/email.py#L247,L255 doesn't include other SMTP params?

It might depend. Make your PR and a reviewer will find it much easier to discuss details over the code that you will propose.

[scottlimmer]

@potiuk Apologies, my ealier initial comment sound grumpier than I'd intended. I was really looking to clarify if there is already an existing larger body of work (as suggested by #34554 (comment)) that might explain why the existing PR went stale.

Happy to have a go at fixing this.