Skip to content

Conversation

@nrobinson-intelycare
Copy link
Contributor

Redo of #45619 to avoid merge commit


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@nrobinson-intelycare
Copy link
Contributor Author

@o-nikolas @vincbeck @eladkal

Anything I'm missing blocking this from being merged?

@vincbeck
Copy link
Contributor

@o-nikolas @vincbeck @eladkal

Anything I'm missing blocking this from being merged?

Just curious, have you tested it manually when submit_job_timeout is not provided? In that case timeout is passed to the the api submit_job with value {"attemptDurationSeconds": None}. I am wondering it Boto3 is okay with that

@nrobinson-intelycare
Copy link
Contributor Author

nrobinson-intelycare commented Jan 15, 2025

@o-nikolas @vincbeck @eladkal
Anything I'm missing blocking this from being merged?

Just curious, have you tested it manually when submit_job_timeout is not provided? In that case timeout is passed to the the api submit_job with value {"attemptDurationSeconds": None}. I am wondering it Boto3 is okay with that

I've changed it to only add timeout to the args dict if submit_job_timeout is not None:

https://github.com/apache/airflow/pull/45660/files#diff-730ff9a9f5255ee440584231fc8733730c53a6a1a3fa3c2edd993c9d5157f219R322

@vincbeck
Copy link
Contributor

@o-nikolas @vincbeck @eladkal
Anything I'm missing blocking this from being merged?

Just curious, have you tested it manually when submit_job_timeout is not provided? In that case timeout is passed to the the api submit_job with value {"attemptDurationSeconds": None}. I am wondering it Boto3 is okay with that

I've changed it to only add timeout to the args dict if submit_job_timeout is not None:

https://github.com/apache/airflow/pull/45660/files#diff-730ff9a9f5255ee440584231fc8733730c53a6a1a3fa3c2edd993c9d5157f219R322

I missed that! LGTM :)

@o-nikolas o-nikolas merged commit caa401d into apache:main Jan 17, 2025
61 checks passed
@boring-cyborg
Copy link

boring-cyborg bot commented Jan 17, 2025

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@nrobinson-intelycare
Copy link
Contributor Author

@o-nikolas @vincbeck

Any idea when a new release of apache-airflow-providers-amazon will be made? Looking forward to using this.

@vincbeck
Copy link
Contributor

Usually releases are done every 2 weeks but @eladkal should be able to give you more details about the next wave.

@eladkal
Copy link
Contributor

eladkal commented Jan 22, 2025

Will have to wait after Celery release.
I don't have time to parallel releases. Sorry.

@nrobinson-intelycare
Copy link
Contributor Author

Will have to wait after Celery release. I don't have time to parallel releases. Sorry.

No worries, thanks!

dauinh pushed a commit to dauinh/airflow that referenced this pull request Jan 24, 2025
An execution timeout for the submit_job api call can now be passed through the operator to the boto3 call.
@nrobinson-intelycare
Copy link
Contributor Author

@eladkal Any chance of a release this week? Thanks.

got686-yandex pushed a commit to got686-yandex/airflow that referenced this pull request Jan 30, 2025
An execution timeout for the submit_job api call can now be passed through the operator to the boto3 call.
@nrobinson-intelycare
Copy link
Contributor Author

@eladkal @vincbeck Any plans for making a release soon?

niklasr22 pushed a commit to niklasr22/airflow that referenced this pull request Feb 8, 2025
An execution timeout for the submit_job api call can now be passed through the operator to the boto3 call.
@nrobinson-intelycare nrobinson-intelycare deleted the feat/support-batch-timeout branch April 8, 2025 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants