Skip to content

Conversation

@potiuk
Copy link
Member

@potiuk potiuk commented May 23, 2023

When EMR describe-cluster, does not have LogURI information, we should silently skip persisting the link to XCom rather than fail on KeyError.

Fixes: #31480


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

When EMR describe-cluster, does not have LogURI information, we
should silently skip persisting the link to XCom rather than fail
on KeyError.

Fixes: apache#31480
@potiuk potiuk requested review from eladkal and o-nikolas as code owners May 23, 2023 17:20
@boring-cyborg boring-cyborg bot added area:providers provider:amazon AWS/Amazon - related issues labels May 23, 2023
@potiuk potiuk requested review from Taragolis and ferruzzi May 23, 2023 17:20
Copy link
Contributor

@ferruzzi ferruzzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had not realized that was a potential result, the boto docs don't make that delineation. Thanks for implementing the fix.

@eladkal
Copy link
Contributor

eladkal commented May 23, 2023

#31482 also attempts to address the issue

@potiuk
Copy link
Member Author

potiuk commented May 23, 2023

The other one is likely nicer. I did not have possibllity to test mine, and I was not sure how to "disable" the link in the BaseAWSLink so I took the "safe" route - but I am happy to close mine in favour of #31482 if that's tested one.

@potiuk potiuk closed this May 23, 2023
@potiuk potiuk deleted the fix-mising-log-uri-in-emr-link branch June 4, 2023 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missing LogUri from emr describe-cluster API when executing EmrCreateJobFlowOperator

3 participants