Skip to content

Fix GCSObjectExistenceSensor xcom push#37949

Closed
dmedora wants to merge 2 commits intoapache:mainfrom
dmedora:gcs-obj-sensor-xcom
Closed

Fix GCSObjectExistenceSensor xcom push#37949
dmedora wants to merge 2 commits intoapache:mainfrom
dmedora:gcs-obj-sensor-xcom

Conversation

@dmedora
Copy link
Contributor

@dmedora dmedora commented Mar 6, 2024

Fix GCSObjectExistenceSensor to return the xcom value if the object already exists upon first run. At present, if the object already exists before the DAG runs, the xcom value is not populated.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg bot added area:providers provider:google Google (including GCP) related issues labels Mar 6, 2024
)
else:
if self._exists:
return "success"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it always success, then how are we going to benefit from this? Can you elaborate the use case here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking a look, I've updated the PR description (didn't realize this would get look at while in draft). Today, if an object already exists when the DAG is first run, the xcom value is not set. "success" is what's returned today by the deferred mode trigger - https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/triggers/gcs.py#L111. The goal of this fix is essentially the same as what was done for GCSObjectsWithPrefixExistenceSensor in #37634.

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Apr 26, 2024
@github-actions github-actions bot closed this May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:google Google (including GCP) related issues stale Stale PRs per the .github/workflows/stale.yml policy file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants