Skip to content

Conversation

@guan404ming
Copy link
Member

Related Issue

closes #44759

Why

How

cc: @eladkal @tirkarthi @avyuktsoni0731


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Co-Authored-By: Avyukt Soni <95626105+avyuktsoni0731@users.noreply.github.com>
Pushing XComs from Deferred Tasks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. versionadded:: 2.10.0
Copy link
Contributor

@eladkal eladkal Apr 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have an easy way to generate a list of all our Trigger that were written before 2.10 and don't support this? so we can patch them? in ~1 week or so the min version for providers will be 2.10 so we can safely fix it.

Also maybe, an idea how to get the list... adding a test to make sure any sub class of BaseTrigger has implementation for this? That would also protect us for future adding new triggers

Copy link
Member Author

@guan404ming guan404ming May 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late. It took some time to deal with this. And I think this func is not supported for all triggers before 2.10.0 since there are not implementation of some thing like self.xcoms or something like the example do.
I only see the trigger test which use the functionality the example show

@pytest.mark.parametrize(
"event_cls, expected",
[
(TaskSuccessEvent, "success"),
(TaskFailedEvent, "failed"),
(TaskSkippedEvent, "skipped"),
],
)
@patch("airflow.utils.timezone.utcnow")
def test_submit_event_task_end(mock_utcnow, session, create_task_instance, event_cls, expected):

please let me know if there is anything I could help

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my point. Minimum version for Airflow in providers is 2.10 so we should open a tracking issue to modify all trigger we have to support it. Don't we?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After more investigation, it seems currently trigger is not accept xcoms #46677 (comment) after reworked. xcoms is passed when the event have xcoms.

Should we modify our example or try to modify trigger to accept "xcoms" when init?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could leave this here until there is any re-implementation plan.

@guan404ming guan404ming changed the title docs: add Pushing XComs from Deferred example Add Pushing XComs from Deferred example May 2, 2025
@guan404ming guan404ming marked this pull request as draft May 4, 2025 08:09
@guan404ming guan404ming closed this May 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Trigger event from deferred task does not get scheduled immediately, leading to timeout.

2 participants