Skip to content

Conversation

@Lee-W
Copy link
Member

@Lee-W Lee-W commented Oct 27, 2025

Why

The original implementation fails when extra contains non-hashable types (e.g., `dict``), even though such structures are valid JsonValue. This caused issues with comparing and registering asset events.

Closes: #56882

What

  • Updated extra in Asset, AssetAlias, and AssetEvent to be typed as dict[str, JsonValue] to allow JSON-serializable data explicitly.
  • Internal improvements:
    • Encode extra as JSON for comparing
    • Decode extra before registering the AssetEvent so that it can be correctly written to the database.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W
Copy link
Member Author

Lee-W commented Oct 28, 2025

The nested dict works fine. The numpy thing doesn't. will mark it as ready once finished adding the tests

@Lee-W Lee-W force-pushed the fix-asset-event-extra branch from 2610c1e to 1dca60d Compare October 28, 2025 08:25
@Lee-W Lee-W marked this pull request as ready for review October 28, 2025 08:58
@Lee-W Lee-W changed the title fix(asset): fix asset event extra Make extra of Asset, AssetAlias, AssetEvent a dict[str, JsonValue] Oct 28, 2025
@Lee-W Lee-W added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Oct 28, 2025
Copy link
Member

@jason810496 jason810496 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! LGTM overall.

@Lee-W Lee-W force-pushed the fix-asset-event-extra branch 2 times, most recently from a887544 to 92091b7 Compare October 28, 2025 09:31
Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are good, it would be nice if either @ashb / @kaxil could confirm if my analogy of cadwyn is indeed correct

@Lee-W Lee-W force-pushed the fix-asset-event-extra branch from daec944 to bce08ec Compare October 30, 2025 08:02
@Lee-W Lee-W added this to the Airflow 3.1.2 milestone Oct 31, 2025
@Lee-W Lee-W force-pushed the fix-asset-event-extra branch from bce08ec to 753de58 Compare October 31, 2025 02:32
@kaxil kaxil modified the milestones: Airflow 3.1.2, Airflow 3.1.3 Oct 31, 2025
@Lee-W Lee-W merged commit c25455e into apache:main Nov 3, 2025
114 checks passed
@Lee-W Lee-W deleted the fix-asset-event-extra branch November 3, 2025 07:29
github-actions bot pushed a commit that referenced this pull request Nov 3, 2025
…, JsonValue]` (#57352)

(cherry picked from commit c25455e)

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
@github-actions
Copy link

github-actions bot commented Nov 3, 2025

Backport successfully created: v3-1-test

Status Branch Result
v3-1-test PR Link

Lee-W added a commit to astronomer/airflow that referenced this pull request Nov 3, 2025
Lee-W added a commit that referenced this pull request Nov 3, 2025
…, JsonValue]` (#57352) (#57728)

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
fixup! [v3-1-test] Make `extra` of Asset, AssetAlias, AssetEvent a `dict[str, JsonValue]` (#57352) (cherry picked from commit c25455e)
ephraimbuddy pushed a commit that referenced this pull request Nov 3, 2025
…, JsonValue]` (#57352) (#57728)

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
fixup! [v3-1-test] Make `extra` of Asset, AssetAlias, AssetEvent a `dict[str, JsonValue]` (#57352) (cherry picked from commit c25455e)
ephraimbuddy pushed a commit that referenced this pull request Nov 4, 2025
…, JsonValue]` (#57352) (#57728)

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
fixup! [v3-1-test] Make `extra` of Asset, AssetAlias, AssetEvent a `dict[str, JsonValue]` (#57352) (cherry picked from commit c25455e)
Copilot AI pushed a commit to jason810496/airflow that referenced this pull request Dec 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:task-sdk backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Metadata.extra fails with nested dict in asset event emission (works with flat dict)

5 participants