Skip to content

Conversation

@fredthomsen
Copy link
Contributor

PR #55111 introduced SerializedParam to decouple from the Task SDK, but inadvertently broke the API contract by changing .dump() to .resolve() in the params serialization. This caused the UI to crash because FlexibleForm expects the full param spec with {value, schema, description} but was receiving only the resolved values.

The original SDK Param class had a .dump() method that returned the full spec, so this restores the API contract rather than creating new behavior.

Related: #55111


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaxil / @uranusjr do you want to take a look at this one?

@jscheffl
Copy link
Contributor

Checked - and thanks for the fix! But a small nit in serialization still blocks "Params UI Tutorial" Dag is you start this from examples:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/python/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.12/site-packages/fastapi/applications.py", line 1082, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 29, in __call__
    await responder(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 130, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 46, in __call__
    await self.app(scope, receive, self.send_with_compression)
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/base.py", line 182, in __call__
    with recv_stream, send_stream, collapse_excgroups():
                                   ^^^^^^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/usr/python/lib/python3.12/site-packages/starlette/_utils.py", line 85, in collapse_excgroups
    raise exc
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/base.py", line 184, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/airflow/airflow-core/src/airflow/api_fastapi/auth/middlewares/refresh_token.py", line 49, in dispatch
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/base.py", line 159, in call_next
    raise app_exc
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/base.py", line 144, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/python/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/python/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/python/lib/python3.12/site-packages/starlette/routing.py", line 716, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
    await route.handle(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/routing.py", line 290, in handle
    await self.app(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/routing.py", line 78, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/python/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/python/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/python/lib/python3.12/site-packages/starlette/routing.py", line 75, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.12/site-packages/fastapi/routing.py", line 334, in app
    content = await serialize_response(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.12/site-packages/fastapi/routing.py", line 188, in serialize_response
    return field.serialize(
           ^^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.12/site-packages/fastapi/_compat.py", line 152, in serialize
    return self._type_adapter.dump_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.12/site-packages/pydantic/type_adapter.py", line 605, in dump_python
    return self.serializer.to_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.PydanticSerializationError: Unable to serialize unknown type: <class 'airflow.sdk.definitions._internal.types.ArgNotSet'>

@jscheffl jscheffl added this to the Airflow 3.1.1 milestone Oct 19, 2025
@jscheffl jscheffl added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Oct 19, 2025
@kaxil kaxil modified the milestones: Airflow 3.1.1, Airflow 3.2.0 Oct 19, 2025
@kaxil kaxil requested a review from uranusjr October 19, 2025 22:32
@kaxil
Copy link
Member

kaxil commented Oct 19, 2025

Changed the GH milestone for this & #55111 to 3.2.0 -- to not risk any regression in 3.1.1

@fredthomsen
Copy link
Contributor Author

@jscheffl ah yikes. Ok update made.

PR apache#55111 introduced SerializedParam to decouple from the Task SDK, but
inadvertently broke the API contract by changing `.dump()` to `.resolve()`
in the params serialization. This caused the UI to crash because
FlexibleForm expects the full param spec with {value, schema, description}
but was receiving only the resolved values.

The original SDK Param class had a `.dump()` method that returned the full
spec, so this restores the API contract rather than creating new behavior.

Related: apache#55111
Copy link
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good now and is working fine with the examples I have. LGTM... after CI is made happy.

Copy link
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uranusjr Any final thoughts on this?

@uranusjr uranusjr merged commit 4791bbb into apache:main Oct 21, 2025
112 checks passed
@github-actions
Copy link

Backport failed to create: v3-1-test. View the failure log Run details

Status Branch Result
v3-1-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 4791bbb v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

@pierrejeambrun
Copy link
Member

@jscheffl need manual backport

@bbovenzi
Copy link
Contributor

@kaxil why is this labeled with 3.1 backport while the milestone is 3.2.0?

@jscheffl jscheffl removed the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Oct 21, 2025
@jscheffl
Copy link
Contributor

@kaxil why is this labeled with 3.1 backport while the milestone is 3.2.0?

Originating PR #55111 is not back-ported and such this fix also does not need backporting. Removed label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:serialization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants