Skip to content

fix(serve-grpc): clearer error when given a new-SDK service#5609

Open
saivedant169 wants to merge 1 commit into
bentoml:mainfrom
saivedant169:fix/serve-grpc-new-sdk-error-5607
Open

fix(serve-grpc): clearer error when given a new-SDK service#5609
saivedant169 wants to merge 1 commit into
bentoml:mainfrom
saivedant169:fix/serve-grpc-new-sdk-error-5607

Conversation

@saivedant169
Copy link
Copy Markdown
Contributor

What does this PR address?

Fixes #5607

bentoml serve-grpc accepts any importable target, including services defined with the @bentoml.service decorator (the bentoml>=1.2 programming model). gRPC is only implemented for the legacy bentoml.Service API, so pointing serve-grpc at a new-SDK service fails with:

Error: [serve] `serve-grpc` failed: <class '_bentoml_sdk.service.factory.Service'> type doesn't support gRPC serving

The message is correct but unhelpful. The user has no way to tell from it that the new programming model has no gRPC path and that the fix is to switch transports or APIs.

Repro

# service.py
import bentoml

@bentoml.service(name="grpc_smoke")
class Supervisor:
    @bentoml.api
    def greet(self, name: str = "world") -> str:
        return f"hello {name}"
$ bentoml serve-grpc service.py:Supervisor --port 9098
Error: [serve] `serve-grpc` failed: <class '_bentoml_sdk.service.factory.Service'> type doesn't support gRPC serving

After this PR

$ bentoml serve-grpc service.py:Supervisor --port 9098
Error: [serve] `serve-grpc` failed: gRPC serving is not supported for services defined with the `@bentoml.service` decorator (the bentoml>=1.2 programming model). Use `bentoml serve` to run the service over HTTP, or rewrite it using the legacy `bentoml.Service` API if gRPC is required.

Changes

  • src/bentoml/serving.py: in serve_grpc_production, detect _bentoml_sdk.Service instances explicitly and raise a BentoMLException that names the programming model and points at the two viable workarounds. Any other unrecognised type still falls through to the existing generic message.
  • tests/unit/test_serve_grpc_new_sdk_error.py: writes a minimal @bentoml.service definition into a tmp dir and asserts serve_grpc_production raises the new wording.

I did not implement gRPC for the new SDK in this PR. That is a feature, not a bug fix, and the serve-grpc CLI is already hidden=True which suggests it is being phased out for new-style services.

Before submitting:

  • Does the Pull Request follow Conventional Commits specification naming? Yes (fix: prefix)
  • Does the code follow BentoML's code style, pre-commit run -a script has passed? Yes, ran on changed files.
  • Did you read through contribution guidelines and follow development guidelines?
  • Did your changes require updates to the documentation? No, error message change only.
  • Did you write tests to cover your changes?

`bentoml serve-grpc` accepts any importable target. When users point it
at a service defined with the `@bentoml.service` decorator (the
bentoml>=1.2 programming model) it fails with:

    <class '_bentoml_sdk.service.factory.Service'> type doesn't support gRPC serving

That message is correct but unhelpful. gRPC is implemented only for the
legacy `bentoml.Service` API, and the new SDK has no gRPC code path, so
this case will not work and there is nothing the user can do beyond
switching transports or APIs.

Detect that situation explicitly in `serve_grpc_production` and raise a
message that names the new programming model, points the user at
`bentoml serve` for HTTP, and at the legacy `bentoml.Service` API if
they need gRPC. Unrelated types still fall through to the existing
generic error.

Adds a unit test that creates a temporary `@bentoml.service` definition,
calls `serve_grpc_production`, and asserts the new wording.

Closes bentoml#5607
@saivedant169 saivedant169 requested a review from a team as a code owner May 13, 2026 01:10
@saivedant169 saivedant169 requested review from bojiang and removed request for a team May 13, 2026 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: [serve] serve-grpc failed: <class '_bentoml_sdk.service.factory.Service'> type doesn't support gRPC serving

1 participant