feat: Replace polling with SSE for real-time InferenceService updates by Prateekbala · Pull Request #160 · kserve/models-web-app

Prateekbala · 2026-02-18T14:01:59Z

Summary

Replace inefficient HTTP polling with Server-Sent Events (SSE) for real-time updates from Kubernetes resources. This implementation reduces network traffic, decreases backend load, and provides immediate visibility into resource state changes.

Changes

Backend

Added 4 new SSE endpoints leveraging Kubernetes Watch API:
- /api/sse/namespaces/<namespace>/inferenceservices - Watch all InferenceServices
- /api/sse/namespaces/<namespace>/inferenceservices/<name> - Watch single InferenceService
- /api/sse/namespaces/<namespace>/inferenceservices/<name>/events - Watch Kubernetes events
- /api/sse/namespaces/<namespace>/inferenceservices/<name>/logs - Stream pod logs
New SSE module with thread-safe connection manager and watchers
Input validation (max 10 components, bounded queues of 500)
Automatic backoff on errors with 5-second retry intervals
Graceful cleanup on client disconnect

Frontend

New SSEService with 4 watch methods matching backend endpoints
Updated components with SSE integration + polling fallback:
- Index page: Watch namespace InferenceServices
- Server info: Watch single InferenceService + events + logs
- Events component: Real-time Kubernetes event updates
- Logs component: Stream component logs
Reconnection logic with max 3 attempts
Heartbeat handling to detect dead connections

Configuration

Added SSE_ENABLED environment variable (default: true) in both deployment configs
Fallback to polling if SSE unavailable

Closes #152

juliusvonkohout · 2026-02-18T15:17:14Z

Very interesting @LogicalGuy77

alokdangre · 2026-02-19T05:39:19Z

@@ -69,6 +71,7 @@ export class ServerInfoComponent implements OnInit, OnDestroy {
    retries: 1,
  });
  private pollingSubscription = new Subscription();


we are replacing polling so this should be removed

it is there as a fallback so I think it should be fine

What is the default approach for controllers? If server side notifications are guaranteed by Kubernetes, why would we want to keep polling Code. Is it a core feature or only available on some kubernetes? Dead code removal is very important.

Watch API can sometimes fail because of network issues. So I built in automatic reconnection logic — it tries 3 times, and if it still fails, it falls back to polling. This way we get fast real-time updates most of the time with SSE, but if that connection drops, polling takes over so you always get your updates.

alokdangre

this cuases lint errors

alokdangre · 2026-02-19T05:49:20Z

@Prateekbala did you checked locally , can you provide ss?

LogicalGuy77 · 2026-02-19T19:07:46Z

@Prateekbala thank you for adding this feature, I will have a look at this next week since I'll be occupied with my exams.

Prateekbala · 2026-02-21T22:32:10Z

I have attached the screenshot.

Prateekbala · 2026-03-05T18:58:29Z

@LogicalGuy77 can you please do a final review for this PR

LogicalGuy77 · 2026-03-06T20:49:20Z

hey @Prateekbala I'd suggest to write a new cypress test file to test SSE, you can mock kubernetes sent events. Just a simple test should suffice

Copilot

Pull request overview

This PR replaces the existing HTTP polling mechanism with Server-Sent Events (SSE) for real-time updates from Kubernetes resources, addressing issue #152. It adds new SSE endpoints on the backend that leverage the Kubernetes Watch API, a new frontend SSEService, and updates four components to use SSE with polling fallback. Configuration is controlled via an SSE_ENABLED environment variable.

Changes:

Added backend SSE module with 4 streaming endpoints (watchers.py, routes.py, manager.py), a thread-safe connection manager, and Kubernetes Watch-based watchers for InferenceServices, events, and logs.
Created frontend SSEService with EventSource-based Observable wrappers and updated IndexComponent, ServerInfoComponent, EventsComponent, and LogsComponent to use SSE with polling fallback.
Added SSE_ENABLED configuration to backend, frontend config service, kustomize manifests, and Cypress test fixtures.

Reviewed changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
`backend/apps/common/sse/__init__.py`	New SSE module init, exports manager and blueprint
`backend/apps/common/sse/manager.py`	SSEConnectionManager for managing watch streams and client queues
`backend/apps/common/sse/watchers.py`	Kubernetes watchers (InferenceService, Events, Logs) using threading
`backend/apps/common/sse/routes.py`	Flask SSE endpoint route handlers
`backend/apps/common/__init__.py`	Registers SSE blueprint
`backend/apps/v1beta1/routes/get.py`	Adds `sseEnabled` to config API response
`backend/requirements.txt`	Adds `gevent` dependency
`frontend/src/app/services/sse.service.ts`	New SSEService with EventSource-based Observable wrappers
`frontend/src/app/services/config.service.ts`	Adds `sseEnabled` default to config
`frontend/src/app/types/config.ts`	Adds `sseEnabled` to AppConfig type
`frontend/src/app/pages/index/index.component.ts`	SSE integration for InferenceService list with polling fallback
`frontend/src/app/pages/server-info/server-info.component.ts`	SSE integration for single InferenceService with polling fallback
`frontend/src/app/pages/server-info/events/events.component.ts`	SSE integration for Kubernetes events with polling fallback
`frontend/src/app/pages/server-info/logs/logs.component.ts`	SSE integration for pod logs with polling fallback
`frontend/package.json`	Adds `eventsource-polyfill` dependency
`config/base/kustomization.yaml`	Adds `SSE_ENABLED` env var
`config/overlays/kubeflow/kustomization.yaml`	Adds `SSE_ENABLED` env var
`frontend/cypress/e2e/*.cy.ts`	Updates Cypress tests to mock SSE config

Prateekbala · 2026-03-09T21:10:11Z

I have resolved all the copilot reviews ,@LogicalGuy77 can you please look into this.

I have added Cypress e2e tests to cover the SSE functionality. The SSE_ENABLED feature flag was also removed to simplify the code, and that change required additional e2e test fixes to align with the updated behavior which i have made.

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

juliusvonkohout · 2026-04-04T16:10:19Z

Please fix the conflict and dco tests

alokdangre reviewed Feb 19, 2026

View reviewed changes

alokdangre suggested changes Feb 19, 2026

View reviewed changes

Comment thread frontend/src/app/pages/server-info/logs/logs.component.ts Outdated

Comment thread frontend/src/app/pages/server-info/logs/logs.component.ts Outdated

LogicalGuy77 reviewed Mar 6, 2026

View reviewed changes

Comment thread config/overlays/kubeflow/kustomization.yaml Outdated

LogicalGuy77 reviewed Mar 6, 2026

View reviewed changes

Comment thread frontend/cypress/e2e/index-page.cy.ts Outdated

LogicalGuy77 reviewed Mar 6, 2026

View reviewed changes

Comment thread frontend/cypress/e2e/model-deletion.cy.ts Outdated

LogicalGuy77 reviewed Mar 6, 2026

View reviewed changes

Comment thread frontend/cypress/e2e/model-edit.cy.ts Outdated

juliusvonkohout requested a review from Copilot March 6, 2026 22:37

Copilot started reviewing on behalf of juliusvonkohout March 6, 2026 22:37 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

Prateekbala force-pushed the sse-feature-new branch 5 times, most recently from 2343bc2 to ce7bca5 Compare March 9, 2026 21:01

Prateekbala added 9 commits March 14, 2026 02:59

feat: Implement SSE real-time updates for InferenceServices

f49d25d

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

docs: Added better comments

0ca7438

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

chore: Update package-lock.json with eventsource-polyfill dependency

70ac83f

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

fix: fixed e2e test failures

122b849

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

fix: removed SSE_ENABLED feature flag

e415f5b

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

added cypress test

6b37ca9

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

fix:fixed test failures

9a5d9ca

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

fixed test failures

5b827a9

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

fix:fixed failed test

476ba5a

Signed-off-by: Prateek Bala <prateekbala28@gmail.com>

Prateekbala force-pushed the sse-feature-new branch from ce7bca5 to 476ba5a Compare March 13, 2026 21:30

fix: removed unwanted logs

7ea9c01

Conversation

Prateekbala commented Feb 18, 2026

Summary

Changes

Backend

Frontend

Configuration

Uh oh!

juliusvonkohout commented Feb 18, 2026

Uh oh!

alokdangre Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

LogicalGuy77 Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

juliusvonkohout Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Prateekbala Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

alokdangre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alokdangre commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LogicalGuy77 commented Feb 19, 2026

Uh oh!

Prateekbala commented Feb 21, 2026

Uh oh!

Prateekbala commented Mar 5, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LogicalGuy77 commented Mar 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Prateekbala commented Mar 9, 2026

Uh oh!

juliusvonkohout commented Apr 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

alokdangre commented Feb 19, 2026 •

edited

Loading