Skip to content

feat: Add streaming endpoint for real-time intervention plan generation#9

Open
keithesanks-prog wants to merge 8 commits intotillioss:mainfrom
keithesanks-prog:feat/streaming-endpoint
Open

feat: Add streaming endpoint for real-time intervention plan generation#9
keithesanks-prog wants to merge 8 commits intotillioss:mainfrom
keithesanks-prog:feat/streaming-endpoint

Conversation

@keithesanks-prog
Copy link
Copy Markdown

Summary

This PR adds a new streaming endpoint that enables real-time token streaming for intervention plan generation using Server-Sent Events (SSE).

Changes

  • ✅ Add POST /stream endpoint with Server-Sent Events (SSE) support
  • ✅ Implement StreamingModel wrapper for Gemini streaming API
  • ✅ Add build_prompt_and_model helper function in gateway.py
  • ✅ Update CORS configuration to allow frontend origins (localhost:8501)
  • ✅ Fix pydantic version conflict in requirements.txt (>=2.9.0)
  • ✅ Add comprehensive documentation for streaming endpoint
  • ✅ Support automatic filling of missing EMT score fields

API Details

Endpoint: POST /stream

Content-Type: application/json
Response: text/event-stream

Example Request:

{
  "scores": {
    "EMT1": [35.0, 40.0, 38.0]
  },
  "metadata": {
    "class_id": "A1",
    "deficient_area": "EMT1",
    "num_students": 25
  }
}

Example Response:

data: {"token": "..."}
data: {"token": "..."}
data: {"status": "complete"}

Testing

Test with curl:

curl -N -X POST http://localhost:8000/stream \
  -H "Content-Type: application/json" \
  -d '{"scores":{"EMT1":[35,40,38]}, "metadata":{"class_id":"A1","deficient_area":"EMT1","num_students":25}}'

Documentation

  • Updated README.md with streaming endpoint documentation
  • Updated DEPLOYMENT.md with testing instructions
  • Updated project structure to show new API endpoints directory

Files Changed

  • app/api/endpoints/stream.py (new)
  • app/api/endpoints/__init__.py (new)
  • app/llm/gateway.py (added StreamingModel and build_prompt_and_model)
  • app/main.py (added router, updated CORS)
  • requirements.txt (fixed pydantic version)
  • README.md (added streaming documentation)
  • DEPLOYMENT.md (added testing instructions)

Keith added 8 commits November 7, 2025 00:37
- Add POST /stream endpoint with Server-Sent Events (SSE) support
- Implement StreamingModel wrapper for Gemini streaming API
- Add build_prompt_and_model helper function in gateway.py
- Update CORS configuration to allow frontend origins
- Update requirements.txt to fix pydantic version conflict
- Add comprehensive documentation for streaming endpoint
- Support automatic filling of missing EMT score fields
- Document SSE implementation approach
- Explain StreamingModel wrapper architecture
- Describe async generator pattern
- Detail prompt building process
- Add code structure reference
…ible

- Updated /health endpoint to return HTTP 200 even if components are degraded
- Added component-level health status (llm_healthy, curriculum_healthy)
- Improved error handling with graceful degradation
- Updated HealthResponse schema to include component status fields
- Updated documentation with health check behavior details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant