feat(ui): 3D stick figure visualization (COCO-17 + smooth animations)#490
Open
schwarztim wants to merge 1 commit intoruvnet:mainfrom
Open
feat(ui): 3D stick figure visualization (COCO-17 + smooth animations)#490schwarztim wants to merge 1 commit intoruvnet:mainfrom
schwarztim wants to merge 1 commit intoruvnet:mainfrom
Conversation
…animations - Add ui/spatial.html: Three.js 3D renderer for spatial dashboard - COCO-17 keypoint definitions inline; 18 bone pairs (STANDING_KEYPOINTS) - Three pose variants: standing (green), sitting (yellow), lying (blue) - Persistent figure map (no recreate-on-every-frame) via FigureState class - Position lerp (factor 0.1) for smooth interpolation between ticks - Posture-transition keypoint blending over ~1s (poseBlend 0->1 at +0.03/frame) - Breathing animation: torso Y-scale at detected BR rate from /api/v1/vital-signs - Activity-based sway: still=0.01m, moderate=0.03m, active=0.08m - Opacity fadeout for disappearing bodies (quality field decrements at -0.05/tick) - WS subscription to spatial_update + body_cluster_update; falls back to polling - Fallback: single standing figure from voxel centroid when no cluster data Depends on /api/v1/spatial/clusters endpoint from body tracking PR (ruvnet#487).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Spatial dashboards currently show voxel "blobs" from CSI tomography — non-intuitive for end users. This PR adds a COCO-17 stick figure renderer to
ui/spatial.htmlthat maps body cluster centroids to anatomical poses, with smooth lerp animations between ticks and posture-aware idle motion (breathing, gentle sway).Changes
ui/spatial.html— Three.js / Canvas-based 3D renderer with:pose_hint: standing (green), sitting (yellow), lying (blue)FigureStatepersistent figure map — no recreate-on-every-frameposeBlend0→1 at +0.03/frame)/api/v1/vital-signsqualitydecrements at −0.05/tick)CHANGELOG.md— entry under[Unreleased]Backend dependency
This UI consumes:
GET /api/v1/spatial/clusters— body cluster centroids + bbox + pose hints (provided by body tracking PR feat(sensing-server): body tracking platform with persistent IDs and zones #487)GET /api/v1/tracking/bodies— persistent body IDs (provided by feat(sensing-server): body tracking platform with persistent IDs and zones #487)GET /api/v1/vital-signs— breathing rate for animation pacing (existing endpoint)For best results merge this AFTER #487. Filing now to surface for parallel review.
WebSocket schema
The renderer subscribes to the existing
/ws/sensingendpoint and handlesspatial_updatemessages that include aclustersarray:{ "type": "spatial_update", "data": { "clusters": [ { "id": 0, "centroid": [x, y, z], "bbox_min": [x, y, z], "bbox_max": [x, y, z], "pose_hint": "standing | sitting | lying" } ] } }Falls back to polling
/api/v1/spatial/clustersevery 2 s if WebSocket is unavailable.COCO-17 joint definitions
18 bone pairs connecting adjacent joints per the COCO topology. Definitions are documented inline in
spatial.htmlasSTANDING_KEYPOINTSandBONESconstants.Visual notes
#050508, overlaysrgba(10,10,15,0.85)— MD3-compatible dark surfacerequestAnimationFrame; lerp keeps motion smooth at 2 s poll intervalsTest results
cargo check --workspace --no-default-features(v2/): PASS (1 pre-existing warning, 0 errors)Notes