Skip to content

feat(sensing-server): adaptive person count — RollingP95 + dedup_factor runtime API#491

Open
schwarztim wants to merge 1 commit intoruvnet:mainfrom
schwarztim:pr/adaptive-person-count
Open

feat(sensing-server): adaptive person count — RollingP95 + dedup_factor runtime API#491
schwarztim wants to merge 1 commit intoruvnet:mainfrom
schwarztim:pr/adaptive-person-count

Conversation

@schwarztim
Copy link
Copy Markdown

Motivation

Person counting in v2/ uses fixed-scale feature normalization which works well in calibrated environments but degrades when signal characteristics drift across rooms, interference levels, or hardware. Two improvements here are deployment-neutral:

1. RollingP95 adaptive normalizer

compute_person_score() previously normalized features with hard-coded denominators (variance/300, motion_band_power/250, spectral_power/500). When live ESP32 values exceed those limits the normalized inputs clamp to 1.0 and dynamic range collapses. RollingP95 is a streaming P95 estimator (600-sample / ~30 s sliding window) that self-calibrates to whatever feature distribution the deployment produces. Cold-start (< 60 samples) falls back to the legacy denominators so day-0 behaviour is fully preserved.

2. dedup_factor runtime API

Exposes the multi-node cluster deduplication divisor via REST so deployments can tune to their environment without rebuilding. Includes an auto-tune endpoint that derives the optimal dedup_factor from a known person count (calibration mode). Config persists across restarts in data/config.json.

Explicitly NOT included

This fork also has additional ISTA lambda tuning (specifically lambda=5.0) for its local 8×8×4 babycube grid. Those values are deployment-specific and intentionally not included in this PR — they would degrade person-count quality on different room geometries. This PR keeps upstream's existing lambda: 0.1 default.

Changes

  • v2/crates/wifi-densepose-sensing-server/src/main.rs

    • New RollingP95 struct + impl (ADR-044 §5.2)
    • New RuntimeConfig struct + load_runtime_config / save_runtime_config (ADR-044 §5.3)
    • AppStateInner: added p95_variance, p95_motion_band_power, p95_spectral_power, dedup_factor, data_dir fields
    • compute_person_score() signature: &FeatureInfo&AppStateInner + &FeatureInfo (adaptive denominators)
    • All 3 call sites updated; P95 push calls added before each scoring call
    • New REST handlers: config_get_dedup_factor, config_set_dedup_factor, config_set_ground_truth
    • Routes registered: GET/POST /api/v1/config/dedup-factor, POST /api/v1/config/ground-truth
    • 5 unit tests in rolling_p95_tests module
  • v2/crates/wifi-densepose-sensing-server/src/multistatic_bridge.rs

    • fuse_or_fallback() gains dedup_factor: f64 parameter; fallback switches from max() to ceil(sum / dedup_factor)
    • Test call site updated

Test results

  • cargo test --workspace --no-default-features: 1636 passed, 0 failed (includes 5 new RollingP95 unit tests)
  • python archive/v1/data/proof/verify.py: VERDICT: FAIL — pre-existing on origin/main (numpy/scipy version drift); not caused by this PR

Notes

  • The auto-tune endpoint accepts a known person count and adjusts dedup_factor to match observed cluster count. Useful for calibration during install.
  • RollingP95 is a generic primitive — could be reused for other adaptive thresholds in future.
  • lambda: 0.1 in tomography.rs is untouched.

…or runtime API

RollingP95 adaptive normalizer (ADR-044 §5.2):
- Streaming P95 estimator (600-sample / ~30 s window) replaces fixed-scale
  denominators (variance/300, motion/250, spectral/500) that saturated against
  live ESP32 values, collapsing dynamic range to zero.
- Cold-start (<60 samples) falls back to legacy denominators — day-0 behaviour
  is preserved.
- Three new fields on AppStateInner: p95_variance, p95_motion_band_power,
  p95_spectral_power (all RollingP95::new(600, 60)).
- compute_person_score() refactored to accept &AppStateInner; all three call
  sites (wifi, wifi-fallback, simulated) updated.
- 5 unit tests in rolling_p95_tests module.

dedup_factor runtime API (ADR-044 §5.3):
- New field dedup_factor: f64 (default 3.0) on AppStateInner.
- fuse_or_fallback() gains dedup_factor param; fallback switches from max() to
  sum/dedup_factor (ceiling), matching the fork's sum-based aggregation.
- RuntimeConfig struct + load/save_runtime_config() for data/config.json
  persistence across restarts.
- Three new REST endpoints:
    GET  /api/v1/config/dedup-factor
    POST /api/v1/config/dedup-factor
    POST /api/v1/config/ground-truth (auto-tune from known person count)

Explicitly NOT included:
- lambda=5.0 (upstream keeps its 0.1 default — deployment-specific tuning)
- CC intensity threshold 0.3 and min-cluster-size 4 hardcodes
- max_cc_size filter removal
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant