diff --git a/Dashboard/DESIGN_ANALYSIS.md b/Dashboard/DESIGN_ANALYSIS.md new file mode 100644 index 0000000..9d8224c --- /dev/null +++ b/Dashboard/DESIGN_ANALYSIS.md @@ -0,0 +1,714 @@ +# Dashboard Design Analysis +**Date:** 2025-11-14 +**Session:** claude/dashboard-design-analysis-01QStwxyiGYGTEbpcQCXYYCh +**Purpose:** Analysis of user design feedback against current dashboard implementation + +--- + +## Executive Summary + +The user feedback reveals a fundamental shift in dashboard philosophy: +- **FROM:** Balanced layout with equal emphasis on map and data panels +- **TO:** Map-centric design with interactive controls and graphical visualization of propagation + +**Key Insight:** The user wants to answer one primary question: +> *"Where in the world can I communicate with now? On which band? Which modes?"* + +This requires: +1. **Map as primary interface** (not 60/40 split, more like 80/20 or full-screen with overlays) +2. **Interactive controls** (band/mode/time selection via radio buttons) +3. **Mode-aware predictions** (FT8/SSB/CW have different SNR requirements) +4. **DXCC on map** (not in sidebar) +5. **Graphical visualization** (isolines, colored regions, maidenhead grid - not just paths) + +--- + +## Design Feedback Categorization + +### 1. Core Philosophy & User Experience + +| Feedback | Current State | Gap Analysis | +|----------|--------------|--------------| +| **"Focus more on map and less on sidebars"** | 60/40 split (map 1.5fr, sidebar 1fr) | Need 80/20 or overlay approach | +| **"More graphical-style"** | Line paths only | Need area fills, isolines, grid squares | +| **"User friendly, modern design"** | Dark theme, clean UI ✅ | Good foundation, but needs simplification | +| **"Initial focus: near-term operations"** | Shows current hour + 72h forecast ✅ | Correct focus | + +**Priority:** HIGH +**Effort:** MEDIUM (layout restructure) + +--- + +### 2. Primary User Question + +#### User Need: +> *"Where in the world can I communicate with now? On which band? Which modes?"* + +#### Current Implementation: +- **Band:** Shown in Band Details tab, not interactive +- **Mode:** NOT IMPLEMENTED ❌ +- **Location:** Map shows paths, but not regions/zones +- **"Now":** Current hour is detected ✅ + +#### Required Changes: +1. **Add mode selection** (FT8/digital, SSB, CW) +2. **Mode-specific SNR thresholds** + - FT8: SNR ≥ -10 dB (very weak signal mode) + - SSB: SNR ≥ 10 dB (standard voice) + - CW: SNR ≥ 3 dB (better than SSB, worse than FT8) +3. **Show "open" regions graphically** on map (not just paths) +4. **Interactive band selector** (radio buttons, not tabs) + +**Priority:** CRITICAL +**Effort:** HIGH (new data model + UI) + +--- + +### 3. Mode Handling + +| Feedback | Current State | Implementation Needed | +|----------|--------------|----------------------| +| **"Are modes a variable in likelihood?"** | No mode awareness | Yes - each mode has different SNR floor | +| **"FT8/digi ; SSB; CW"** | Only one SNR threshold | Three mode profiles | +| **"Different SNR requirements"** | Current: SNR ≥ 10 dB = GOOD | FT8: -10 dB / CW: 3 dB / SSB: 10 dB | +| **"Radio buttons for mode select"** | No mode selector | Add radio button group | + +#### Recommended SNR Thresholds: + +```javascript +const MODE_THRESHOLDS = { + 'FT8': { + good: -10, // FT8 can decode at -20 dB, use conservative -10 + fair: -15, + poor: -20 + }, + 'CW': { + good: 3, // CW readable at 0 dB, comfortable at 3+ + fair: 0, + poor: -3 + }, + 'SSB': { + good: 10, // Current threshold (voice needs higher SNR) + fair: 5, + poor: 0 + } +}; +``` + +#### UI Control: +```html +
+ + + + + + + + +
+``` + +**Priority:** HIGH +**Effort:** MEDIUM + +--- + +### 4. Map Visualization Enhancements + +#### 4.1 Current Map Features ✅ +- Leaflet.js interactive map +- Greyline visualization (updates every 5 minutes) +- Default zoom to global view (zoom level 2) +- Home station marker (VE1ATM) +- Region markers (10 regions) +- Propagation paths (color-coded) + +#### 4.2 Requested Enhancements + +| Feature | Current | Requested | Implementation Method | +|---------|---------|-----------|----------------------| +| **Visualization method** | Paths only | Isolines / Vector shapes / Grid | Leaflet polygon layers | +| **DXCC entities** | Sidebar list | Map pins/zones | GeoJSON overlay + markers | +| **"What's open"** | Implicit from paths | Explicit region highlighting | Filled polygons with color coding | +| **Tooltips** | On region markers | On hover everywhere | Leaflet tooltip API | +| **Greyline** | ✅ Implemented | ✅ Keep | Already working | +| **Zoom default** | ✅ Both hemispheres | ✅ Keep | Already correct | +| **Home banner green lines** | Propagation paths | Remove/simplify | User finds non-intuitive | + +#### 4.3 Visualization Options Analysis + +**Option A: Isolines (Contour Lines)** +- Draw contours of constant signal strength +- Similar to weather isobar maps +- **Pro:** Familiar to radio operators, shows gradients +- **Con:** Complex calculation, can be cluttered + +**Option B: Vector Shape Areas (Filled Polygons)** +- Define regions as filled shapes (GOOD/FAIR/POOR/CLOSED) +- Color-code by quality +- **Pro:** Clear, unambiguous, easy to understand +- **Con:** Sharp boundaries may not reflect gradual propagation + +**Option C: Maidenhead Grid** +- Overlay 6-digit grid squares +- Color each square by signal quality +- **Pro:** Ham radio standard, contest-friendly +- **Con:** Grid may be too fine/coarse depending on zoom + +**Option D: Hybrid Approach** ⭐ **RECOMMENDED** +- Base layer: Filled region polygons (10 regions) +- Overlay: DXCC entity pins for "most wanted" +- Optional toggle: Maidenhead grid overlay +- Tooltip on hover: Band/mode/SNR/S-meter + +**Priority:** HIGH +**Effort:** HIGH + +--- + +### 5. DXCC Integration + +| Feedback | Current State | Recommended Change | +|----------|--------------|-------------------| +| **"Logbook import identifies desired DXCC"** | Supported via JSON | ✅ Keep | +| **"Can that be a map layer of pins or zones?"** | Sidebar list only | Add GeoJSON markers | +| **"DXCC tracking takes up space"** | Full tab dedicated | Move to overlay/popup | +| **"Not friendly, takes up space for closed areas"** | Shows all needed entities | Filter to "workable now" | +| **"Maybe a map?"** | List-based | Convert to map markers | + +#### Implementation Approach: + +1. **Create DXCC GeoJSON** + - Use DXCC entity coordinates (already in codebase mapping) + - Add properties: entity_id, name, worked, confirmed, needed + +2. **Map Markers** + - Green pin: Worked & confirmed + - Yellow pin: Worked, not confirmed + - Red pin: Needed + propagation open NOW + - Gray pin: Needed but closed + +3. **Filtering** + - Default view: Show only "needed + open" entities + - Toggle: "Show all needed" (even if closed) + - Toggle: "Show worked entities" + +4. **Popup on Click** + - Entity name + - Worked/Confirmed status + - Current propagation: band + S-meter + - Distance/bearing + - Link to QRZ or DXCC info + +**Priority:** MEDIUM +**Effort:** MEDIUM + +--- + +### 6. Interactive Controls + +| Control | Requested | Current | Implementation | +|---------|-----------|---------|----------------| +| **Band select** | Radio buttons | Tab-based implicit | Add radio button group (7 bands) | +| **Mode select** | Radio buttons | None | Add radio button group (FT8/SSB/CW) | +| **Time window** | Radio buttons | Implicit "now" | Add radio buttons (Now / +3h / +6h / +12h / Custom) | + +#### Recommended UI Layout: + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ 📡 VE1ATM HF Propagation Monitor │ +├─────────────────────────────────────────────────────────────────┤ +│ Band: ⦿ 40m ○ 30m ○ 20m ○ 17m ○ 15m ○ 12m ○ 10m │ +│ Mode: ⦿ FT8 ○ SSB ○ CW │ +│ Time: ⦿ Now ○ +3h ○ +6h ○ +12h ○ Custom │ +├─────────────────────────────────────────────────────────────────┤ +│ │ +│ 🌍 WORLD MAP (80% height) │ +│ [Interactive Leaflet map] │ +│ • Filled regions (color-coded) │ +│ • DXCC pins (needed entities) │ +│ • Greyline overlay │ +│ • Tooltips on hover │ +│ │ +├─────────────────────────────────────────────────────────────────┤ +│ Quick Info: [Stats]│ +│ • 20m OPEN to EU, JA, VK (FT8) │ +│ • Best band now: 20m (8 regions) │ +│ • DXCC needed + open: 12 entities │ +└─────────────────────────────────────────────────────────────────┘ +``` + +**Priority:** HIGH +**Effort:** MEDIUM + +--- + +### 7. Current UI Issues + +#### 7.1 Quick Summary + +**Feedback:** *"Gives facts but not very usable or friendly"* + +**Current Implementation:** +``` +Quick Summary +───────────── +Best Bands Right Now +• 40m: 3 regions open +• 20m: 5 regions open +• 15m: 2 regions open + +DXCC Progress +• Worked: 147 +• Confirmed: 89 +• Needed: 193 +``` + +**Problem:** Too generic, doesn't answer "what should I do NOW?" + +**Recommended Replacement:** +``` +🎯 RIGHT NOW (0045 UTC) +────────────────────── +BEST OPPORTUNITY +• 20m to JA: S9+20 (FT8) +• 15m to VK: S7 (SSB marginal) + +DXCC OPPORTUNITIES +• VP8 (Falklands) - 20m S9 +• 3B8 (Mauritius) - 17m S7 +[Show on map] button + +NEXT 3 HOURS +• 40m opens to EU at 0200 UTC +• 10m closes to SA at 0130 UTC +``` + +**Priority:** MEDIUM +**Effort:** LOW + +--- + +#### 7.2 72-Hour Forecast Bug + +**Feedback:** *"Clock updating to current time so showing past rather than now + 72"* + +**Problem Diagnosis:** +The forecast should show: +- **T+0:** Current hour (e.g., 0045 UTC → show 0100 UTC block) +- **T+72:** 72 hours from now + +If it's showing past hours, likely causes: +1. Time zone conversion error +2. Array indexing starting at wrong offset +3. Display logic showing "hours ago" instead of "hours ahead" + +**Required Investigation:** +- Check `renderTimeline()` function in dashboard.html:1047-1140 +- Verify `propData.timeline_24h.hours` array structure +- Confirm UTC hour detection logic + +**Priority:** HIGH (BUG FIX) +**Effort:** LOW + +--- + +#### 7.3 Home Banner with Green Lines + +**Feedback:** *"Current home banner with green lines to some continents is not helpful / non intuitive"* + +**Current Implementation:** +- Propagation paths from VE1ATM to regions +- Color-coded by quality (green/yellow/red) + +**User Perception:** +- Unclear what the lines represent +- Too abstract +- Doesn't convey "I can talk to here" + +**Recommended Change:** +**Option A:** Remove paths, use filled regions only +**Option B:** Keep paths but add legend and labels +**Option C:** Make paths toggle-able (default OFF) + +**Recommendation:** Option A (remove paths, use filled polygons) + +**Priority:** LOW (cosmetic) +**Effort:** LOW + +--- + +### 8. Missing Features + +| Feature | User Mentioned? | Priority | Effort | +|---------|----------------|----------|--------| +| Mode selection | ✅ Yes | CRITICAL | MEDIUM | +| Mode-specific SNR | ✅ Yes | HIGH | MEDIUM | +| DXCC on map | ✅ Yes | HIGH | MEDIUM | +| Filled regions | ✅ Implied | HIGH | HIGH | +| Maidenhead grid | ✅ Yes | MEDIUM | MEDIUM | +| Tooltips everywhere | ✅ Yes | MEDIUM | LOW | +| Radio button controls | ✅ Yes | HIGH | MEDIUM | +| Time window selector | ✅ Yes | MEDIUM | MEDIUM | +| Better Quick Summary | ✅ Implied | MEDIUM | LOW | +| Fix 72h forecast | ✅ Yes | HIGH | LOW | + +--- + +## Implementation Priority Matrix + +### Phase 1: Critical Fixes (Week 1) +**Goal:** Fix bugs and add mode awareness + +1. **Fix 72-hour forecast bug** ⚠️ + - Ensure timeline shows T+0 to T+72, not past hours + - File: `dashboard.html:1047-1140` + - Effort: 2 hours + +2. **Add mode selection UI** + - Radio buttons: FT8 / SSB / CW + - File: `dashboard.html` (add control panel) + - Effort: 4 hours + +3. **Implement mode-specific SNR thresholds** + - Update quality calculation logic + - File: `dashboard.html` (update status determination) + - Effort: 4 hours + +4. **Add tooltips on map hover** + - Show band/mode/SNR/S-meter for regions + - File: `dashboard.html` (Leaflet tooltip API) + - Effort: 3 hours + +**Total Phase 1:** ~13 hours + +--- + +### Phase 2: Map-Centric Redesign (Week 2-3) +**Goal:** Make map the primary interface + +5. **Restructure layout to 80/20 (map-focused)** + - Reduce sidebar prominence + - Make Quick Summary a collapsible overlay + - File: `dashboard.html` (CSS grid + HTML structure) + - Effort: 6 hours + +6. **Add band selection radio buttons** + - Replace tab-based navigation + - All controls in single row + - File: `dashboard.html` + - Effort: 4 hours + +7. **Implement filled region polygons** + - Replace paths with color-coded areas + - Use Leaflet polygon layers + - File: `dashboard.html` (add polygon drawing) + - Effort: 8 hours + +8. **Improve Quick Summary** + - Show actionable "best opportunity now" + - DXCC opportunities (needed + open) + - Next 3-hour preview + - File: `dashboard.html` (rewrite renderQuickSummary) + - Effort: 4 hours + +**Total Phase 2:** ~22 hours + +--- + +### Phase 3: DXCC Map Integration (Week 4) +**Goal:** Move DXCC from sidebar to map + +9. **Create DXCC GeoJSON layer** + - Entity coordinates → map markers + - Color-code by worked/confirmed/needed status + - File: New file `dxcc_geojson.py` + `dashboard.html` + - Effort: 8 hours + +10. **Add DXCC filtering controls** + - Show only "needed + open" by default + - Toggle switches for worked/confirmed + - File: `dashboard.html` + - Effort: 4 hours + +11. **DXCC entity popups** + - Click marker → show entity details + - Current propagation info + - File: `dashboard.html` + - Effort: 3 hours + +**Total Phase 3:** ~15 hours + +--- + +### Phase 4: Advanced Visualization (Week 5-6) +**Goal:** Add optional visualization layers + +12. **Maidenhead grid overlay** + - Toggle-able grid squares + - Color-code by signal quality + - File: `dashboard.html` (new overlay layer) + - Effort: 10 hours + +13. **Time window selector** + - Radio buttons: Now / +3h / +6h / +12h / Custom + - Update map to show future propagation + - File: `dashboard.html` + `enhanced_predictions.json` structure + - Effort: 6 hours + +14. **Remove/simplify propagation paths** + - Make toggle-able (default OFF) + - Or remove entirely in favor of filled regions + - File: `dashboard.html` + - Effort: 2 hours + +**Total Phase 4:** ~18 hours + +--- + +## Total Implementation Estimate +- **Phase 1 (Critical):** 13 hours +- **Phase 2 (Map-centric):** 22 hours +- **Phase 3 (DXCC):** 15 hours +- **Phase 4 (Advanced):** 18 hours +- **TOTAL:** ~68 hours (~9 working days) + +--- + +## Technical Specifications + +### Data Model Changes + +#### Current Structure: +```json +{ + "current_conditions": { + "bands": { + "40m": { + "regions": { + "EU": {"reliability": 75, "snr": 15} + } + } + } + } +} +``` + +#### Proposed Structure (add mode awareness): +```json +{ + "current_conditions": { + "bands": { + "40m": { + "regions": { + "EU": { + "reliability": 75, + "snr": 15, + "modes": { + "FT8": "GOOD", + "CW": "GOOD", + "SSB": "GOOD" + } + } + } + } + } + } +} +``` + +**Change Required:** `transform_data.py` must calculate mode-specific status + +--- + +### Region Polygon Coordinates + +Define region boundaries as Leaflet polygon coordinates: + +```javascript +const REGION_POLYGONS = { + 'EU': [ + [71.0, -10.0], // Northwest (Iceland) + [71.0, 40.0], // Northeast (Scandinavia) + [36.0, 40.0], // Southeast (Greece) + [36.0, -10.0], // Southwest (Portugal) + ], + 'NA': [ + [70.0, -170.0], // Alaska + [70.0, -50.0], // Labrador + [25.0, -80.0], // Florida + [30.0, -120.0], // California + ], + // ... define all 10 regions +}; +``` + +--- + +### DXCC Entity Coordinates + +Expand current entity-to-region mapping with lat/lon: + +```javascript +const DXCC_COORDS = { + '1': {name: 'Canada', lat: 60.0, lon: -95.0, region: 'NA'}, + '2': {name: 'United States', lat: 38.0, lon: -97.0, region: 'NA'}, + '21': {name: 'Crete', lat: 35.24, lon: 24.89, region: 'EU'}, + // ... all 340 entities +}; +``` + +**Source:** Can extract from ADIF DXCC specification or ClubLog database + +--- + +## Risk Assessment + +| Risk | Probability | Impact | Mitigation | +|------|------------|--------|------------| +| **Region polygon overlap** | Medium | Medium | Carefully define boundaries, test edge cases | +| **DXCC coordinate accuracy** | Low | Medium | Use authoritative source (ClubLog/ARRL) | +| **Mode threshold tuning** | High | Low | Make thresholds configurable, gather user feedback | +| **Performance (map rendering)** | Medium | High | Use Leaflet layer groups, toggle visibility | +| **Mobile responsiveness** | Medium | Medium | Test on tablets/phones, add media queries | +| **Data file size growth** | Low | Low | Current 51 KB, mode data adds ~10% | + +--- + +## User Feedback Alignment Summary + +| User Request | Current State | Alignment | Action Required | +|-------------|---------------|-----------|-----------------| +| Near-term ops focus | ✅ Implemented | 100% | None | +| User-friendly design | ✅ Good base | 80% | Minor UX improvements | +| Modern design | ✅ Dark theme | 90% | Keep current aesthetic | +| Map > sidebars | ❌ 60/40 split | 40% | Restructure to 80/20 | +| Graphical style | ⚠️ Paths only | 30% | Add filled regions | +| "Where can I talk?" | ⚠️ Implicit | 50% | Make explicit with regions | +| Mode selection | ❌ None | 0% | Critical addition | +| Mode-specific SNR | ❌ None | 0% | Critical addition | +| Band radio buttons | ❌ Tabs | 0% | Replace tabs | +| Time radio buttons | ❌ Implicit | 0% | Add selector | +| DXCC on map | ❌ Sidebar | 0% | Add GeoJSON layer | +| Tooltips | ⚠️ Markers only | 40% | Expand to regions | +| Greyline | ✅ Implemented | 100% | None | +| Both hemispheres | ✅ Implemented | 100% | None | +| Remove green lines | N/A | N/A | User preference (remove paths) | +| Better Quick Summary | ⚠️ Generic | 30% | Make actionable | +| Fix 72h forecast | ❌ Bug | 0% | Fix time offset | +| DXCC friendly | ❌ Takes space | 20% | Move to map | + +**Overall Alignment Score:** 42% (needs significant work) + +--- + +## Recommended Next Steps + +### Immediate Actions (This Week) +1. ✅ **Create this design analysis document** +2. 🔧 **Fix 72-hour forecast bug** (2 hours) +3. 🎨 **Prototype mode selection UI** (mockup or HTML prototype) +4. 📊 **Define mode SNR thresholds** (research + document) + +### Short-Term (Next 2 Weeks) +5. 🚀 **Implement Phase 1** (critical fixes + mode support) +6. 🧪 **User testing** of mode selection +7. 🗺️ **Prototype filled region polygons** (single band test) + +### Medium-Term (Month 1-2) +8. 🎨 **Complete Phase 2** (map-centric redesign) +9. 📍 **Complete Phase 3** (DXCC map integration) +10. 📝 **Gather user feedback** on new design + +### Long-Term (Month 3+) +11. 🌐 **Complete Phase 4** (advanced visualizations) +12. 🔄 **Iterate based on user feedback** +13. 📚 **Update documentation** (user guide + screenshots) + +--- + +## Questions for User + +Before proceeding with implementation, please clarify: + +1. **Region visualization preference:** + - A) Filled polygons (solid color regions) + - B) Isolines (contour lines) + - C) Maidenhead grid squares + - D) Combination (which?) + +2. **DXCC display:** + - Show all 193 needed entities on map (may be cluttered)? + - Or only show "needed + currently open" (dynamic)? + - Pin style preference (color-coded? sized by signal strength?) + +3. **Time window selector:** + - Predefined intervals (Now/+3h/+6h/+12h)? + - Or slider (continuous 0-72 hours)? + - Or both? + +4. **Propagation paths:** + - Remove entirely? + - Keep as toggle-able layer? + - Keep for selected region only? + +5. **Quick Summary location:** + - Bottom bar (minimal)? + - Collapsible right panel? + - Popup overlay? + - Remove entirely? + +6. **Priority confirmation:** + - Agree with Phase 1-4 priority order? + - Any features to move up/down? + - Budget/timeline constraints? + +--- + +## References + +### Current Implementation Files +- **Dashboard UI:** `/home/user/dvoacap-python/Dashboard/dashboard.html` (1,294 lines) +- **Data Transform:** `/home/user/dvoacap-python/Dashboard/transform_data.py` (170 lines) +- **Prediction Engine:** `/home/user/dvoacap-python/Dashboard/generate_predictions.py` (340 lines) +- **API Server:** `/home/user/dvoacap-python/Dashboard/server.py` (244 lines) + +### External Resources +- **Leaflet.js Documentation:** https://leafletjs.com/reference.html +- **DXCC Entity List:** ARRL DXCC specification +- **Maidenhead Grid:** https://en.wikipedia.org/wiki/Maidenhead_Locator_System +- **FT8 SNR Requirements:** WSJT-X documentation (-20 dB decode threshold) +- **SSB/CW SNR:** ITU-R recommendations + +--- + +## Conclusion + +The user feedback reveals a clear vision for a **map-centric, mode-aware, graphical propagation dashboard** that directly answers "where can I communicate now?" + +**Key Gaps:** +1. ❌ No mode selection (FT8/SSB/CW) +2. ❌ Map is secondary (should be primary) +3. ❌ DXCC in sidebar (should be on map) +4. ❌ No filled region visualization +5. ⚠️ 72-hour forecast bug + +**Strengths to Preserve:** +1. ✅ Greyline visualization +2. ✅ Modern dark theme +3. ✅ Interactive Leaflet map foundation +4. ✅ Real-time solar data integration +5. ✅ 72-hour forecasting capability + +**Estimated Implementation:** 68 hours (9 working days) over 4 phases + +**Recommendation:** Proceed with **Phase 1** immediately (bug fixes + mode support), then iterate based on user feedback before committing to full Phase 2-4 implementation. + +--- + +**Document Status:** DRAFT for review +**Next Action:** User review and clarification of questions above +**Author:** Claude Code Agent +**Session ID:** 01QStwxyiGYGTEbpcQCXYYCh