Add Idaho weather stations analysis tasks

[ScuttleBot]

Adds 3 new data-analysis tasks based on the Idaho weather stations CSV dataset:

1. task_csv_stations_by_elevation - Rank stations by elevation, compute summary stats, compare NWS vs NRCS agency averages
2. task_csv_stations_coverage - Analyze geographic coverage gaps across Idaho's 44 counties, identify missing Payette County, flag data quality issues
3. task_csv_stations_filter - Multi-criteria filtering with DMS coordinate parsing, cross-tabulation by agency and elevation band

All tasks use assets/csvs/idaho_weather_stations.csv (213 stations, 10 columns). Each includes hybrid grading with automated Python checks and LLM judge rubrics.

Closes #220, Closes #223, Closes #235

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Solid additions — the task definitions are well-structured with clear prompts, accurate expected values, and thorough hybrid grading that combines automated regex checks with LLM judge rubrics. The DMS coordinate parsing challenge in the filter task is a nice test of real-world data handling.

Files Reviewed (4 files)

• tasks/manifest.yaml
• tasks/task_csv_stations_by_elevation.md
• tasks/task_csv_stations_coverage.md
• tasks/task_csv_stations_filter.md

---

_{Reviewed by claude-4.6-sonnet-20260217 · 91,779 tokens}

[ScuttleBot]

🧪 Test Started

Instance: f751856b (Vultr ATL, vc2-2c-4gb)
Branch: tasks/csv-weather-stations
Tasks:

• task_csv_stations_by_elevation
• task_csv_stations_coverage
• task_csv_stations_filter

Models (running in parallel):

Model	Provider
openrouter/anthropic/claude-opus-4.6	Anthropic
openrouter/openai/gpt-5.4	OpenAI
openrouter/google/gemini-3.1-pro-preview	Google

ETA: ~30-40 minutes (3 tasks × 3 models, parallel execution)

Instance booting now. Will post results when complete.

[ScuttleBot]

🧪 Test Results — PR #322 (Idaho Weather Stations Tasks)

Instance: f751856b (Vultr ATL, vc2-2c-4gb, Ubuntu 22.04)
Branch: tasks/csv-weather-stations
Benchmark: PinchBench 2.0.0-rc1
Run time: ~15 minutes (3 models in parallel)

Overall Scores

Model	Mean Score	Cost	Tokens	Tokens/Task
claude-opus-4.6	0.964	$1.33	409,790	136,597
gpt-5.4	0.778	$0.32	281,216	93,739
gemini-3.1-pro-preview	0.648	$0.46	292,836	97,612

Per-Task Breakdown

task_csv_stations_by_elevation

Model	Score	Time	Notes
claude-opus-4.6	0.916	113s	All automated checks passed ✅. Judge: accurate data, good agency analysis, minor recommendation clarity issues
gpt-5.4	0.600	72s	All automated checks passed ✅. Judge gave 0 across the board — incomplete transcript sent to judge (missing parts 2/5, 3/5)
gemini-3.1-pro	0.600	120s	All automated checks passed ✅. Judge failed: "no parseable response after all attempts"

task_csv_stations_coverage

Model	Score	Time	Notes
claude-opus-4.6	1.000	178s	Perfect score! All automated + all judge criteria passed. Correctly identified Payette gap, county counts, elevation bands, NWS/NRCS roles
gpt-5.4	0.960	157s	All automated passed ✅. Judge: slight deductions on agency analysis visibility and recommendations verifiability (transcript truncation)
gemini-3.1-pro	0.557	61s	6/7 automated passed, partial on agency_comparison (0.5). Judge failed entirely

task_csv_stations_filter

Model	Score	Time	Notes
claude-opus-4.6	0.975	90s	Near-perfect. All 4 filters correct. Minor: cross-tab table truncated in transcript but stats verify correct
gpt-5.4	0.773	57s	⚠️ NRCS Custer County filter returned 0 instead of 4 stations — logic bug. Other 3 filters correct
gemini-3.1-pro	0.787	93s	NRCS Custer count check failed (0 credit), but found correct top station. Cross-tab partial (0.5). Judge partially worked here

Key Observations

Tasks are working well:

• All 3 tasks loaded from the manifest correctly
• CSV asset (idaho_weather_stations.csv) was properly placed in agent workspaces
• Hybrid grading (automated + LLM judge) executed for all tasks
• Automated grading checks are solid — correctly catching real errors (e.g., GPT-5.4 missing NRCS/Custer stations)

⚠️ LLM Judge Issues:

• Gemini got "LLM judge failed: no parseable response after all attempts" on 2/3 tasks — this suppressed 40% of the possible score
• GPT-5.4 got 0/4 from judge on task 1 due to "transcript incomplete" — the judge couldn't evaluate work it couldn't see
• Claude was the only model that got judge scores on all 3 tasks
• This is likely a benchmark infrastructure issue, not a task design issue. The judge (claude-opus-4.5) may be hitting rate limits or context issues when multiple models are graded in parallel

Task design feedback:

• task_csv_stations_by_elevation: Clean, well-validated. 7/7 automated checks differentiate well
• task_csv_stations_coverage: Excellent task — separated Claude (1.0) from GPT-5.4 (0.96) from Gemini (0.557). Good discriminating power
• task_csv_stations_filter: The NRCS/Custer County filter is a good trap — caught real bugs in both GPT-5.4 and Gemini's CSV filtering logic. DMS coordinate parsing worked across all models

Verdict: Tasks look solid. The main issue is intermittent LLM judge failures that penalize models unfairly. Consider adding retry logic or a fallback judge model.

Instance Cleanup

Instance f751856b will be destroyed after posting this comment.