diff --git a/docs/roadmap-plan.md b/docs/roadmap-plan.md new file mode 100644 index 0000000..891d7af --- /dev/null +++ b/docs/roadmap-plan.md @@ -0,0 +1,248 @@ +# Improvement Roadmap Execution Plan + +This document converts the roadmap checklist into a prioritized, dependency-aware +execution plan based on the current repository state. + +## Current Technical Baseline + +The roadmap should build on the repository as it exists today rather than +restart work that is already partially done. + +- **Package layout is already modular.** The main public library entry points are + `py_moodle.MoodleSession`, `py_moodle.MoodleSessionError`, + `py_moodle.Settings`, and `py_moodle.load_settings` + (`src/py_moodle/__init__.py`). The largest and most central modules are + `src/py_moodle/module.py`, `src/py_moodle/course.py`, `src/py_moodle/auth.py`, + and `src/py_moodle/compat.py`. +- **The CLI already has a clear command structure.** `src/py_moodle/cli/app.py` + registers Typer sub-apps for courses, categories, sections, modules, users, + admin, folders, pages, resources, URLs, and site info. +- **Tests already have an initial layer split.** `tests/unit/` contains the fast + smoke tests, while the rest of the suite is automatically marked as + integration and skipped unless `--integration` is used (`tests/conftest.py`). +- **CI already covers the supported Python versions.** The unit test matrix in + `.github/workflows/ci.yml` runs on Python 3.8 through 3.13, and integration + coverage spans Moodle 4.5.5 and 5.0.1. +- **Version-sensitive parsing already has a foothold.** + `src/py_moodle/compat.py` centralizes Moodle-version detection and selector + strategies, and session bootstrap wires compatibility state into login flows. +- **Documentation already has a contributor home.** MkDocs builds from `docs/`, + with `docs/development.md` acting as the main contributor guide. + +### Priority Risk Areas + +The highest-value next changes are the ones that improve safety and +maintainability without forcing broad refactors. + +1. **CLI output remains mostly human-first.** Commands generally expose a + `--json` flag, but there is no unified `--output` contract, no YAML mode, and + no field filtering for scripting workflows. +2. **Timeouts are hardcoded and inconsistent.** HTTP operations use several + different timeout values across modules instead of a centralized policy. +3. **Retry behavior is effectively absent.** Transient network failures are + mostly surfaced immediately. +4. **Logging and debugging are still ad hoc.** There is no project-wide logging + strategy, and the current debug behavior is not yet a redacted tracing + system. +5. **Dict-heavy boundaries are still concentrated in core flows.** + `course.py`, `module.py`, `upload.py`, and `draftfile.py` still exchange + large `dict[str, Any]` payloads and return values. +6. **The module architecture still has extension cost.** The repository already + supports several Moodle module types, but the registration and dispatch story + is still implicit. + +## Prioritized Execution Plan + +The sequence below starts with the smallest changes that reduce risk and unlock +later work. + +### Phase A: Lock in the baseline and smooth contributor workflows + +- **Subtask 1: Publish the technical baseline and roadmap plan.** + - Deliverable: this document. + - Why first: every later PR can reference shared assumptions instead of + repeating the same audit. +- **Subtask 2: Keep CI coverage aligned with the declared support policy.** + - This is effectively complete in CI today and should now be maintained rather + than redesigned. +- **Subtask 3: Finish documenting the test layers.** + - The unit vs. integration split is partially complete in the test layout and + should be finished by documenting how to run each layer and by clearly + identifying brittle HTML fixtures when they are introduced. +- **Subtask 25: Add HTML fixture regression coverage for fragile parsing.** + - This gives later compatibility and logging changes a safer landing zone. +- **Subtask 10: Expand task-oriented documentation recipes.** + - This can proceed in parallel with other foundation work once the baseline is + published. + +### Phase B: Improve CLI ergonomics before deeper refactors + +- **Subtask 4: Add structured CLI output formats (`table`, `json`, `yaml`).** + - This is a small, high-value improvement for automation users. + - It also creates a stable output contract needed for later field filtering. +- **Subtask 5: Add field filtering for machine-readable output.** + - Build directly on the structured output work. +- **Subtask 34: Improve CLI help text and option consistency.** + - Best done after the output contract is settled so wording stays stable. +- **Subtask 35: Standardize exit codes and batch summaries.** + - Build on the CLI help cleanup. +- **Subtask 36: Evaluate shell completion after the command surface is stable.** + +### Phase C: Improve reliability and observability + +- **Subtask 6: Improve exception wording and troubleshooting guidance.** + - This is the safest first slice because it does not need architecture + changes. +- **Subtask 7: Add a redacted debug mode for HTTP and parsing flows.** + - Build on the exception and wording cleanup so messages stay consistent. +- **Subtask 8: Centralize timeout configuration.** + - Introduce one config surface before adding retry behavior. +- **Subtask 9: Add safe retry support with backoff for read paths.** + - Limit the first pass to idempotent operations and clearly document scope. + +### Phase D: Improve API quality incrementally + +- **Subtask 11: Write the typed model migration note.** + - Decide the public boundary between `TypedDict`, dataclasses, and backward + compatibility adapters. +- **Subtask 12: Introduce typed `Course` and `Section` models.** + - These are the most central entities and touch both CLI and library usage. +- **Subtask 13: Add typed `User`, `Module`, and `UploadResult` models.** +- **Subtask 14: Ship `py.typed` only after the initial typed surface is real.** +- **Subtask 17: Design the ensure-style API semantics.** +- **Subtask 18: Implement `ensure_section`.** +- **Subtask 19: Implement `ensure_label` and `ensure_resource`.** +- **Subtask 20: Implement `ensure_folder` and `create_or_update_course`.** + +### Phase E: Continue compatibility and backend hardening + +- **Subtask 21: Keep the version-sensitive flow audit current.** + - The compatibility layer already exists, so the next work should focus on + documenting remaining fragile flows and filling test gaps. +- **Subtask 22: Keep version detection centralized during session initialization.** + - This is already partially implemented and should be extended, not + restarted. +- **Subtask 23: Document the compatibility-layer contribution model.** +- **Subtask 24: Migrate one additional fragile scraping flow at a time.** +- **Subtask 25: Expand HTML fixture-based regression tests alongside each migration.** +- **Subtask 26: Audit current webservice usage and backend candidates.** +- **Subtask 27: Design a hybrid backend selection strategy.** +- **Subtask 28: Implement one high-value operation with transparent backend + fallback.** + +### Phase F: Make extension and feature work cheaper + +- **Subtask 29: Audit the current module/resource architecture with concrete + extension pain points.** +- **Subtask 30: Design a minimal module registration system.** +- **Subtask 31: Migrate one or two existing modules through the new registration + path.** +- **Subtask 32: Pick the next high-value feature slice only after the extension + model is clearer.** +- **Subtask 33: Implement that single feature slice cleanly.** + +### Phase G: Finish with low-risk polish + +- **Subtask 15: Add dry-run support to the highest-risk mutating commands.** +- **Subtask 16: Add confirmations and `--yes` bypass for destructive + operations.** +- **Subtask 37: Review packaging and tooling coherence.** +- **Subtask 38: Apply the minimal packaging/tooling cleanup plan.** + +## Dependency-Aware Subtask Tree + +The tree below groups the checklist into workstreams and shows the main blocking +relationships. Items marked as **already landed** or **partially landed** should +be extended instead of reimplemented. + +- **Baseline and contributor workflow** + - **1. Technical baseline audit** → foundation for 3, 4, 6, 8, 10, 15, 21, 26, + 29, 34, 37 + - **2. CI Python matrix** (**already landed**) + - **3. Test layering** (**partially landed**) → supports 25 + - **10. Documentation recipes** +- **CLI ergonomics** + - **4. Structured output** → **5. Field filtering** + - **34. CLI help consistency** → **35. Exit codes and summaries**, + **36. Shell completion** +- **Reliability and observability** + - **6. Exception clarity** → **7. Debug mode** + - **8. Configurable timeouts** → **9. Safe retry support** +- **Typed API and idempotent operations** + - **11. Typed model migration note** + - **12. Course and Section models** + - **13. User, Module, UploadResult models** + - **14. `py.typed` packaging** + - **17. Ensure API design** + - **18. `ensure_section`** + - **19. `ensure_label` and `ensure_resource`** + - **20. `ensure_folder` and `create_or_update_course`** +- **Compatibility and backend strategy** + - **21. Moodle-version-sensitive audit** (**partially landed**) + - **22. Version detection during session init** (**partially landed**) + - **23. Compatibility-layer design** (**partially landed**) + - **24. First migrated fragile flow** (**partially landed**) + - **25. HTML fixture regression tests** + - **26. Webservice usage audit** + - **27. Backend selection strategy** + - **28. First hybrid backend implementation** +- **Module architecture and feature expansion** + - **29. Module/resource architecture audit** + - **30. Plugin-friendly registration system** + - **31. Migrate one or two existing modules** + - **32. Select next feature slice** + - **33. Implement selected capability** +- **Safety and polish** + - **15. Dry-run support** + - **16. Confirmation prompts and `--yes` bypass** + - **34. CLI help consistency** + - **35. Exit codes and summaries** + - **36. Shell completion** + - **37. Packaging/tooling coherence review** + - **38. Packaging/tooling cleanup** + - **10. Documentation recipes** + +## Proposed Sequence of Small PRs + +The PR sequence below favors narrow scope, clear validation, and reusable +foundations. + +| PR | Scope | Why this order | +| --- | --- | --- | +| 1 | Publish the technical baseline and roadmap execution plan | Gives every later PR a shared reference point | +| 2 | Finish documenting test layers and contributor test commands | Small change, reduces onboarding friction immediately | +| 3 | Add HTML fixture scaffolding for brittle parsing tests | Strengthens safety before more compatibility work | +| 4 | Introduce `--output table|json|yaml` for a first command family | High-value UX improvement with contained surface area | +| 5 | Add `--fields` support for machine-readable output | Small follow-up once the output contract exists | +| 6 | Standardize user-visible exception messages and troubleshooting docs | Improves day-to-day usability without architecture churn | +| 7 | Add redacted debug tracing for HTTP/session flows | Builds on clearer errors and pays off during later refactors | +| 8 | Centralize timeout configuration | Small reliability improvement with broad leverage | +| 9 | Add safe retries with backoff for read-only operations | Depends on timeout config and should stay narrowly scoped | +| 10 | Publish the typed model migration note | Design-first step before touching public return values | +| 11 | Introduce typed `Course` and `Section` models | First incremental API improvement on central entities | +| 12 | Add `py.typed` packaging after the typed surface is credible | Keeps typing support honest and low-risk | +| 13 | Add dry-run to one mutating command family | Safety improvement with a small blast radius | +| 14 | Add confirmation prompts and `--yes` bypass for destructive commands | Natural follow-up to dry-run | +| 15 | Publish ensure-style API semantics and implement `ensure_section` | Smallest idempotent automation slice | +| 16 | Extend ensure support to labels/resources, then folders/courses | Builds on the same result semantics incrementally | +| 17 | Audit remaining version-sensitive flows and migrate one more fragile path | Reuses the compatibility layer already in the repo | +| 18 | Audit webservice usage, then implement one hybrid backend path | Good medium-sized architecture proof point | +| 19 | Audit module architecture and migrate one or two module handlers | Defers cost until lower-risk wins are complete | +| 20 | Pick and implement one new feature slice | Best done after the extension and compatibility story improves | +| 21 | Final CLI polish: help text, exit codes, completion | Keeps user-facing polish aligned with the final command surface | +| 22 | Packaging/tooling coherence cleanup | Low-risk cleanup after product-facing changes settle | +| 23 | Expand documentation recipes for the stabilized workflows | Best when new behavior is no longer moving quickly | + +## Recommended Next PRs + +If the roadmap is executed strictly for impact-per-line changed, the best next +small PRs are: + +1. **Structured CLI output formats** +2. **Field filtering for machine-readable output** +3. **Exception clarity and troubleshooting** +4. **Centralized timeout configuration** +5. **Documentation recipes** + +That order keeps the early work mergeable, user-visible, and low-risk while +setting up the more architectural phases for success. diff --git a/mkdocs.yml b/mkdocs.yml index bdeab3e..b87b7e4 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -128,3 +128,4 @@ nav: - User: api/user.md - Examples: examples.md - Development: development.md + - Improvement Roadmap: roadmap-plan.md