Subtasks breakdown for the `python-moodle` improvement roadmap

[erseco]

This checklist breaks the roadmap into smaller, mergeable subtasks that can be assigned to an agentic AI one by one.

Each subtask should result in a focused PR with code, tests, and documentation updates where relevant.

---

Phase 1 — Foundation and quick wins

1. Audit current repository structure and produce a technical baseline

• Inspect the current package layout, public API, CLI commands, tests, CI, and documentation
• Identify the most fragile modules and the most central public entry points
• Identify where dict-based models are used most heavily
• Identify all Moodle-version-sensitive scraping and parsing paths
• Identify current timeout, retry, logging, and exception behavior
• Identify current CLI output modes and command ergonomics
• Produce a short architecture note to guide the rest of the roadmap

Acceptance criteria

• A short technical baseline comment or document exists
• The baseline identifies priority areas, risks, and dependencies
• The rest of the subtasks can reference this baseline

---

2. Expand CI to test all supported Python versions

• Audit the Python versions declared in project metadata
• Update GitHub Actions to test all officially supported Python versions
• Keep CI readable and maintainable
• Ensure test jobs still pass in the updated matrix
• Document the tested Python support policy

Acceptance criteria

• CI covers all officially supported Python versions
• Documentation matches actual CI coverage

Dependencies

• None

---

3. Categorize the current test suite into unit, integration, and regression layers

• Audit the existing tests
• Separate fast unit tests from slower integration tests where practical
• Mark or reorganize brittle HTML/UI-dependent tests
• Document the test categories for contributors

Acceptance criteria

• Tests are easier to understand and run selectively
• Contributors can tell which tests are fast and which require a Moodle environment

Dependencies

• Subtask 1

---

4. Add structured CLI output formats

• Audit current CLI rendering and data-return flow

• Add --output support with at least:

  • table
  • json
  • yaml

• Keep human-friendly table output as the default

• Ensure both list and detail commands return stable machine-readable structures

• Add tests for JSON and YAML output

• Update CLI documentation with examples

Acceptance criteria

• JSON output is valid and stable
• YAML output is valid and stable
• Current human-oriented output still works

Dependencies

• Subtask 1

---

5. Add basic field filtering for machine-readable CLI output

• Add support for selecting specific output fields where practical, such as --fields id,fullname
• Ensure filtered output works in JSON/YAML modes
• Add tests for field filtering
• Document usage examples

Acceptance criteria

• Users can reduce output shape for scripting
• Output remains consistent and predictable

Dependencies

• Subtask 4

---

6. Improve logging and exception clarity

• Audit current logging behavior
• Audit exception hierarchy and messages
• Improve exception messages to include actionable context
• Standardize error wording across the project
• Add tests where exception behavior is user-visible
• Update troubleshooting docs

Acceptance criteria

• Common failures are easier to understand
• Errors provide enough context without leaking secrets

Dependencies

• Subtask 1

---

7. Add debug mode for HTTP/session operations

• Add a debug flag or configuration for session-level tracing
• Log request targets, status codes, redirect behavior, and relevant parsing steps
• Ensure secrets, tokens, and credentials are redacted
• Optionally save failure artifacts such as response HTML for debugging
• Document how to use debug mode safely

Acceptance criteria

• Debug mode helps diagnose login and scraping failures
• Sensitive data is redacted

Dependencies

• Subtask 6

---

8. Add configurable timeouts

• Audit current hardcoded timeout values
• Add session-level or client-level timeout configuration
• Ensure all relevant HTTP operations use the configured timeout
• Add tests for timeout configuration behavior
• Document the new configuration option

Acceptance criteria

• Timeouts are configurable from a central place
• Default behavior remains sensible

Dependencies

• Subtask 1

---

9. Add safe retry support with backoff

• Identify read operations that are safe to retry
• Add retry logic with backoff for safe operations
• Avoid blind retries for unsafe writes unless explicitly safe and documented
• Handle transient 5xx, connection failures, and timeouts more gracefully
• Add tests for retry behavior
• Update documentation

Acceptance criteria

• Network resilience improves without causing duplicate writes
• Retry behavior is predictable and documented

Dependencies

• Subtask 8

---

10. Improve documentation with real-world recipes

• Audit current documentation structure

• Add practical guides for:

  • username/password authentication
  • CAS authentication
  • course creation
  • section management
  • file upload
  • common resources/activities
  • automation/CI use cases

• Add a troubleshooting section

• Fix outdated or inconsistent details in the docs

Acceptance criteria

• Documentation is more task-oriented
• Common workflows are easier to discover and follow

Dependencies

• Subtask 1

---

Phase 2 — Safety and API quality

11. Audit dict-heavy return values and define a typed model migration plan

• Inspect the public API and internal flows for common dict-based structures
• Identify the highest-value entities to type first
• Decide whether to use dataclasses, TypedDict, or a hybrid approach
• Write a short migration note before implementation

Acceptance criteria

• There is a clear plan for introducing typed models incrementally
• Priority entities and boundaries are defined

Dependencies

• Subtask 1

---

12. Introduce typed models for Course and Section

• Add explicit typed structures for Course
• Add explicit typed structures for Section
• Refactor a limited slice of the codebase to use them
• Preserve backward compatibility where practical
• Add tests
• Update docs/examples

Acceptance criteria

• Course and Section have explicit typed representations
• Existing behavior still works or migration notes are documented

Dependencies

• Subtask 11

---

13. Introduce typed models for User, Module, and UploadResult

• Add explicit typed structures for User
• Add explicit typed structures for Module
• Add explicit typed structures for UploadResult
• Refactor affected code to use them consistently
• Add tests
• Update docs/examples

Acceptance criteria

• Core entities have explicit typed representations
• Type hints are improved across affected modules

Dependencies

• Subtask 12

---

14. Add py.typed and expose type information to users

• Decide whether the project should ship type information as part of the package
• Add py.typed if appropriate
• Ensure packaging includes the marker correctly
• Document typing support for users

Acceptance criteria

• Type-aware consumers can benefit from packaged type information
• Packaging is correct and documented

Dependencies

• Subtask 12
• Subtask 13

---

15. Add dry-run support to high-impact CLI commands

• Identify commands that create, update, or delete Moodle resources
• Add --dry-run to a first safe subset of commands
• Ensure dry-run reports intended actions without changing Moodle
• Add tests
• Update docs

Acceptance criteria

• Dry-run never performs real changes
• Dry-run output is clear and useful

Dependencies

• Subtask 1

---

16. Add confirmation prompts and --yes bypass for risky CLI actions

• Identify risky or destructive commands
• Add confirmation prompts where appropriate
• Add --yes or equivalent bypass for automation
• Keep non-interactive usage practical
• Add tests
• Update docs

Acceptance criteria

• Risky commands are safer by default
• Automation remains possible through explicit bypass flags

Dependencies

• Subtask 15

---

17. Design the idempotent “ensure” API

• Audit current create/update workflows

• Define a consistent API for ensure-style operations

• Decide result semantics such as:

  • created
  • updated
  • unchanged

• Write a small design note for the first implementation slice

Acceptance criteria

• There is a clear and consistent ensure API design
• It is ready to implement incrementally

Dependencies

• Subtask 11

---

18. Implement ensure_section

• Add logic to detect existing sections
• Create when missing
• Update only when needed
• Return explicit result status
• Add tests for create/update/no-op cases
• Update docs

Acceptance criteria

• Re-running the same section operation does not create duplicates
• No-op behavior is explicit

Dependencies

• Subtask 17

---

19. Implement ensure_label and ensure_resource

• Add ensure logic for labels
• Add ensure logic for resources
• Detect when update is needed versus no-op
• Return explicit result objects
• Add tests
• Update docs

Acceptance criteria

• Label/resource ensure operations are idempotent
• Repeated runs do not create duplicates

Dependencies

• Subtask 17

---

20. Implement ensure_folder and create_or_update_course

• Add ensure logic for folders
• Add create-or-update logic for courses
• Ensure duplicate creation is avoided
• Add tests for create/update/no-op cases
• Update docs

Acceptance criteria

• Folder and course operations are safer and more automation-friendly
• Status reporting is clear

Dependencies

• Subtask 17

---

Phase 3 — Compatibility and architecture

21. Audit Moodle-version-sensitive code paths

• Identify code that depends on selectors, field names, HTML layout, or UI flow assumptions
• Group those paths by Moodle area and risk level
• Document which flows are most fragile

Acceptance criteria

• There is a clear list of version-sensitive flows to migrate first

Dependencies

• Subtask 1

---

22. Detect Moodle version during session initialization

• Identify reliable ways to determine Moodle version
• Store version information in session or client state
• Add tests where practical
• Document how version detection works

Acceptance criteria

• Version information is available centrally after login/session setup

Dependencies

• Subtask 21

---

23. Design the compatibility layer for Moodle-specific behavior

• Decide on the abstraction style:

  • selector maps
  • strategy classes
  • feature probes
  • hybrid approach

• Keep the design small and incremental

• Document how new compatibility rules should be added

Acceptance criteria

• There is a clear compatibility abstraction ready for first adoption

Dependencies

• Subtask 22

---

24. Migrate one fragile scraping flow to the compatibility layer

• Choose one high-value brittle flow
• Replace hardcoded assumptions with compatibility abstractions
• Add regression tests
• Update docs or contributor notes

Acceptance criteria

• At least one fragile workflow is handled through the compatibility layer
• The pattern is reusable for later migrations

Dependencies

• Subtask 23

---

25. Add HTML fixture-based regression tests for brittle parsing logic

• Capture representative HTML fixtures from current supported flows
• Add fixture-based parser tests
• Add regression coverage for brittle selectors/field extraction
• Keep fixtures readable and maintainable

Acceptance criteria

• Fragile parsing logic has stronger regression protection
• Tests fail clearly when HTML assumptions break

Dependencies

• Subtask 3
• Subtask 21

---

26. Audit current use of official Moodle web services

• Identify where web services are already used
• Identify operations that could prefer web services when available
• Document current and potential backend split

Acceptance criteria

• A backend capability map exists
• Candidates for hybrid execution are clearly identified

Dependencies

• Subtask 1

---

27. Design backend selection strategy

• Define when to use web services
• Define when to fall back to session automation
• Decide how to expose backend choice to users
• Keep the public API consistent

Acceptance criteria

• A clear hybrid backend strategy exists
• It can be implemented incrementally

Dependencies

• Subtask 26

---

28. Implement hybrid backend for one high-value operation

• Choose one operation with clear web service and scraping alternatives
• Implement backend selection
• Add tests for both paths where feasible
• Document configuration and behavior

Acceptance criteria

• One operation can transparently use either backend
• Fallback behavior is reliable

Dependencies

• Subtask 27

---

29. Audit current module/resource architecture

• Inspect how supported Moodle module types are implemented
• Identify common patterns and duplicated behavior
• Identify extension pain points

Acceptance criteria

• There is a clear picture of current module architecture and extension costs

Dependencies

• Subtask 1

---

30. Design a plugin-friendly module registration system

• Define a registration/dispatch model for module handlers
• Keep the initial design minimal
• Document how future modules should plug in
• Ensure the design supports existing modules without large breakage

Acceptance criteria

• A practical extension model exists
• It is ready for incremental adoption

Dependencies

• Subtask 29

---

31. Migrate one or two existing module types to the new plugin architecture

• Refactor one or two current module types to use the new module registration flow
• Add tests
• Update contributor docs

Acceptance criteria

• The plugin architecture is proven on real module types
• Existing behavior keeps working

Dependencies

• Subtask 30

---

Phase 4 — Coverage and polish

32. Audit missing high-value Moodle capabilities and choose the next feature slice

• Review current functional coverage

• Evaluate next features by value and implementation risk

• Prioritize candidates such as:

  • quiz
  • forum
  • enrolments
  • groups
  • groupings
  • completion settings
  • availability conditions

• Select one realistic feature slice for first implementation

Acceptance criteria

• One high-value feature slice is selected with rationale and scope

Dependencies

• Subtask 1

---

33. Implement one new high-value Moodle capability

• Implement the selected feature slice
• Follow existing or improved architectural conventions
• Add tests
• Update docs

Acceptance criteria

• One substantial new capability is added cleanly and documented

Dependencies

• Subtask 32

---

34. Improve CLI help text and option consistency

• Audit command names, help text, argument wording, and option naming
• Improve consistency across commands
• Reduce ambiguity in help messages
• Add tests where practical
• Update docs/examples

Acceptance criteria

• CLI help is clearer and more consistent
• Users can discover commands more easily

Dependencies

• Subtask 1

---

35. Add consistent CLI exit codes and summary output

• Audit current CLI exit behavior
• Standardize success/failure exit codes
• Improve command summaries for batch operations
• Add tests
• Document any guarantees

Acceptance criteria

• Exit behavior is more predictable for scripts and CI
• Batch operations provide useful summaries

Dependencies

• Subtask 34

---

36. Add shell completion support if the CLI framework allows it cleanly

• Evaluate feasibility with the current CLI stack
• Add shell completion support if practical
• Document installation/usage

Acceptance criteria

• Shell completion is available without introducing unnecessary complexity

Dependencies

• Subtask 34

---

37. Review packaging and tooling coherence

• Audit pyproject.toml, linting, formatting, typing, release config, and package metadata
• Identify unnecessary duplication or friction
• Propose a minimal cleanup plan

Acceptance criteria

• Tooling improvement opportunities are documented clearly
• Scope is kept controlled

Dependencies

• Subtask 1

---

38. Implement packaging/tooling cleanup

• Simplify tooling where useful
• Clarify dependency ranges if needed
• Improve release or package metadata
• Keep the change low-risk
• Update contributor docs if necessary

Acceptance criteria

• Tooling is more coherent and easier to maintain
• Packaging metadata is clearer

Dependencies

• Subtask 37

---

Cross-cutting requirements for every implementation subtask

• Add or update tests
• Update documentation if public behavior changes
• Preserve backward compatibility where practical
• Add migration notes when compatibility cannot be preserved fully
• Keep PRs focused and mergeable
• Avoid bundling unrelated changes together

---

Suggested dependency notes

Good first subtasks

• Subtask 2 — Expand CI to test all supported Python versions
• Subtask 4 — Add structured CLI output formats
• Subtask 6 — Improve logging and exception clarity
• Subtask 8 — Add configurable timeouts
• Subtask 10 — Improve documentation with real-world recipes

Best foundation before deeper architecture changes

• Subtask 1 — Technical baseline audit
• Subtask 3 — Test categorization
• Subtask 11 — Typed model migration plan
• Subtask 21 — Audit Moodle-version-sensitive paths
• Subtask 26 — Audit use of official Moodle web services
• Subtask 29 — Audit current module/resource architecture

Architectural subtasks that should not start too early

• Subtask 23 — Design compatibility layer
• Subtask 27 — Design backend selection strategy
• Subtask 30 — Design plugin-friendly module registration system

---

Suggested PR slicing

Small PRs

• CI matrix update
• CLI JSON/YAML output
• timeout configuration
• logging/error cleanup
• docs recipes
• dry-run for one command family

Medium PRs

• first typed models
• ensure API first slice
• first compatibility-layer migration
• first hybrid backend operation
• first plugin-architecture migration

Larger but still controlled PRs

• new Moodle feature slice
• module architecture introduction
• broader compatibility-layer rollout

---

Final instruction for the assigned AI

Please convert this checklist into:

1. a prioritized execution plan,
2. a dependency-aware subtask tree,
3. and a proposed sequence of small PRs.

Do not try to implement everything at once. Start with the smallest high-value changes that reduce risk and improve maintainability.