Step metadata catalogs owned by step packs

[blindzero]

Problem Statement

IdLE needs step metadata (e.g., RequiredCapabilities) to plan and execute workflows. Today, step metadata resolution is partially hard-coded and workflow authors may still encounter guidance to specify capabilities in workflow definitions. This creates:

• unclear ownership of step metadata
• inconsistent discovery/merge behavior for step packs
• avoidable ambiguity for implementers and workflow authors

This issue standardizes ownership and discovery so that step packs (IdLE.Steps.*) own step metadata and IdLE.Core discovers and merges their catalogs deterministically.

Proposed Solution

High-level behavior

• Step packs (IdLE.Steps.*) own step metadata for their step types.
• Step packs expose metadata via Get-IdleStepMetadataCatalog.
• IdLE.Core discovers loaded step packs and merges their catalogs deterministically.
• Host metadata (Providers.StepMetadata) remains supported, but supplement-only (only for new/host-defined step types; no overrides).
• Fail fast on duplicate step types across step packs (and between packs and host supplements).
• Ensure error messages are unambiguous and separated (duplicate vs missing).

Breaking change (pre-1.0)

Workflows/step definitions must not define RequiredCapabilities (or RequiresCapabilities).
Capabilities come from step metadata catalogs (step pack ownership).

Contract: Get-IdleStepMetadataCatalog

Each step pack MAY export Get-IdleStepMetadataCatalog:

• Returns a case-insensitive Hashtable
  • Key: StepType (string)
  • Value: metadata Hashtable with at least:
    • RequiredCapabilities: string or string[] (Core normalizes to string[])

Notes:

• StepType is the key. The value does not need a StepType property.
• Step packs must not emit duplicate keys within their own catalog.

Discovery rule (Core)

Core resolves step metadata catalogs only from loaded modules that:

1. Match module name pattern: IdLE.Steps.*, and
2. Export a function named Get-IdleStepMetadataCatalog

Determinism:

• Discovery order must be deterministic (e.g., module name ascending) before merging.

Merge rules

Step pack catalogs

• Core merges catalogs from discovered step packs.
• If the same StepType appears in more than one step pack catalog:
  • Fail fast with error: DuplicateStepTypeMetadata
  • Error message must include the StepType and the conflicting module names.

Host supplement (Providers.StepMetadata) — supplement-only

Providers.StepMetadata remains supported, but must not override step-pack-owned metadata:

• Host may provide metadata only for StepTypes that do not exist in the merged step pack catalog.
• If host provides metadata for a StepType that already exists from a step pack:
  • Fail fast with error: DuplicateStepTypeMetadata (source: Host vs StepPack)

Outcome:

• Final catalog = (merged step pack catalog) + (host supplement entries for new step types)

Error behavior (must be clearly separated)

1) Duplicate step type metadata

• Error id/type: DuplicateStepTypeMetadata
• Thrown during catalog resolution/merge (resolver), not during plan build.
• Message must mention:
  • the duplicate StepType
  • the involved sources (module names, and/or Host)

2) Missing step type metadata

• Error id/type: MissingStepTypeMetadata
• Thrown during plan building when a workflow references a StepType that is absent from the final catalog.
• Message must mention:
  • the missing StepType
  • remediation:
    • import/load the step pack that provides Get-IdleStepMetadataCatalog for that step type, OR
    • provide host Providers.StepMetadata (only for host-defined/custom step types)

Required code changes

IdLE.Core

• Update Resolve-IdleStepMetadataCatalog to:

  1. Discover loaded step packs (IdLE.Steps.*) exporting Get-IdleStepMetadataCatalog
  2. Merge returned catalogs deterministically
  3. Apply Providers.StepMetadata as supplement-only (no override)
  4. Throw DuplicateStepTypeMetadata on any duplicates (pack↔pack, host↔pack)

• Update plan building validation to throw MissingStepTypeMetadata (and ensure it is not conflated with duplicates).

Step packs

• IdLE.Steps.Common

  • Keep catalog export, but ensure it only contains step types owned by this pack.
  • Remove any step types owned by other packs.

• IdLE.Steps.DirectorySync

  • Add Get-IdleStepMetadataCatalog and move ownership of:
    • IdLE.Step.TriggerDirectorySync
  • Remove IdLE.Step.TriggerDirectorySync metadata entry from IdLE.Steps.Common.

Docs & examples updates (in scope)

• Remove all workflow authoring guidance that uses RequiresCapabilities / RequiredCapabilities in workflow definitions.
• Document:
  • step pack ownership of metadata
  • Get-IdleStepMetadataCatalog contract
  • discovery rule (loaded IdLE.Steps.* modules)
  • host supplement behavior (new step types only)
  • error cases (DuplicateStepTypeMetadata, MissingStepTypeMetadata)

Update at least:

• docs/concept.md, docs/workflows.md, docs/steps.md, plus any examples referencing capabilities in workflows.

Tests (in scope)

Add/adjust tests to cover:

1. Discovery

   • only loaded IdLE.Steps.* modules exporting Get-IdleStepMetadataCatalog are considered

2. Merge + determinism

   • multiple step pack catalogs merge successfully (order independent, deterministic outcome)

3. Duplicates

   • duplicate step type across two step packs → DuplicateStepTypeMetadata
   • host provides step type already present in step pack catalog → DuplicateStepTypeMetadata

4. Host supplements

   • host can provide metadata for a step type not present in packs and it appears in final catalog

5. Missing metadata

   • workflow references unknown step type with no metadata → MissingStepTypeMetadata

6. Ownership correction

   • IdLE.Step.TriggerDirectorySync metadata is resolved from IdLE.Steps.DirectorySync, not from IdLE.Steps.Common

Acceptance Criteria

• Workflows do not accept/require RequiredCapabilities in step definitions (breaking change applied and documented).
• Core resolves step metadata catalogs by discovering loaded step packs (IdLE.Steps.*) exporting Get-IdleStepMetadataCatalog.
• Duplicate step types across packs or between host and pack result in DuplicateStepTypeMetadata during resolver execution.
• Missing step metadata results in MissingStepTypeMetadata during plan build with clear remediation text.
• IdLE.Step.TriggerDirectorySync metadata is owned by IdLE.Steps.DirectorySync via its catalog.
• Docs/examples/tests updated accordingly; Pester green; ScriptAnalyzer green.

Alternatives Considered

1. Allow host overrides of step-pack metadata
   Rejected: overrides would weaken ownership, make behavior environment-dependent, and complicate debugging. Host metadata is kept as supplement-only.

2. Discover catalogs from any loaded module exporting Get-IdleStepMetadataCatalog
   Rejected: too permissive and risks accidental collisions. Discovery is limited to IdLE.Steps.*.

3. Auto-import step packs based on workflow contents
   Rejected (non-goal): engine should not dynamically import modules based on workflow; only already loaded packs are considered.

Impact

• Breaking change (pre-1.0): workflow definitions must not specify capabilities; docs/examples must be updated.
• Step pack authors must ensure their packs export Get-IdleStepMetadataCatalog for owned step types.
• Hosts may still define custom step types, but must provide metadata only for step types not covered by loaded step packs.
• Improved determinism and clearer errors reduce ambiguity and implementation risk.

Additional Context

Implementation guidance:

• Prefer case-insensitive hashtables for catalogs ([System.StringComparer]::OrdinalIgnoreCase) to avoid casing bugs.
• Normalize RequiredCapabilities to a string array at the resolver boundary.
• Keep errors structured and consistent with existing project error patterns (include error id/type and clear message).