Standardize step metadata catalog ownership to step packs

[Copilot]

Summary

Step metadata (RequiredCapabilities) is now owned and exported by step packs via Get-IdleStepMetadataCatalog. Core discovers loaded IdLE.Steps.* modules (including nested modules), merges their catalogs deterministically, and enforces ownership boundaries. Host metadata becomes supplement-only.

Motivation

Metadata ownership was unclear: workflows could define capabilities, host could override, and step packs had partial control. This caused ambiguity for implementers and workflow authors. Standardizing on step pack ownership eliminates configuration drift and makes capability requirements explicit and discoverable.

Type of Change

• Breaking change
• New feature
• Refactoring / internal improvement
• Documentation update

Changes

Core discovery and merge

• Resolve-IdleStepMetadataCatalog discovers loaded IdLE.Steps.* modules exporting Get-IdleStepMetadataCatalog
• Module discovery includes nested modules via Get-Module -All
• Merges catalogs deterministically (sorted by module name)
• Throws DuplicateStepTypeMetadata on collision (step pack vs step pack, host vs step pack)
• Refactored ScriptBlock validation to eliminate code duplication by reusing existing Assert-IdleNoScriptBlock function

Host metadata: supplement-only

• Providers.StepMetadata may only add metadata for step types not present in step pack catalogs
• Attempting to override step pack metadata → DuplicateStepTypeMetadata
• Use case: custom/host-defined step types only

Error separation

• MissingStepTypeMetadata: workflow references unknown step type (remediation: import step pack OR provide host metadata)
• DuplicateStepTypeMetadata: ownership conflict (step pack vs step pack, or host override attempt)

Step pack updates

• IdLE.Steps.DirectorySync: added Get-IdleStepMetadataCatalog exporting IdLE.Step.TriggerDirectorySync metadata
• IdLE.Steps.Common: removed IdLE.Step.TriggerDirectorySync (ownership transferred)
• Each step pack exports its own Get-IdleStepMetadataCatalog function; Core calls each module's function individually to avoid conflicts

Code quality improvements

• Eliminated duplicate Test-IdleMetadataForScriptBlocks function definitions
• Created shared Assert-IdleStepMetadataNoScriptBlock wrapper that reuses existing Assert-IdleNoScriptBlock
• Reduced code duplication while maintaining metadata-specific error messages

Documentation updates

• Streamlined documentation to describe only current behavior without historical references
• Removed all before/after comparisons and references to deprecated workflow-based capability definitions
• Documentation focuses exclusively on how the step metadata system works today

Breaking change

• Workflows must not define RequiredCapabilities or RequiresCapabilities on steps
• Capabilities are derived from step metadata catalogs during plan building

Example: step pack catalog

# IdLE.Steps.DirectorySync/Public/Get-IdleStepMetadataCatalog.ps1
function Get-IdleStepMetadataCatalog {
    $catalog = [hashtable]::new([System.StringComparer]::OrdinalIgnoreCase)
    
    $catalog['IdLE.Step.TriggerDirectorySync'] = @{
        RequiredCapabilities = @('IdLE.DirectorySync.Trigger', 'IdLE.DirectorySync.Status')
    }
    
    return $catalog
}

Example: workflow step

@{
  Name = 'Trigger sync'
  Type = 'IdLE.Step.TriggerDirectorySync'
  With = @{
    AuthSessionName = 'DirSync'
    PolicyType      = 'Delta'
    Provider        = 'DirectorySync'
  }
}
# Capabilities derived from IdLE.Steps.DirectorySync catalog

Testing

• Unit tests
• Manual testing

New test suite Resolve-IdleStepMetadataCatalog.Tests.ps1 covers:

• Discovery of loaded step packs (including nested modules)
• Deterministic merge from multiple packs
• Host supplement for new step types
• Duplicate detection (pack vs pack, host vs pack)
• Missing metadata error handling
• DirectorySync metadata ownership transfer

Updated New-IdlePlan.Capabilities.Tests.ps1 to reflect breaking change (host override → rejection).

Test fixture joiner-with-dirsync.psd1 documented to clarify it focuses on DirectorySync metadata resolution.

Test results: 240/240 passing, ScriptAnalyzer clean.

How to test & review

1. Import IdLE.Steps.DirectorySync and verify Get-IdleStepMetadataCatalog exports TriggerDirectorySync metadata
2. Attempt host override of IdLE.Step.DisableIdentity → expect DuplicateStepTypeMetadata
3. Reference unknown step type without metadata → expect MissingStepTypeMetadata with remediation text
4. Verify workflows using TriggerDirectorySync derive capabilities correctly (no workflow changes needed)

Checklist

• Code follows STYLEGUIDE.md
• Tests added or updated
• Documentation updated
• No UI/auth logic added to IdLE.Core
• Breaking changes documented and justified (pre-1.0 cleanup)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Step metadata catalogs owned by step packs</issue_title>
<issue_description>## Problem Statement

IdLE needs step metadata (e.g., RequiredCapabilities) to plan and execute workflows. Today, step metadata resolution is partially hard-coded and workflow authors may still encounter guidance to specify capabilities in workflow definitions. This creates:

• unclear ownership of step metadata
• inconsistent discovery/merge behavior for step packs
• avoidable ambiguity for implementers and workflow authors

This issue standardizes ownership and discovery so that step packs (IdLE.Steps.*) own step metadata and IdLE.Core discovers and merges their catalogs deterministically.

Proposed Solution

High-level behavior

• Step packs (IdLE.Steps.*) own step metadata for their step types.
• Step packs expose metadata via Get-IdleStepMetadataCatalog.
• IdLE.Core discovers loaded step packs and merges their catalogs deterministically.
• Host metadata (Providers.StepMetadata) remains supported, but supplement-only (only for new/host-defined step types; no overrides).
• Fail fast on duplicate step types across step packs (and between packs and host supplements).
• Ensure error messages are unambiguous and separated (duplicate vs missing).

Breaking change (pre-1.0)

Workflows/step definitions must not define RequiredCapabilities (or RequiresCapabilities).
Capabilities come from step metadata catalogs (step pack ownership).

Contract: Get-IdleStepMetadataCatalog

Each step pack MAY export Get-IdleStepMetadataCatalog:

• Returns a case-insensitive Hashtable
  • Key: StepType (string)
  • Value: metadata Hashtable with at least:
    • RequiredCapabilities: string or string[] (Core normalizes to string[])

Notes:

• StepType is the key. The value does not need a StepType property.
• Step packs must not emit duplicate keys within their own catalog.

Discovery rule (Core)

Core resolves step metadata catalogs only from loaded modules that:

1. Match module name pattern: IdLE.Steps.*, and
2. Export a function named Get-IdleStepMetadataCatalog

Determinism:

• Discovery order must be deterministic (e.g., module name ascending) before merging.

Merge rules

Step pack catalogs

• Core merges catalogs from discovered step packs.
• If the same StepType appears in more than one step pack catalog:
  • Fail fast with error: DuplicateStepTypeMetadata
  • Error message must include the StepType and the conflicting module names.

Host supplement (Providers.StepMetadata) — supplement-only

Providers.StepMetadata remains supported, but must not override step-pack-owned metadata:

• Host may provide metadata only for StepTypes that do not exist in the merged step pack catalog.
• If host provides metadata for a StepType that already exists from a step pack:
  • Fail fast with error: DuplicateStepTypeMetadata (source: Host vs StepPack)

Outcome:

• Final catalog = (merged step pack catalog) + (host supplement entries for new step types)

Error behavior (must be clearly separated)

1) Duplicate step type metadata

• Error id/type: DuplicateStepTypeMetadata
• Thrown during catalog resolution/merge (resolver), not during plan build.
• Message must mention:
  • the duplicate StepType
  • the involved sources (module names, and/or Host)

2) Missing step type metadata

• Error id/type: MissingStepTypeMetadata
• Thrown during plan building when a workflow references a StepType that is absent from the final catalog.
• Message must mention:
  • the missing StepType
  • remediation:
    • import/load the step pack that provides Get-IdleStepMetadataCatalog for that step type, OR
    • provide host Providers.StepMetadata (only for host-defined/custom step types)

Required code changes

IdLE.Core

• Update Resolve-IdleStepMetadataCatalog to:

  1. Discover loaded step packs (IdLE.Steps.*) exporting Get-IdleStepMetadataCatalog
  2. Merge returned catalogs deterministically
  3. Apply Providers.StepMetadata as supplement-only (no override)
  4. Throw DuplicateStepTypeMetadata on any duplicates (pack↔pack, host↔pack)

• Update plan building validation to throw MissingStepTypeMetadata (and ensure it is not conflated with duplicates).

Step packs

• IdLE.Steps.Common

  • Keep catalog export, but ensure it only contains step types owned by this pack.
  • Remove any step types owned by other packs.

• IdLE.Steps.DirectorySync

  • Add Get-IdleStepMetadataCatalog and move ownership of:
    • IdLE.Step.TriggerDirectorySync
  • Remove IdLE.Step.TriggerDirectorySync metadata entry from IdLE.Steps.Common.

Docs & examples updates (in scope)

• Remove all workflow authoring guidance that uses RequiresCapabilities / RequiredCapabilities in workflow definitions...

• Fixes Step metadata catalogs owned by step packs #99

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b6cf8eb3e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

Pull request overview

This PR standardizes step metadata catalog ownership to step packs, eliminating ambiguity about where step capabilities are defined. Step metadata (RequiredCapabilities) is now owned and exported by step packs via Get-IdleStepMetadataCatalog, with Core discovering loaded IdLE.Steps.* modules and merging their catalogs deterministically. Host metadata becomes supplement-only for custom step types.

Changes:

• Core discovers and merges step pack metadata catalogs with duplicate detection
• Host metadata (Providers.StepMetadata) restricted to supplement-only (no overrides)
• Error separation: MissingStepTypeMetadata vs DuplicateStepTypeMetadata
• DirectorySync step pack now owns TriggerDirectorySync metadata (moved from Common)
• Breaking change: workflows must not define RequiredCapabilities

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/fixtures/workflows/joiner-with-dirsync.psd1	New test fixture for DirectorySync step without capability declarations
tests/Resolve-IdleStepMetadataCatalog.Tests.ps1	Comprehensive test suite for catalog discovery, merge, duplicate detection, and host supplement validation
tests/New-IdlePlan.Capabilities.Tests.ps1	Updated to reflect breaking change - host override now rejected instead of allowed
src/IdLE.Steps.DirectorySync/Public/Get-IdleStepMetadataCatalog.ps1	New function exporting TriggerDirectorySync metadata
src/IdLE.Steps.DirectorySync/IdLE.Steps.DirectorySync.psm1	Added Get-IdleStepMetadataCatalog to module exports
src/IdLE.Steps.DirectorySync/IdLE.Steps.DirectorySync.psd1	Added Get-IdleStepMetadataCatalog to FunctionsToExport
src/IdLE.Steps.Common/Public/Get-IdleStepMetadataCatalog.ps1	Removed TriggerDirectorySync metadata (ownership transferred to DirectorySync pack)
src/IdLE.Core/Public/New-IdlePlanObject.ps1	Updated error message for missing metadata with clear remediation guidance
src/IdLE.Core/Private/Resolve-IdleStepMetadataCatalog.ps1	Implements step pack discovery, deterministic merge, duplicate detection, and supplement-only host metadata
docs/reference/providers/provider-ad.md	Updated to reflect metadata ownership by step packs
docs/advanced/provider-capabilities.md	Comprehensive documentation of new ownership model, discovery process, and error handling