feat(config): add optional metadata dict to workflow definition

[PolyphonyRequiem]

Problem

External tooling (dashboards, trackers, CI integrations) that consumes conductor event logs currently has no structured way to know what kind of workflow is running or how to enrich its data. The dashboard, for example, hardcodes ADO URL templates, twig SQLite DB paths, and work-item extraction logic — all tightly coupled to specific workflow implementations.

This coupling means:

• Non-twig workflows get wrong ADO links or no enrichment at all
• Adding support for a new tracker (GitHub Issues, Jira) requires dashboard code changes
• Workflow authors can't declare their own integration metadata

Proposal

Add an optional metadata dict to the workflow section of the YAML config:

workflow:
  name: twig-sdlc
  entry_point: intake
  metadata:
    tracker: ado
    project_url: https://dev.azure.com/org/Project
    work_item_id_agent: intake
    work_item_id_field: epic_id

The metadata is:

• Passed through to the workflow_started event as-is
• Available to any event log consumer without parsing YAML source
• Completely optional — existing workflows emit "metadata": {}

Use Cases

Dashboard enrichment plugins

A dashboard can dispatch to enricher modules based on metadata.tracker:

• tracker: ado → look up work item hierarchy from twig SQLite DB, generate ADO links
• tracker: github → fetch issue status from GitHub API
• tracker: jira → query Jira REST API for ticket details

Work item ID extraction

Instead of hardcoding "look for epic_id in the intake agent output", the dashboard reads metadata.work_item_id_agent and metadata.work_item_id_field to generically extract the tracking ID from any agent's structured output.

Custom workflow tagging

Teams can add arbitrary tags for filtering, grouping, or routing:

metadata:
  team: platform
  environment: staging
  priority: high

Implementation

Two changes (~8 lines total):

1. Add metadata: dict[str, Any] = Field(default_factory=dict) to WorkflowDef in schema.py
2. Include "metadata": self.config.workflow.metadata in the workflow_started event emission in workflow.py

No breaking changes. No migration needed. Default is empty dict.

[PolyphonyRequiem]

Further context here, I'm working on an extendable dashboard currently for conductor to show all working and terminated conductor workflows with some statuses and tools to help me keep track of work in flight.

I want to be able to enrich the dashboard in various ways, but want to keep it somewhat agnostic to the conductor workflows themselves, so the metadata can be used for the purpose.

[PolyphonyRequiem]

Design decision: CLI --metadata format

We chose repeated flags (-m key=value -m key2=value2) over semicolon-delimited (-m "key=value;key2=value2") for consistency with the existing --input flag, which uses the same pattern.

Considered alternatives:

• Semicolon-delimited: more compact for many keys, but introduces a new parsing convention and makes values with semicolons in them ambiguous
• JSON blob (-m '{"key":"value"}'): powerful but verbose for simple key=value pairs

Rationale: Metadata is typically declared in the workflow YAML. The CLI flag is for occasional per-invocation overrides (e.g. --metadata work_item_id=1814), not bulk key entry. Consistency with --input reduces cognitive load.

If semicolon support becomes needed later, it can be added without breaking the repeated-flag syntax.

[PolyphonyRequiem]

Update: System metadata (system dict) added to PR #107

In addition to user-controlled metadata, we've added auto-injected system metadata to the workflow_started event, checkpoint files, and the dashboard /api/info endpoint. This captures runtime diagnostics that are lost if the process crashes or exits.

Fields (always present)

Field	Example
pid	12345
platform	win32, linux, darwin
python_version	3.12.4
conductor_version