[WIP] Fix critical recurring failure in GenAIScript model

[Copilot]

Thanks for assigning this issue to me. I'm starting to work on it and will keep this PR's description up to date as I form a plan and make progress.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[smoke-detector] 🚨 CRITICAL RECURRING: GenAIScript Invalid Model (gpt-4.1) - 3rd Occurrence</issue_title>
<issue_description># 🚨 CRITICAL RECURRING FAILURE - 3rd Occurrence

Summary

The Smoke GenAIScript workflow has FAILED AGAIN with the same root cause that was previously reported in #2157. This is the 3rd occurrence of this critical issue in less than 24 hours. Issue #2157 was closed as "not_planned" but the underlying configuration problem was never fixed, resulting in continued failures.

Failure Details

• Run: #18739169072
• Commit: 6b2c9e7 - "Remove 'defaults' section from main JSON schema (Remove "defaults" section from main JSON schema #2200)"
• Trigger: schedule (automated smoke test)
• Duration: 3.1 minutes
• Failed Job: detection (55 seconds)
• Workflow: Smoke GenAIScript

Recurrence Timeline

Occurrence	Run ID	Date	Status
1st	18727962258	2025-10-22 19:45 UTC	Issue #2157 created
2nd	18733557489	2025-10-23 00:19 UTC	#2157 closed as "not_planned"
3rd	18739169072 (current)	2025-10-23 06:07 UTC	This investigation

Root Cause (UNCHANGED)

The problem remains exactly the same as documented in #2157:

The GenAIScript configuration uses an invalid OpenAI model name:

# .github/workflows/shared/genaiscript.md:6
GH_AW_AGENT_MODEL_VERSION: "openai:gpt-4.1"

gpt-4.1 does not exist in OpenAI's model catalog. Valid models include:

• gpt-4o (recommended)
• gpt-4-turbo
• gpt-4
• gpt-3.5-turbo

Error Chain

1. GenAIScript attempts to use model openai:gpt-4.1
2. OpenAI API rejects the invalid model name
3. GenAIScript receives undefined/null response
4. GenAIScript crashes: TypeError: Cannot read properties of undefined (reading 'text')
5. Detection job fails with exit code 255

Stack Trace

2025-10-23T06:09:52.672Z genaiscript:error {
  name: 'TypeError',
  message: "Cannot read properties of undefined (reading 'text')",
  stack: "TypeError: Cannot read properties of undefined (reading 'text')\n" +
    '    at githubActionSetOutputs ((redacted))\n' +
    '    at async Command.runScriptWithExitCode ((redacted))'
}

Failed Jobs and Errors

Job Execution Summary

1. ✅ activation - succeeded (2s)
2. ✅ agent - succeeded (1.4m) - Agent completed successfully
3. ❌ detection - FAILED (55s) - Threat detection crashed
4. ✅ create_issue - succeeded (7s)
5. ⏭️ missing_tool - skipped

Impact Assessment

Severity: 🔴 CRITICAL

• All scheduled smoke tests for GenAIScript are failing
• Threat detection is NOT running (security implications)
• False confidence in system health due to skipped validations
• 3 consecutive failures in ~10 hours

Urgency: 🔴 IMMEDIATE

• Simple one-line configuration fix
• Blocking critical security and quality checks
• Issue will continue to recur daily via scheduled runs

Scope:

• Affects: All workflows using shared/genaiscript.md
• Frequency: Every scheduled smoke test run (multiple times per day)
• Duration: Ongoing since 2025-10-22 19:45 UTC (>10 hours)

Why This Needs Immediate Attention

1. Security Risk: Threat detection is disabled due to these failures
2. False Negatives: Team may assume smoke tests are passing when they're actually failing
3. Resource Waste: Every scheduled run consumes CI minutes while producing no value
4. Pattern Established: Without intervention, this will fail indefinitely
5. Simple Fix: One-line configuration change to resolve

Recommended Actions

🔴 IMMEDIATE - Fix Configuration (5 minutes)

Update .github/workflows/shared/genaiscript.md line 6:

- GH_AW_AGENT_MODEL_VERSION: "openai:gpt-4.1"
+ GH_AW_AGENT_MODEL_VERSION: "openai:gpt-4o"

🟡 SHORT-TERM - Prevent Recurrence

1. Add Model Validation - Create a pre-flight check that validates model names before execution
2. Update Documentation - Document valid model names in configuration files
3. Monitor Pattern - Track if this issue pattern appears in other workflows

🟢 LONG-TERM - Systemic Improvements

1. Schema Validation - Add JSON schema validation for workflow configurations
2. Better Error Messages - Work with GenAIScript team to improve error handling
3. Automated Alerts - Configure alerts for recurring failure patterns

Investigation Findings

Configuration Location

• File: .github/workflows/shared/genaiscript.md
• Line: 6
• Variable: GH_AW_AGENT_MODEL_VERSION
• Current Value:...

Fixes #2204

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.