Skip to content

perf: eliminate redundant YAML parsing in import field extraction#22491

Merged
pelikhan merged 4 commits intomainfrom
copilot/fix-performance-regression-compile-complex-workflo
Mar 23, 2026
Merged

perf: eliminate redundant YAML parsing in import field extraction#22491
pelikhan merged 4 commits intomainfrom
copilot/fix-performance-regression-compile-complex-workflo

Conversation

Copy link
Contributor

Copilot AI commented Mar 23, 2026

extractAllImportFields was calling ExtractFrontmatterFromContent (~YAML parse) ~20 times per imported file — once per field. Since engine: copilot injects a builtin file as an implicit import, this hit every copilot workflow compilation.

Changes

  • pkg/parser/import_field_extractor.go — parse frontmatter once at the top of extractAllImportFields, then pass the map[string]any to all downstream extractors; inline the extractOnGitHubToken/App/TopLevel helpers that existed solely to wrap a re-parse

  • pkg/parser/content_extractor.go — add map-based extraction helpers (extractFieldJSONFromMap, extractYAMLFieldFromMap, extractOnSectionFieldFromMap, extractOnSectionAnyFieldFromMap); simplify extractFrontmatterField to delegate to extractFieldJSONFromMap; remove 5 now-dead content-string helpers (extractStepsFromContent, extractServicesFromContent, extractPostStepsFromContent, extractOnSectionField, extractOnSectionAnyField)

Result

BenchmarkCompileComplexWorkflow: −18% latency, −27% allocations

Before After
ns/op 5,621,660 4,612,102
allocs/op 33,933 24,751
B/op 2,962,421 2,479,944

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw @v1.1.3/cpu/cpu.rev-parse 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/compile /usr/bin/git _.a FBrfd97FY ache/go/1.25.0/x--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw git k/_temp/ghcca-no-json git 0/x6�� --show-toplevel git 64/bin/node --show-toplevel git 0/x64/bin/node go (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GO111MODULE 64/bin/go ache/node/24.14.0/x64/bin/node 3433�� --show-toplevel go /usr/bin/git ddnS/ohC_c_O6YVXgit GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name go1.25.0 -c=4 -nolocalimports -importcfg /tmp/go-build612262074/b252/importcfg -pack /home/REDACTED/go/pkg/mod/go.yaml.in/yaml/v3@v3.0.4/apic.go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha remove origin /usr/bin/git -json GO111MODULE x_amd64/compile git rev-�� --git-dir 9fRgrZr/aHz0DOVU-trimpath /usr/bin/git -json GO111MODULE x_amd64/compile git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha GOSUMDB GOWORK 64/bin/go json' --ignore-pgo git /usr/bin/git git ache�� --show-toplevel git 64/bin/go rkflow/js/**/*.jgo git erignore git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE /usr/bin/git GOINSECURE GOMOD GOMODCACHE git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha sistency_GoAndJavaScript2182455461/001/test-simple-frontmatter.md -trimpath ache/node/24.14.0/x64/bin/node -p crypto/rc4 -lang=go1.25 /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile t-24�� bility_SameInputSameOutput4000842859/001/stability-test.md -trimpath /usr/bin/git -p github.com/githurev-parse -lang=go1.25 git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha --show-toplevel git /usr/bin/git --show-toplevel 64/pkg/tool/linu-c /usr/bin/git git rev-�� --show-toplevel git k/gh-aw/gh-aw/node_modules/.bin/sh --show-toplevel 64/pkg/tool/linurev-parse /usr/bin/git git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha _.a /cpu/byteorder.go 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile env _.a 7t-OTOrXu ache/go/1.25.0/x64/pkg/tool/linux_amd64/compile GOINSECURE o8601 GOMODCACHE ache/go/1.25.0/x64/pkg/tool/linu.github/workflows/test.md (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel x_amd64/compile /usr/bin/git */*.ts' '**/*.jsgit GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git json ]; then \ git GO111MODULE 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel ache/go/1.25.0/x64/pkg/tool/linux_amd64/asm /usr/bin/git -json om/segmentio/asmrev-parse ache/go/1.25.0/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.0/x64/pkg/tool/linux_amd64/compile n-dir/node 262074/b221/_pkggit GOPROXY x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha th .prettierignore --log-level=error -buildtags 64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert -d/pprof 64/pkg/tool/linux_amd64/vet ode_�� --noprofile -d/pprof ash (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha -bool -buildtags 64/bin/bash -errorsas -ifaceassert -nilfunc bash ode_�� --noprofile -tests che/go-build/2a/2ae29706cbbff57cc55097dca44088bc-importcfg (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha -bool -buildtags ache/go/1.25.0/x64/bin/bash -errorsas -ifaceassert -nilfunc bash --no�� --noprofile -tests 0/x64/lib/node_modules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin/node (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -unreachable=false /tmp/go-build3184607200/b074/vet.cfg x86_64/bash (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -bool -buildtags de/node/bin/bash -errorsas -ifaceassert -nilfunc ache/go/1.25.0/x64/pkg/tool/linux_amd64/compile --no�� --noprofile -tests 0/x64/bin/npx (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha g_.a -buildtags k/_temp/ghcca-node/node/bin/bash -errorsas er -nilfunc bash --no�� dJ8v2Bu60 -tests x86_64/node (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha ion_reference.go ion_reference_test.go odules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin/node ion_resolver_tesgit ion_sha_checker.rev-parse ion_sha_checker_--show-toplevel ion_sha_checker_test.go ode_�� ivation_checkout_test.go ivation_github_token_test.go ndor/bin/sh (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha uts.version test@example.com (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha -bool -buildtags 64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert -nilfunc 64/pkg/tool/linux_amd64/vet ode_�� --noprofile -tests /usr/bin/tail (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env run_id }} GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha GOMODCACHE go /usr/bin/git on' --ignore-patgit GO111MODULE 64/bin/go git conf�� user.name Test User /opt/hostedtoolcache/node/24.14.0/x64/bin/node -json GO111MODULE x_amd64/compile node (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha 8551868/b432/_pkg_.a Bm9j/dNPAF48LeEY-J4ojBm9j 8551868/b432=> -p b/gh-aw/pkg/slicrev-parse -lang=go1.25 /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile -o runs/20260323-194241-57105/test-4130212216/.github/workflows -trimpath /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile l crypto/internal/rev-parse -lang=go1.25 8551868/b432/importcfg (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha x_amd64/compile git /usr/bin/git --git-dir x_amd64/compile /usr/bin/git git rev-�� blog-auditor.md git 64/pkg/tool/linux_amd64/link --show-toplevel x_amd64/vet /usr/bin/git 64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/bash GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha runs/20260323-194241-57105/test-1687368163/.github/workflows -trimpath /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile l math -lang=go1.25 /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile -o k/gh-aw/gh-aw -trimpath /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile -p encoding/hex -lang=go1.25 /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha xterm-color git tions/setup/js/node_modules/.bin/node --show-toplevel x_amd64/compile /usr/bin/infocmp--show-toplevel git 0/x6�� --show-toplevel infocmp 0/x64/bin/node xterm-color x_amd64/vet /usr/bin/git git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha 2850/001/stability-test.md GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha 262074/b160/importcfg -trimpath 0/x64/bin/node -I /tmp/go-build612rev-parse -I /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile t-ha�� ithub/workflows/auto-triage-issues.md -trimpath /opt/hostedtoolcache/go/1.25.0/x64/bin/go -p crypto/internal/rev-parse -lang=go1.25 go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha agent-performance-analyzer.md /opt/hostedtoolc--jq /usr/bin/git needs.build.outpgit x_amd64/compile /usr/bin/infocmp--show-toplevel git rev-�� '**/*.ts' '**/*.json' --ignore-path ../../../.prettierignore infocmp 0/x64/bin/node xterm-color x_amd64/vet /usr/bin/git git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/pkg/tool/linux_amd64/asm GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/asm (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE 64/pkg/tool/linux_amd64/asm GOINSECURE GOMOD (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/pkg/tool/linux_amd64/asm GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com env -json GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuorigin (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuTest User (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link env -json GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE ef/N6GE9dzJuLpfUremote.origin.url (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuTest User env _.a GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env _.a GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/pkg/tool/linux_amd64/asm GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com estl�� -json GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path go1.25.0 -c=4 -nolocalimports -importcfg /tmp/go-build612262074/b253/importcfg -pack /home/REDACTED/go/pkg/mod/golang.org/x/mod@v0.34.0/semver/semver.go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE go env on' --ignore-pat-errorsas GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE x_amd64/compile env flags="-w -s" -o@{u} GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuTest User env y_with_repos_array_c2143333823/001 elect.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha --show-toplevel /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile /usr/bin/git runs/20260323-19git pkg/mod/golang.orev-parse /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile k/_temp/uv-python-dir/node /tmp/go-build612sh l /usr/bin/infocmp"prettier" --write '**/*.cjs' '**/*.ts' '**/*.json' --ignore-path ../../../.prettierignore git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ck '**/*.cjs' '*GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env on' --ignore-pat-p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE ode_modules/.bin-lang=go1.25 GOINSECURE GOMOD GOMODCACHE go env mportMap,TestImp-p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE es/.bin/node GOINSECURE GOMOD GOMODCACHE go ode_�� -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE wasm.s sm.s�� e_wasm.s GO111MODULE x_amd64/asm GOINSECURE GOMOD GOMODCACHE x_amd64/asm (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go ode_�� -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha ty-test.md GO111MODULE At,event,headBranch,headSha,displayTitle GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile env _.a GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha --show-toplevel UCYY4uw-nj1ib/fQkmIpjEVxgPIJbBLrkR/yyjxJESSHc3089fRgrZr/aHz0DOVU-trimpath /usr/bin/git ithub/workflows/git -trimpath 8551868/b419/_pk--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile bin/node /tmp/go-build612sh -trimpath ache/node/24.14."prettier" --write '**/*.cjs' '**/*.ts' '**/*.json' --ignore-path ../../../.prettierignore git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ck '**/*.cjs' '*GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE s9ZXZGY/X4XoDkfiiEtxJ64HjgrP env 801436953/.github/workflows GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo -nolocalimports -importcfg /tmp/go-build612262074/b139/importcfg -pack /home/REDACTED/go/pkg/mod/golang.org/x/oauth2@v0.34.0/internal/doc.go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo -nolocalimports -importcfg /tmp/go-build612262074/b002/importcfg -embedcfg /tmp/go-build612262074/b002/embedcfg env -json GO111MODULE odules/npm/node_-lang=go1.25 GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build2588551868/b400/cli.test /tmp/go-build2588551868/b400/cli.test -test.testlogfile=/tmp/go-build2588551868/b400/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true -nolocalimports -importcfg /tmp/go-build612262074/b206/importcfg -pack env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name go1.25.0 -c=4 -nolocalimports -importcfg /tmp/go-build612262074/b242/importcfg -pack /home/REDACTED/go/pkg/mod/golang.org/x/text@v0.35.0/internal/language/compact/compact.go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)

If you need me to access, download, or install something from one of these locations, you can either:


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

…tFields

The extractAllImportFields function was calling ExtractFrontmatterFromContent
~20 times per imported file (once per field extraction). Each call independently
parsed the entire YAML frontmatter, causing O(fields × imports) YAML parse
operations instead of O(imports).

Fix: parse frontmatter once at the start of extractAllImportFields and reuse
the result via new map-based helper functions.

Add to content_extractor.go:
- extractFieldJSONFromMap: JSON extraction from pre-parsed frontmatter
- extractYAMLFieldFromMap: YAML extraction from pre-parsed frontmatter
- extractOnSectionFieldFromMap: on: section array field extraction
- extractOnSectionAnyFieldFromMap: on: section any-type field extraction

Remove now-unused content-string-based helpers:
- extractStepsFromContent, extractServicesFromContent
- extractPostStepsFromContent, extractOnSectionField
- extractOnSectionAnyField, extractOnGitHubToken
- extractOnGitHubApp, extractTopLevelGitHubApp (parser pkg versions)

Performance improvement (BenchmarkCompileComplexWorkflow):
- Speed: 5.6ms → 4.6ms (~18% faster)
- Allocations: 33,933 → 24,751 allocs/op (~27% fewer)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/208ce1c0-fe55-48b2-bfb4-f6cd0e8cdb80
Copilot AI changed the title [WIP] Fix performance regression in CompileComplexWorkflow perf: eliminate redundant YAML parsing in import field extraction Mar 23, 2026
Copilot AI requested a review from pelikhan March 23, 2026 19:50
@github-actions github-actions bot mentioned this pull request Mar 23, 2026
@pelikhan pelikhan marked this pull request as ready for review March 23, 2026 20:09
Copilot AI review requested due to automatic review settings March 23, 2026 20:09
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors import frontmatter field extraction to avoid repeatedly parsing YAML for each extracted field, improving workflow compilation performance (especially for implicit copilot imports).

Changes:

  • Parse imported file frontmatter once in extractAllImportFields and reuse the resulting map[string]any for all extractions.
  • Add map-based extraction helpers in content_extractor.go (JSON/YAML/on-section) and route existing string-based helpers through them.
  • Remove now-dead content-string helper functions that existed primarily to wrap repeated frontmatter parsing.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
pkg/parser/import_field_extractor.go Parses frontmatter once per imported file and reuses it across all field extractors; inlines old wrapper helpers.
pkg/parser/content_extractor.go Introduces *FromMap helpers to extract JSON/YAML fields from pre-parsed frontmatter; removes redundant content-based helpers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +111 to +119
// Parse frontmatter once to avoid redundant YAML parsing for each field extraction.
// All subsequent field extractions use the pre-parsed result.
parsed, err := ExtractFrontmatterFromContent(string(content))
var fm map[string]any
if err == nil {
fm = parsed.Frontmatter
} else {
fm = make(map[string]any)
}
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extractAllImportFields still re-parses frontmatter even though the BFS import loop already calls ExtractFrontmatterFromContent on the same content to discover nested imports. Consider passing the already-parsed FrontmatterResult (or at least the Frontmatter map) into extractAllImportFields so each imported file is YAML-parsed only once end-to-end.

Copilot uses AI. Check for mistakes.
Comment on lines +77 to +78
contentExtractorLog.Printf("Extracting field from map: %s", fieldName)

Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extractFieldJSONFromMap logs "Extracting field from map" even when called via extractFrontmatterField, which already logs an "Extracting field" message. With DEBUG enabled this will double-log every extraction call; consider removing one of these log lines (or making one log only at a more verbose level) to reduce noise and overhead.

Suggested change
contentExtractorLog.Printf("Extracting field from map: %s", fieldName)

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Contributor

Hey @Copilot 👋 — nice work on the YAML parse-once refactor! The benchmark numbers speak for themselves (−18% latency, −27% allocations on BenchmarkCompileComplexWorkflow), and the description is thorough and clear.

One thing that would help land this with confidence:

  • Add unit tests for the new map-based helpersextractFieldJSONFromMap, extractYAMLFieldFromMap, extractOnSectionFieldFromMap, and extractOnSectionAnyFieldFromMap are all new public-ish surface introduced by this PR, but neither content_extractor_features_test.go nor import_field_extractor_test.go (the two closest test files) cover them. The existing pattern in content_extractor_features_test.go uses table-driven tests with testify/assert — matching that style would be ideal.

If you'd like a hand, you can assign this prompt to your coding agent:

Add unit tests for the new map-based extraction helpers introduced in pkg/parser/content_extractor.go.

Create a new file pkg/parser/content_extractor_map_test.go (build tag: //go:build !integration) with table-driven tests using testify/assert. Cover the following functions and scenarios:

1. extractFieldJSONFromMap
   - Field present with a string value → returns JSON-encoded string
   - Field present with a map value → returns JSON-encoded map
   - Field absent → returns the emptyValue default
   - Empty frontmatter map → returns the emptyValue default

2. extractYAMLFieldFromMap
   - Field present with a slice value → returns YAML string (trimmed)
   - Field absent → returns ""
   - Field present with a map value → returns YAML string

3. extractOnSectionFieldFromMap
   - "on" key absent → returns "[]"
   - "on" key present but target field absent → returns "[]"
   - Target field is a single string → returns JSON array of one element
   - Target field is already a []any → returns JSON array

4. extractOnSectionAnyFieldFromMap
   - "on" key absent → returns ""
   - Target field is a string → returns JSON string
   - Target field is a map → returns JSON object

Follow the existing test style in pkg/parser/content_extractor_features_test.go (package parser, testify/assert, require.NoError).

Generated by Contribution Check ·

@pelikhan pelikhan merged commit 3c75759 into main Mar 23, 2026
50 of 53 checks passed
@pelikhan pelikhan deleted the copilot/fix-performance-regression-compile-complex-workflo branch March 23, 2026 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[performance] Regression in CompileComplexWorkflow: +82.9% slower

3 participants