[TS-664] Analysis upload fails with "rejecting SARIF, as there are more runs than allowed"

[ben-manes]

This only fails for codacy, so I am wondering if the error is referring to the sarif file itself having too many "runs"? I tried running it in isolation w/o any other jobs and it failed, so I think it has to be something to do with the report contents.

Run github/codeql-action/upload-sarif@cdcdbb57970684[1](https://github.com/ben-manes/caffeine/actions/runs/6915822206/job/18815190960#step:7:1)c47f7063dda365e292e5cad7a
Uploading results
Waiting for processing to finish
Error: Code Scanning could not process the submitted SARIF file:
rejecting SARIF, as there are more runs than allowed (21 > 20)
Error: Code Scanning could not process the submitted SARIF file:
rejecting SARIF, as there are more runs than allowed (21 > 20)
    at Object.waitForProcessing (/home/runner/work/_actions/github/codeql-action/cdcdbb579706841c47f7063dda365e292e5cad7a/lib/upload-lib.js:337:23)
    at async run (/home/runner/work/_actions/github/codeql-action/cdcdbb579706841c47f7063dda365e292e5cad7a/lib/upload-sarif-action.js:55:13)
    at async runWrapper (/home/runner/work/_actions/github/codeql-action/cdcdbb579706841c47f7063dda365e292e5cad7a/lib/upload-sarif-action.js:70:9)

See the github-actions log.

github/codeql-action#2000

[github-actions]

Internal ticket created : TS-664

[ben-manes]

From the GitHub docs, this is the analyzer exceeding the limits.

[ben-manes]

I went ahead and fixed this by removing the duplicate runs to fix the integration.

jq '.runs |= unique_by({tool, invocations})' results.sarif > codacy.sarif

original sarif

$ gron results.sarif.json | rg "tool.driver.name"
json.runs[0].tool.driver.name = "Checkov (reported by Codacy)";
json.runs[1].tool.driver.name = "Semgrep (reported by Codacy)";
json.runs[2].tool.driver.name = "Pmd (reported by Codacy)";
json.runs[3].tool.driver.name = "Semgrep (reported by Codacy)";
json.runs[4].tool.driver.name = "Cppcheck (reported by Codacy)";
json.runs[5].tool.driver.name = "Pmd (reported by Codacy)";
json.runs[6].tool.driver.name = "Markdownlint (reported by Codacy)";
json.runs[7].tool.driver.name = "Trivy (reported by Codacy)";
json.runs[8].tool.driver.name = "Semgrep (reported by Codacy)";
json.runs[9].tool.driver.name = "Shellcheck (reported by Codacy)";
json.runs[10].tool.driver.name = "Semgrep (reported by Codacy)";
json.runs[11].tool.driver.name = "Trivy (reported by Codacy)";
json.runs[12].tool.driver.name = "Jacksonlinter (reported by Codacy)";
json.runs[13].tool.driver.name = "Detekt (reported by Codacy)";
json.runs[14].tool.driver.name = "Flawfinder (reported by Codacy)";
json.runs[15].tool.driver.name = "Remark-lint (reported by Codacy)";
json.runs[16].tool.driver.name = "Spectral (reported by Codacy)";
json.runs[17].tool.driver.name = "Trivy (reported by Codacy)";
json.runs[18].tool.driver.name = "Trivy (reported by Codacy)";
json.runs[19].tool.driver.name = "Spectral (reported by Codacy)";
json.runs[20].tool.driver.name = "Checkov (reported by Codacy)";

fixed sarif

$ gron codacy.sarif | rg "tool.driver.name"
json.runs[0].tool.driver.name = "Checkov (reported by Codacy)";
json.runs[1].tool.driver.name = "Cppcheck (reported by Codacy)";
json.runs[2].tool.driver.name = "Detekt (reported by Codacy)";
json.runs[3].tool.driver.name = "Flawfinder (reported by Codacy)";
json.runs[4].tool.driver.name = "Jacksonlinter (reported by Codacy)";
json.runs[5].tool.driver.name = "Markdownlint (reported by Codacy)";
json.runs[6].tool.driver.name = "Pmd (reported by Codacy)";
json.runs[7].tool.driver.name = "Remark-lint (reported by Codacy)";
json.runs[8].tool.driver.name = "Semgrep (reported by Codacy)";
json.runs[9].tool.driver.name = "Shellcheck (reported by Codacy)";
json.runs[10].tool.driver.name = "Spectral (reported by Codacy)";
json.runs[11].tool.driver.name = "Trivy (reported by Codacy)";

[NanashiTheNameless]

I went ahead and fixed this by removing the duplicate runs to fix the integration.

jq '.runs |= unique_by({tool, invocations})' results.sarif > codacy.sarif

original sarif

fixed sarif

how does one implement this fix?

[ben-manes]

how does one implement this fix?

https://github.com/ben-manes/caffeine/blob/b3b8d78d24fff4bfb037c2fd32a7519d2cb7955d/.github/workflows/codacy.yml#L50-L52