Skip to content

Add pagination to bot-detection API calls with 500-item cap #15318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mnkiefer merged 6 commits into from
Feb 13, 2026
Merged

Add pagination to bot-detection API calls with 500-item cap #15318

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
db09ce0
Initial plan
Copilot Feb 13, 2026
4300b87
Initial plan for pagination implementation
Copilot Feb 13, 2026
e11c6e6
Use github.paginate for all PR/issue API calls with 500-item cap
Copilot Feb 13, 2026
93c8d67
Fix pagination cap logic to track total accumulated items
Copilot Feb 13, 2026
c1e5918
Enforce exact 500-item cap by slicing response data
Copilot Feb 13, 2026
0b2adb3
Fix pagination logic: check total before updating, use remaining corr…
Copilot Feb 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 99 additions & 34 deletions .github/workflows/bot-detection.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

125 changes: 97 additions & 28 deletions .github/workflows/bot-detection.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,39 +232,90 @@ jobs:

let issueComments = [];
try {
const { data } = await github.rest.issues.listComments({
owner,
repo,
issue_number: it.number,
per_page: 100,
});
issueComments = data || [];
let total = 0;
issueComments = await github.paginate(
github.rest.issues.listComments,
{
owner,
repo,
issue_number: it.number,
per_page: 100,
},
(response, done) => {
const remaining = 500 - total;
if (remaining <= 0) {
done();
return [];
}
if (total + response.data.length >= 500) {
total = 500;
Comment on lines +245 to +251
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cap value 500 is hard-coded multiple times inside this pagination callback (and repeated across the other paginate calls). Consider defining a single constant (e.g., MAX_PAGINATE_ITEMS) and referencing it here to avoid inconsistencies if the cap ever changes, and to reduce duplicated magic numbers across the script.

Copilot uses AI. Check for mistakes.
done();
return response.data.slice(0, remaining);
}
total += response.data.length;
return response.data;
}
);
} catch {
// ignore
}

let reviewComments = [];
try {
const { data } = await github.rest.pulls.listReviewComments({
owner,
repo,
pull_number: it.number,
per_page: 100,
});
reviewComments = data || [];
let total = 0;
reviewComments = await github.paginate(
github.rest.pulls.listReviewComments,
{
owner,
repo,
pull_number: it.number,
per_page: 100,
},
(response, done) => {
const remaining = 500 - total;
if (remaining <= 0) {
done();
return [];
}
if (total + response.data.length >= 500) {
total = 500;
done();
return response.data.slice(0, remaining);
}
total += response.data.length;
return response.data;
}
);
} catch {
// ignore
}

let reviews = [];
try {
const { data } = await github.rest.pulls.listReviews({
owner,
repo,
pull_number: it.number,
per_page: 100,
});
reviews = data || [];
let total = 0;
reviews = await github.paginate(
github.rest.pulls.listReviews,
{
owner,
repo,
pull_number: it.number,
per_page: 100,
},
(response, done) => {
const remaining = 500 - total;
if (remaining <= 0) {
done();
return [];
}
if (total + response.data.length >= 500) {
total = 500;
done();
return response.data.slice(0, remaining);
}
total += response.data.length;
return response.data;
}
);
} catch {
// ignore
}
Expand Down Expand Up @@ -314,13 +365,31 @@ jobs:
const s = ensureAuthor(login);

try {
const filesResp = await github.rest.pulls.listFiles({
owner,
repo,
pull_number: it.number,
per_page: 100,
});
const filenames = (filesResp.data || []).map(f => f.filename);
let total = 0;
const files = await github.paginate(
github.rest.pulls.listFiles,
{
owner,
repo,
pull_number: it.number,
per_page: 100,
},
(response, done) => {
const remaining = 500 - total;
if (remaining <= 0) {
done();
return [];
}
if (total + response.data.length >= 500) {
total = 500;
done();
return response.data.slice(0, remaining);
}
total += response.data.length;
return response.data;
}
);
const filenames = files.map(f => f.filename);
for (const fn of filenames) {
if (fn.startsWith(".github/workflows/") || fn.startsWith(".github/actions/")) s.touchesWorkflows = true;
if (fn === "Dockerfile" || fn === "Makefile" || fn.startsWith("scripts/") || fn.startsWith("actions/")) s.touchesCI = true;
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/smoke-copilot.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 7 additions & 7 deletions actions/setup/js/merge_remote_agent_github_folder.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,23 +146,23 @@ function validateGitParameter(value, name) {
*/
function validateSafePath(userPath, basePath, name) {
// Reject paths with null bytes
if (userPath.includes('\0')) {
if (userPath.includes("\0")) {
throw new Error(`Invalid ${name}: contains null bytes`);
}

// Reject paths that attempt to traverse up (..)
if (userPath.includes('..')) {
if (userPath.includes("..")) {
throw new Error(`Invalid ${name}: path traversal detected`);
}

// Resolve the full path and ensure it's within the base path
const resolvedPath = path.resolve(basePath, userPath);
const resolvedBase = path.resolve(basePath);

if (!resolvedPath.startsWith(resolvedBase + path.sep) && resolvedPath !== resolvedBase) {
throw new Error(`Invalid ${name}: path escapes base directory`);
}

return resolvedPath;
}

Expand Down Expand Up @@ -239,7 +239,7 @@ function mergeGithubFolder(sourcePath, destPath) {
for (const relativePath of sourceFiles) {
// Validate relative path to prevent path traversal
validateSafePath(relativePath, sourcePath, "relative file path");

// Check if the file is in one of the allowed subfolders
const pathParts = relativePath.split(path.sep);
const topLevelFolder = pathParts[0];
Expand Down
Loading