Skip to content

feat(pagination): preserve pagination state on truncation and natural…#659

Merged
sang-neo03 merged 2 commits intomainfrom
fix/page-all-has-more
Apr 25, 2026
Merged

feat(pagination): preserve pagination state on truncation and natural…#659
sang-neo03 merged 2 commits intomainfrom
fix/page-all-has-more

Conversation

@sang-neo03
Copy link
Copy Markdown
Collaborator

@sang-neo03 sang-neo03 commented Apr 25, 2026

Summary

Fix --page-all silently truncating data at the default --page-limit=10: merged responses hard-coded has_more=false, so downstream consumers couldn't detect truncation (e.g. a
wiki node with 705 children returned 200 items with has_more=false, silently losing 505 rows). The merge now surfaces the last page's real has_more, letting callers detect
truncation and re-run with a larger --page-limit.

Changes

  • internal/client/pagination.go: replace the hard-coded has_more=false in mergePagedResults with the real value from the last fetched page. Page tokens are still dropped from
    the merged view — the aggregate is not a resume cursor; callers should re-run with a larger --page-limit to fetch more.
  • internal/client/client_test.go: extend TestPaginateAll_PageLimitStopsPagination to assert truncation now surfaces has_more=true and that page_token is dropped from the
    aggregate; add TestPaginateAll_NaturalEndClearsPageToken to lock in the clean-exit path.

Test Plan

  • Unit tests pass (go test ./internal/client/... ./cmd/api/... ./cmd/service/...)
  • go build ./... passes
  • Manual local verification confirms lark-cli wiki spaces list --as user --page-all --page-limit 2 --format json works as expected — old binary (1.0.17) returns
    has_more=false; patched binary returns has_more=true; items count is identical (40) on both.

Related Issues

Summary by CodeRabbit

Release Notes

  • Bug Fixes

    • Improved pagination reporting to accurately indicate when more data is available, even when pagination stops due to configured limits.
  • Tests

    • Enhanced test coverage for pagination behavior when reaching page limits and when pagination completes naturally.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 25, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fb156478-a6d2-4e06-a146-46731cb75549

📥 Commits

Reviewing files that changed from the base of the PR and between f223827 and 2636502.

📒 Files selected for processing (2)
  • internal/client/client_test.go
  • internal/client/pagination.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/client/client_test.go
  • internal/client/pagination.go

📝 Walkthrough

Walkthrough

mergePagedResults now derives the merged data.has_more from the last page in the results slice (defaulting to false on type mismatch) while still removing pagination tokens from the merged view. Tests updated to assert correct has_more behavior for both page-limit truncation and natural end.

Changes

Cohort / File(s) Summary
Pagination Logic
internal/client/pagination.go
Changed mergePagedResults to read data.has_more from the last page in results instead of always setting it to false. Pagination tokens (page_token, next_page_token) are still removed from the merged result.
Pagination Tests
internal/client/client_test.go
Updated TestPaginateAll_PageLimitStopsPagination to inspect merged response ensuring data.has_more=true when pagination stops due to page-limit and token is absent; added TestPaginateAll_NaturalEndClearsPageToken to verify data.has_more=false and token absence when pagination ends naturally.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 I hopped through pages, counting each part,
Mended the truth of the has_more in heart.
Tokens tuck away when the end is in sight,
But when limits cut short, I keep has_more bright. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly refers to the core fix: preserving pagination state (has_more flag) when pagination ends naturally or is truncated by page-limit.
Description check ✅ Passed The description covers all required template sections: a clear summary of the bug and fix, detailed list of changes with file-level specifics, comprehensive test plan with verification steps, and proper issue closure reference.
Linked Issues check ✅ Passed The PR fully addresses the primary objective from issue #590: it preserves the real has_more flag from the last page in merged output so callers can detect truncation caused by page-limit, enabling re-runs with larger limits.
Out of Scope Changes check ✅ Passed All changes are scoped to pagination logic (mergePagedResults function) and its tests; no unrelated modifications or cleanup tasks are present.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/page-all-has-more

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added the size/M Single-domain feat or fix with limited business impact label Apr 25, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
internal/client/client_test.go (1)

239-276: Natural-end test correctly exercises the deletion branch.

The fixture flips has_more to false on the second call and omits page_token, validating that mergePagedResults clears the token when the API reports no more pages.

Optional nit: consider also asserting len(data["items"]) == 2 to lock in that natural-end merging still aggregates items across both pages — pure pagination-state focus is fine, but a one-line items-count check would catch regressions in the merge loop while you're already exercising it.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/client/client_test.go` around lines 239 - 276, Add an assertion in
TestPaginateAll_NaturalEndClearsPageToken to also verify that the merged items
list actually contains both pages' items (e.g., assert len(data["items"]) == 2)
so the test not only checks page_token clearing but also confirms
mergePagedResults aggregated items across both calls; locate this in the test
after extracting resultMap/data and add a single-line length check against
data["items"].
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@internal/client/client_test.go`:
- Around line 239-276: Add an assertion in
TestPaginateAll_NaturalEndClearsPageToken to also verify that the merged items
list actually contains both pages' items (e.g., assert len(data["items"]) == 2)
so the test not only checks page_token clearing but also confirms
mergePagedResults aggregated items across both calls; locate this in the test
after extracting resultMap/data and add a single-line length check against
data["items"].

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d03758d1-e177-4bff-a726-71c5425ce091

📥 Commits

Reviewing files that changed from the base of the PR and between 2e7a11a and f223827.

📒 Files selected for processing (2)
  • internal/client/client_test.go
  • internal/client/pagination.go

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 25, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.95%. Comparing base (fd4c35b) to head (2636502).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #659      +/-   ##
==========================================
+ Coverage   62.67%   62.95%   +0.27%     
==========================================
  Files         434      436       +2     
  Lines       38072    38597     +525     
==========================================
+ Hits        23862    24299     +437     
- Misses      12076    12150      +74     
- Partials     2134     2148      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 25, 2026

🚀 PR Preview Install Guide

🧰 CLI update

npm i -g https://pkg.pr.new/larksuite/cli/@larksuite/cli@2636502d694c639d1a7ac14aa5abb36ee120f3fe

🧩 Skill update

npx skills add larksuite/cli#fix/page-all-has-more -y -g

@sang-neo03 sang-neo03 merged commit ddf6f0c into main Apr 25, 2026
21 checks passed
@sang-neo03 sang-neo03 deleted the fix/page-all-has-more branch April 25, 2026 09:54
@liangshuo-1 liangshuo-1 mentioned this pull request Apr 27, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Single-domain feat or fix with limited business impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: --page-all silently truncates data at default --page-limit=10 and falsely reports has_more=false

2 participants