feat(job:list): add --no-limit option #588
Conversation
Previously, --limit constrained build fetches before filtering, causing jobs to be missed when using --duration, --state, or --queue filters. Now automatically fetches more builds (up to 5000) when client-side filters are present, and applies --limit only to final results. Also uses --state as server-side hint to optimize API calls. Before: bk job list --duration 1m --limit 10 -> 1 result After: bk job list --duration 1m --limit 10 -> 10 results
JoeColeman95
left a comment
JoeColeman95
left a comment
There was a problem hiding this comment.
Left comments on 2 major issues/concerns here, I get the concept behind the PR (increase the sample) but we're inherently limited by the API itself in this case, and note the filtering types to make this more apparent.
I suspect we could just detach the --limit flag from the number of results we query (although that was the intended design), or even simply bumping the default value of *2 to a larger number or switching this to be a pre-set value of 200 (which is max for 1 page) and then keeping --limit as a post-filter cap.
On the filter, I'd drop that entirely as it's unreliable, we're using builds as a source of truth here, filtering them by state skews results entirely.
|
As a side note, I think that the PR title needs to be tweaked as this doesn't seem to be a fix, but more of a change to design choice? |
scadu
changed the title
Description
When using client-side filters (
--duration,--state,--queue), the default behavior only fetches a limited number of builds, which can cause matching jobs to be missed.This adds a
--no-limitflag (consistent with build list added in #540) to allow users to explicitly fetch all builds whenDefault: fetches up to
limit*2--no-limit: fetches all buildsExample:
bk job list --duration 1m --limit 10-> 4 results (from ~20 builds)
bk job list --duration 1m --limit 10 --no-limit-> 4179 results (from all builds)
Disclosures / Credits