feat(usage): classify finish_reason values and flag error reasons in has_error#3071
Open
kilo-code-bot[bot] wants to merge 1 commit intomainfrom
Open
feat(usage): classify finish_reason values and flag error reasons in has_error#3071kilo-code-bot[bot] wants to merge 1 commit intomainfrom
kilo-code-bot[bot] wants to merge 1 commit intomainfrom
Conversation
…has_error Introduce a shared zod enum + helper (isErrorFinishReason) for the set of finish_reason / stop_reason / status values we observe across OpenAI chat completions, OpenRouter, Anthropic Messages API, OpenAI Responses API, and Vercel AI SDK style responses. Reasons that indicate truncation, refusal, content filtering, upstream failure, or an interrupted in_progress stream now flip has_error to true in all three usage parsers (chat completions, messages, responses string path). Normal completion reasons (stop, end_turn, tool_use, completed, stop_sequence, tool_calls/tool-calls) and unclassified catch-alls (unknown, other, null) keep has_error driven only by status code and abort signals as before.
Contributor
Author
Code Review SummaryStatus: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Other Observations (not in diff)Issues found in unchanged code that cannot receive inline comments:
Files Reviewed (5 files)
Fix these issues in Kilo Cloud Reviewed by gpt-5.5-20260423 · 384,645 tokens |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
finishReasonmodule (apps/web/src/lib/ai-gateway/finishReason.ts) with a zod enum (FinishReasonSchema) covering everyfinish_reason/stop_reason/ Responses APIstatusvalue we currently see across OpenAI/OpenRouter chat completions, Anthropic Messages, OpenAI Responses, and Vercel AI SDK style outputs.ERROR_FINISH_REASONS(truncation/refusal/upstream failure/interrupted) andNON_ERROR_FINISH_REASONS(normal completion + catch-alls), and exposesisErrorFinishReason().isErrorFinishReason(finish_reason)into thehasErrorcalculation of all three microdollar usage parsers — the OpenRouter chat completions stream + string parser (processUsage.ts), the Anthropic Messages stream + string parser (processUsage.messages.ts), and the OpenAI Responses stream parser (processUsage.responses.ts). The Responses API string parser keeps its existing stricterstatus !== 'completed'rule untouched.Net effect: a 200 OK response that ends with
stop_reason: "refusal",finish_reason: "length" / "content_filter" / "error", etc. is now recorded withhas_error: trueinmicrodollar_usage, instead of being silently logged as a successful row. Unrecognised provider strings remain non-errors so we don't generate spurious failures when a provider invents a new value.Verification
coreProps.hasErrorfor both error and non-error finish_reason values.finish_reason: 'stop', approved snapshots with'end_turn'/'completed') — all classified as non-error so existing assertions are unaffected.Visual Changes
N/A
Reviewer Notes
lengthandmax_tokensas errors (truncation surfaces something product/customers usually want to see). If you'd rather keep truncation as a non-error, move those two entries between the two arrays infinishReason.tsand the test will still pass after a one-line update.unknownandotherare treated as non-errors so that brand-new upstream values don't immediately spike error rates; pair this with existing Sentry warnings if we want visibility on novel values.status !== 'completed'as its error rule, which is strictly broader thanisErrorFinishReason. I left it alone to avoid weakening that path.pnpm typecheck/pnpm test/pnpm formatper the request not to run long-running tasks before pushing — please re-run CI if you want a clean signal.