Skip to content

Update enhanced-inference.ts #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
HO-git wants to merge 1 commit into
base: Cache-error-test
Choose a base branch
from
Open

Update enhanced-inference.ts #10

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion deprecated-claude-app/backend/src/services/enhanced-inference.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -700,6 +700,13 @@ export class EnhancedInferenceService {
?? INPUT_PRICING_PER_MILLION[model.id];

if (price === undefined) {
if (model.provider === 'openai-compatible') {
console.warn(
`[Pricing] No pricing configured for openai-compatible model ${model.id} (${model.providerModelId || 'none'}), assuming $0 for metrics`
);
return 0;
}

// Throw error instead of silently returning $0 - prevents untracked charges
throw new PricingNotConfiguredError(model.id, model.provider, model.providerModelId);
}
Expand Down Expand Up @@ -727,6 +734,13 @@ export class EnhancedInferenceService {
?? OUTPUT_PRICING_PER_MILLION[model.id];

if (price === undefined) {
if (model.provider === 'openai-compatible') {
console.warn(
`[Pricing] No pricing configured for openai-compatible model ${model.id} (${model.providerModelId || 'none'}), assuming $0 for metrics`
);
return 0;
}
Comment on lines 703 to +742
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validatePricingAvailable not updated to match

The upstream validatePricingAvailable function (line 232) does not have a corresponding bypass for openai-compatible providers. This means a non-UUID, non-customEndpoint openai-compatible model without any configured pricing will still be blocked at the handler's validation gate before reaching these functions — so the new return 0 path is currently only reachable by user-defined (isUUID / hasCustomEndpoint) models.

This is consistent with the original intent (user-defined models bypass the pricing gate and accept responsibility for their own costs), but the two behaviours are not obviously co-located. Consider adding a matching bypass or comment in validatePricingAvailable to make it clear that openai-compatible models are expected to gracefully fall back to $0 pricing:

// In validatePricingAvailable, after step 3:
if (model.provider === 'openai-compatible') {
  // openai-compatible models are self-hosted; cost tracking is best-effort
  return { valid: true };
}

This would make the two functions consistent and prevent future confusion if validatePricingAvailable is called independently.


// Throw error instead of silently returning $0 - prevents untracked charges
throw new PricingNotConfiguredError(model.id, model.provider, model.providerModelId);
}
Expand Down Expand Up @@ -786,4 +800,4 @@ export class EnhancedInferenceService {
return this.contextManager.getCacheMarker(conversationId, participantId);
}

}
}