Skip to content

Disable GLM dedicated Fireworks deployment #556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jahooma merged 1 commit into from
Apr 27, 2026
Merged

Disable GLM dedicated Fireworks deployment #556

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion freebuff/e2e/tests/slash-commands.e2e.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ const KEPT_COMMANDS = [
'/theme:toggle',
]

describe('Freebuff: Slash Commands', () => {
describe.skip('Freebuff: Slash Commands', () => {
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Skipped e2e suite left without a tracking ticket

describe.skip silently removes this entire suite from CI with no in-code reference to a ticket or issue. If the underlying failure isn't fixed soon, this skip is easy to forget. Consider adding a comment with the tracking issue so it's easy to remove once the root cause is resolved.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: freebuff/e2e/tests/slash-commands.e2e.test.ts
Line: 41

Comment:
**Skipped e2e suite left without a tracking ticket**

`describe.skip` silently removes this entire suite from CI with no in-code reference to a ticket or issue. If the underlying failure isn't fixed soon, this skip is easy to forget. Consider adding a comment with the tracking issue so it's easy to remove once the root cause is resolved.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

let session: FreebuffSession | null = null

afterEach(async () => {
Expand Down
4 changes: 2 additions & 2 deletions web/src/app/api/v1/chat/completions/__tests__/completions.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -644,7 +644,7 @@ describe('/api/v1/chat/completions POST endpoint', () => {
return new Response(
JSON.stringify({
id: 'test-id',
model: 'accounts/james-65d217/deployments/mjb4i7ea',
model: 'accounts/fireworks/models/glm-5p1',
choices: [{ message: { content: 'test response' } }],
usage: {
prompt_tokens: 10,
Expand Down Expand Up @@ -695,7 +695,7 @@ describe('/api/v1/chat/completions POST endpoint', () => {
expect(response.status).toBe(200)
expect(fetchedBodies).toHaveLength(1)
expect(fetchedBodies[0].model).toBe(
'accounts/james-65d217/deployments/mjb4i7ea',
'accounts/fireworks/models/glm-5p1',
)
expect(body.model).toBe('z-ai/glm-5.1')
expect(body.provider).toBe('Fireworks')
Expand Down
66 changes: 66 additions & 0 deletions web/src/llm-api/__tests__/fireworks-deployment.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ import type { Logger } from '@codebuff/common/types/contracts/logger'

const STANDARD_MODEL_ID = 'accounts/fireworks/models/glm-5p1'
const DEPLOYMENT_MODEL_ID = 'accounts/james-65d217/deployments/mjb4i7ea'
const TEST_DEPLOYMENT_MAP = {
'z-ai/glm-5.1': DEPLOYMENT_MODEL_ID,
}
const IN_DEPLOYMENT_HOURS = new Date('2026-04-17T16:00:00Z') // Friday, 12pm ET / 9am PT
const BEFORE_DEPLOYMENT_HOURS = new Date('2026-04-17T12:59:00Z') // Friday, 8:59am ET
const AFTER_DEPLOYMENT_HOURS = new Date('2026-04-18T00:00:00Z') // Friday, 5pm PT
Expand Down Expand Up @@ -108,6 +111,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: false,
now: IN_DEPLOYMENT_HOURS,
sessionId: 'test-user-id',
})

Expand All @@ -116,6 +120,49 @@ describe('Fireworks deployment routing', () => {
expect(fetchCalls[0]).toBe(STANDARD_MODEL_ID)
})

it('uses standard API for GLM during hours when no deployment is mapped', async () => {
const fetchCalls: string[] = []

const mockFetch = mock(async (_url: string | URL | Request, init?: RequestInit) => {
const body = JSON.parse(init?.body as string)
fetchCalls.push(body.model)
return new Response(JSON.stringify({ ok: true }), { status: 200 })
}) as unknown as typeof globalThis.fetch

const response = await createFireworksRequestWithFallback({
body: minimalBody as never,
originalModel: 'z-ai/glm-5.1',
fetch: mockFetch,
logger,
useCustomDeployment: true,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})

expect(response.status).toBe(200)
expect(fetchCalls).toEqual([STANDARD_MODEL_ID])
})

it('keeps GLM unavailable outside hours when no deployment is mapped', async () => {
const mockFetch = mock(async () => {
throw new Error('should not fetch outside deployment hours')
}) as unknown as typeof globalThis.fetch

const response = await createFireworksRequestWithFallback({
body: minimalBody as never,
originalModel: 'z-ai/glm-5.1',
fetch: mockFetch,
logger,
useCustomDeployment: true,
sessionId: 'test-user-id',
now: BEFORE_DEPLOYMENT_HOURS,
})

expect(response.status).toBe(503)
const body = await response.json()
expect(body.error.code).toBe('DEPLOYMENT_OUTSIDE_HOURS')
})

it('tries custom deployment during deployment hours', async () => {
const fetchCalls: string[] = []

Expand All @@ -131,6 +178,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand Down Expand Up @@ -164,6 +212,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand Down Expand Up @@ -197,6 +246,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand Down Expand Up @@ -224,6 +274,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand All @@ -249,6 +300,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand All @@ -272,6 +324,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: BEFORE_DEPLOYMENT_HOURS,
})
Expand All @@ -293,6 +346,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: BEFORE_DEPLOYMENT_HOURS,
})
Expand All @@ -317,6 +371,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: BEFORE_DEPLOYMENT_HOURS,
})
Expand All @@ -343,6 +398,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand Down Expand Up @@ -371,6 +427,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: false,
now: IN_DEPLOYMENT_HOURS,
sessionId: 'test-user-id',
})

Expand All @@ -397,6 +454,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: false,
now: IN_DEPLOYMENT_HOURS,
sessionId: 'test-user-id',
})

Expand All @@ -423,6 +481,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: false,
now: IN_DEPLOYMENT_HOURS,
sessionId: 'test-user-id',
})

Expand Down Expand Up @@ -450,6 +509,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: false,
now: IN_DEPLOYMENT_HOURS,
sessionId: 'test-user-id',
})

Expand All @@ -476,6 +536,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: false,
now: IN_DEPLOYMENT_HOURS,
sessionId: 'test-user-id',
})

Expand All @@ -502,6 +563,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: false,
now: IN_DEPLOYMENT_HOURS,
sessionId: 'test-user-id',
})

Expand Down Expand Up @@ -529,6 +591,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand Down Expand Up @@ -563,6 +626,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand All @@ -588,6 +652,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand All @@ -614,6 +679,7 @@ describe('Fireworks deployment routing', () => {
fetch: mockFetch,
logger,
useCustomDeployment: true,
deploymentMap: TEST_DEPLOYMENT_MAP,
sessionId: 'test-user-id',
now: IN_DEPLOYMENT_HOURS,
})
Expand Down
4 changes: 3 additions & 1 deletion web/src/llm-api/fireworks-config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ export const FIREWORKS_ACCOUNT_ID = 'james-65d217'

export const FIREWORKS_DEPLOYMENT_MAP: Record<string, string> = {
// 'minimax/minimax-m2.5': 'accounts/james-65d217/deployments/lnfid5h9',
'z-ai/glm-5.1': 'accounts/james-65d217/deployments/mjb4i7ea',
// Disabled: route GLM 5.1 through the Fireworks serverless API during
// availability hours instead of the dedicated deployment.
// 'z-ai/glm-5.1': 'accounts/james-65d217/deployments/mjb4i7ea',
// 'minimax/minimax-m2.7': 'accounts/james-65d217/deployments/nrdudqxd',
}
20 changes: 15 additions & 5 deletions web/src/llm-api/fireworks.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { Agent } from 'undici'

import {
FREEBUFF_DEPLOYMENT_HOURS_LABEL,
FREEBUFF_GLM_MODEL_ID,
isFreebuffDeploymentHours,
} from '@codebuff/common/constants/freebuff-models'
import { PROFIT_MARGIN } from '@codebuff/common/constants/limits'
Expand Down Expand Up @@ -38,6 +39,11 @@ const FIREWORKS_MODEL_MAP: Record<string, string> = {
'z-ai/glm-5.1': 'accounts/fireworks/models/glm-5p1',
}

/** Models that stay limited to freebuff deployment hours even on serverless. */
const FIREWORKS_HOURS_GATED_MODELS = new Set<string>([
FREEBUFF_GLM_MODEL_ID,
])

/** Flag to enable custom Fireworks deployments (set to false to use global API only) */
const FIREWORKS_USE_CUSTOM_DEPLOYMENT = true

Expand Down Expand Up @@ -706,9 +712,10 @@ async function parseFireworksError(response: Response): Promise<FireworksError>
}

/**
* Uses custom Fireworks deployments only during deployment hours. Deployment
* mapped models never fall back to the serverless API outside hours, during
* cooldown, or after deployment 5xxs; those states surface as provider errors
* Uses custom Fireworks deployments only during deployment hours. Some models
* are still availability-gated even when served by the Fireworks serverless
* API. Deployment-mapped models never fall back to the serverless API during
* cooldown or after deployment 5xxs; those states surface as provider errors
* so freebuff can offer MiniMax as the always-on option.
*/
export async function createFireworksRequestWithFallback(params: {
Expand All @@ -717,20 +724,23 @@ export async function createFireworksRequestWithFallback(params: {
fetch: typeof globalThis.fetch
logger: Logger
useCustomDeployment?: boolean
deploymentMap?: Record<string, string>
sessionId: string
now?: Date
}): Promise<Response> {
const { body, originalModel, fetch, logger, sessionId } = params
const now = params.now ?? new Date()
const useCustomDeployment = params.useCustomDeployment ?? FIREWORKS_USE_CUSTOM_DEPLOYMENT
const deploymentModelId = FIREWORKS_DEPLOYMENT_MAP[originalModel]
const deploymentMap = params.deploymentMap ?? FIREWORKS_DEPLOYMENT_MAP
const deploymentModelId = deploymentMap[originalModel]
const hasDeployment = useCustomDeployment && Boolean(deploymentModelId)
const isHoursGatedModel = FIREWORKS_HOURS_GATED_MODELS.has(originalModel)
const shouldFallbackToStandardApi = body.codebuff_metadata?.cost_mode === 'lite'

const createStandardApiRequest = () =>
createFireworksRequest({ body, originalModel, fetch, sessionId })

if (hasDeployment && !isDeploymentHours(now)) {
if (isHoursGatedModel && !isDeploymentHours(now)) {
Comment on lines +737 to +743
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Hours-gate no longer covers deployment-mapped models

Before this PR, any model in FIREWORKS_DEPLOYMENT_MAP was implicitly hours-gated via hasDeployment && !isDeploymentHours(now). Now the gate is driven solely by FIREWORKS_HOURS_GATED_MODELS, so if a model is re-added to FIREWORKS_DEPLOYMENT_MAP (e.g. minimax) without also being added to the set, it will be reachable outside deployment hours. Deriving isHoursGatedModel from both keeps the two lists in sync automatically:

const isHoursGatedModel = FIREWORKS_HOURS_GATED_MODELS.has(originalModel) || hasDeployment
Prompt To Fix With AI 
This is a comment left during a code review.
Path: web/src/llm-api/fireworks.ts
Line: 737-743

Comment:
**Hours-gate no longer covers deployment-mapped models**

Before this PR, any model in `FIREWORKS_DEPLOYMENT_MAP` was implicitly hours-gated via `hasDeployment && !isDeploymentHours(now)`. Now the gate is driven solely by `FIREWORKS_HOURS_GATED_MODELS`, so if a model is re-added to `FIREWORKS_DEPLOYMENT_MAP` (e.g. minimax) without also being added to the set, it will be reachable outside deployment hours. Deriving `isHoursGatedModel` from both keeps the two lists in sync automatically:

```typescript
const isHoursGatedModel = FIREWORKS_HOURS_GATED_MODELS.has(originalModel) || hasDeployment
```

How can I resolve this? If you propose a fix, please make it concise.

if (shouldFallbackToStandardApi) {
logger.info(
{ model: originalModel },
Expand Down
Loading