Make audio separator async with Firestore + GCS#280
Merged
Conversation
Introduces FirestoreJobStore to replace the in-memory job_status_store dict, enabling any Cloud Run instance to read/write job status for multi-instance GPU scaling. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GCSOutputStore to upload separation results to GCS so any Cloud Run instance can serve download requests, replacing local disk storage. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Convert the /separate endpoint from synchronous (await) to fire-and-forget pattern so it returns immediately while separation runs in background. Replace in-memory job_status_store with Firestore reads/writes and local file downloads with GCS, enabling cross-instance status polling and file serving. Add GPU semaphore to serialize concurrent separation requests on a single instance. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 tasks
Move google.cloud imports inside __init__ methods so modules can be imported without the packages installed. Add google-cloud-firestore and google-cloud-storage to dev dependencies so CI has them available. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The TestLazyInit tests import deploy_cloudrun.py which requires uvicorn and fastapi - server-only dependencies not in the test environment. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
beveradb
added a commit
to nomadkaraoke/karaoke-gen
that referenced
this pull request
Mar 26, 2026
## Summary - Update audio separator Cloud Run service: `max_instance_request_concurrency` 1→50, `max_instances` 5→10 - Add `OUTPUT_BUCKET` and `GCP_PROJECT` env vars for the separator service - Create `nomadkaraoke-audio-separator-outputs` GCS bucket with 1-day lifecycle policy - Grant separator service account `roles/datastore.user` for Firestore and `roles/storage.objectAdmin` for the output bucket - Add Firestore composite index on `audio_separation_jobs` (status + updated_at) for cleanup queries Companion to nomadkaraoke/python-audio-separator#280. Deploy order: this PR's infra first (`pulumi up`), then the separator service. ## Test plan - [ ] `pulumi preview` shows expected resource changes - [ ] `pulumi up` applies cleanly - [ ] Verify GCS bucket created with lifecycle policy - [ ] Verify Firestore index created - [ ] Verify separator service updated with new concurrency/scaling settings @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/separateendpoint return immediately (fire-and-forget), enabling client polling via existing/statusand/downloadendpointsjob_status_storedict with Firestore (audio_separation_jobscollection) for multi-instance scalingnomadkaraoke-audio-separator-outputsbucket) so any instance can serve downloadsthreading.Semaphore(1)to serialize GPU access per instance, preventing OOMgoogle-cloud-firestoreto DockerfileFixes the 300s HTTP timeout when ensemble presets take >5 minutes on large WAV files. Backward-compatible — old clients work with new server and vice versa.
Test plan
status: submittedqueued→processing→completedtransitions@coderabbitai ignore
🤖 Generated with Claude Code