Skip to content

feat: add cache control middleware and improve response handling#2565

Merged
baktun14 merged 9 commits intomainfrom
fix/caching-api-requests
Jan 26, 2026
Merged

feat: add cache control middleware and improve response handling#2565
baktun14 merged 9 commits intomainfrom
fix/caching-api-requests

Conversation

@baktun14
Copy link
Contributor

@baktun14 baktun14 commented Jan 24, 2026

Summary by CodeRabbit

  • New Features

    • Global response compression, a cache-control middleware, and per-route cache policies.
  • Performance

    • Short-lived in-memory caching for service lookups and optimized provider resolution to reduce latency.
  • Reliability

    • 10s request timeouts added for deployment and lease API calls.
  • Chores / Data

    • Added an addressReference height column and a batched backfill script.
  • Tests

    • Unit tests covering route cache configuration lookup.

✏️ Tip: You can customize this high-level summary in your review settings.

@baktun14 baktun14 requested a review from a team as a code owner January 24, 2026 01:15
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 24, 2026

📝 Walkthrough

Walkthrough

Adds route-level cache metadata and registry, a cache-control middleware and global compression, memoization on provider/deployment methods with providerMap optimizations, 10s HTTP client timeouts, and an indexer migration plus a batched backfill script.

Changes

Cohort / File(s) Summary
Route cache registry & API
apps/api/src/core/lib/create-route/create-route.ts
Added CacheConfig and ExtendedRouteConfig, an in-memory cacheConfigRegistry, getCacheConfig(path), and updated createRoute to register route cache metadata (path pattern → config).
Cache-control middleware & app wiring
apps/api/src/middlewares/cacheControlMiddleware.ts, apps/api/src/rest-app.ts
New cacheControlMiddleware that sets Cache-Control for GET 200 responses using getCacheConfig(req.routePath) (preserves existing header if present); registered globally. Added global compress() middleware.
Service memoization & provider lookup
apps/api/src/deployment/services/deployment-reader/deployment-reader.service.ts, apps/api/src/provider/services/provider/provider.service.ts
Added @Memoize decorators to public methods (TTLs 30–60s), introduced providerMap to replace linear provider lookups in deployment reader, and changed getProviderList signature to getProviderList(trial = false).
Route-level cache additions
apps/api/src/dashboard/routes/..., apps/api/src/gpu/routes/gpu.router.ts, apps/api/src/deployment/routes/deployments/deployments.router.ts, apps/api/src/provider/routes/providers/providers.router.ts, apps/api/src/proposal/routes/proposals/proposals.router.ts, apps/api/src/template/routes/templates.templates.router.ts, apps/api/src/validator/routes/validators.validators.router.ts
Added cache configs (various maxAge and staleWhileRevalidate) to many GET routes via createRoute; no handler logic changes.
HTTP client timeouts
packages/http-sdk/src/deployment/deployment-http.service.ts, packages/http-sdk/src/lease/lease-http.service.ts
Added timeout: 10000 to relevant HTTP requests.
Minor call-site signature change
apps/api/src/provider/controllers/provider/provider.controller.ts
Updated caller to pass boolean trial to ProviderService.getProviderList (signature changed).
Indexer migration & batched backfill
apps/indexer/drizzle/0005_add_height_to_address_reference.sql, apps/indexer/drizzle/meta/_journal.json, apps/indexer/scripts/backfill-address-reference-height.sql
Added height column to addressReference, adjusted two journal timestamps, removed inline backfill from migration, and added a separate batched backfill SQL script for post-migration execution.
Tests for cache registry
apps/api/src/core/lib/create-route/create-route.spec.ts
Added unit tests verifying getCacheConfig resolves static and parameterized paths and returns undefined when absent.

Sequence Diagram(s)

sequenceDiagram
  participant Client as Client
  participant App as HonoApp
  participant Compress as CompressMiddleware
  participant CacheMW as CacheControlMiddleware
  participant Registry as CacheConfigRegistry
  participant Auth as AuthCheck
  participant Handler as RouteHandler

  Client->>App: GET /v1/...
  App->>Compress: run compress middleware
  Compress-->>App: pass-through
  App->>CacheMW: invoke cache-control middleware
  CacheMW->>Registry: getCacheConfig(req.routePath)
  Registry-->>CacheMW: CacheConfig | undefined
  CacheMW->>Auth: determine authentication (header / JWT)
  Auth-->>CacheMW: authenticated? true|false
  CacheMW->>Handler: call downstream handler
  Handler-->>CacheMW: Response (200)
  CacheMW-->>App: attach/modify Cache-Control header (public/private, max-age, swr)
  App-->>Client: Response (compressed, with Cache-Control)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

🐰 I hop through routes with eager paws,
I tuck in TTLs and mend the laws,
Timeouts hum and compression sings,
Cache-control flutters on nimble wings,
A tiny rabbit celebrates these things.

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: introducing cache control middleware and optimizing response handling through caching mechanisms across multiple services and routes.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link

codecov bot commented Jan 24, 2026

Codecov Report

❌ Patch coverage is 95.65217% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.86%. Comparing base (5f7a629) to head (f4ed5f5).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/api/src/middlewares/cacheControlMiddleware.ts 91.66% 2 Missing ⚠️

❌ Your project status has failed because the head coverage (79.38%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2565      +/-   ##
==========================================
+ Coverage   50.78%   50.86%   +0.07%     
==========================================
  Files        1069     1070       +1     
  Lines       29735    29793      +58     
  Branches     6565     6580      +15     
==========================================
+ Hits        15102    15154      +52     
- Misses      14221    14239      +18     
+ Partials      412      400      -12     
Flag Coverage Δ *Carryforward flag
api 79.38% <95.65%> (+0.06%) ⬆️
deploy-web 31.46% <ø> (ø)
log-collector 75.35% <ø> (ø)
notifications 87.94% <ø> (ø)
provider-console 81.48% <ø> (ø) Carriedforward from 9fb55c2
provider-proxy 84.35% <ø> (ø) Carriedforward from 9fb55c2

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
apps/api/src/core/lib/create-route/create-route.ts 100.00% <100.00%> (ø)
...ard/routes/dashboard-data/dashboard-data.router.ts 100.00% <ø> (ø)
...dashboard/routes/market-data/market-data.router.ts 100.00% <ø> (ø)
...routes/network-capacity/network-capacity.router.ts 100.00% <ø> (ø)
...eployment/routes/deployments/deployments.router.ts 94.52% <ø> (ø)
...ces/deployment-reader/deployment-reader.service.ts 82.35% <100.00%> (+0.35%) ⬆️
apps/api/src/gpu/routes/gpu.router.ts 95.65% <ø> (ø)
.../src/proposal/routes/proposals/proposals.router.ts 100.00% <ø> (ø)
...ovider/controllers/provider/provider.controller.ts 81.81% <100.00%> (ø)
.../src/provider/routes/providers/providers.router.ts 96.15% <ø> (ø)
... and 5 more

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In `@apps/api/src/middlewares/cacheControlMiddleware.ts`:
- Around line 27-44: The cacheControlMiddleware currently marks matched routes
as "public" and unconditionally overwrites Cache-Control, which can expose
authenticated responses; update cacheControlMiddleware to first check if a
Cache-Control header already exists on c.res and leave it untouched if present,
and when deciding directives for a matched config, detect whether the request is
authenticated (inspect the request/auth context used by your auth layer—e.g.,
presence of c.req.headers.authorization, c.req.user, or the same auth flag your
handlers rely on) and if authenticated, do not include "public" (use "private"
or "no-store" based on sensitivity) so user-specific responses are not cached
publicly; keep using CACHE_CONFIGS for maxAge and staleWhileRevalidate but guard
composition of directives accordingly and only call c.header("Cache-Control",
...) when you have a safe header to set.

In `@apps/api/src/provider/services/provider/provider.service.ts`:
- Around line 139-141: The Memoize cache is colliding because getProviderList({
trial }) accepts an object and the Memoize helper only hashes primitives; ensure
the cache key includes the trial boolean to avoid returning wrong results. Fix
by either changing the method signature to accept a primitive (e.g.,
getProviderList(trial = false): Promise<ProviderList[]>) so Memoize receives a
simple boolean, or update the Memoize usage to provide an explicit
cache-key/resolver that serializes the { trial } object (or returns `trial`),
keeping the call to providerRepository.getWithAttributesAndAuditors({ trial })
unchanged.
🧹 Nitpick comments (1)
packages/http-sdk/src/lease/lease-http.service.ts (1)

75-86: Consider a shared timeout constant.

Inline timeouts are harder to tune and keep consistent across services. A shared constant (or HttpClient default) makes future adjustments safer.

♻️ Example refactor
+const DEFAULT_HTTP_TIMEOUT_MS = 10000;
+
 export class LeaseHttpService {
   constructor(private readonly httpClient: HttpClient) {}
 
   public async list({ owner, dseq, state, pagination }: LeaseListParams): Promise<RestAkashLeaseListResponse> {
     return extractData(
       await this.httpClient.get<RestAkashLeaseListResponse>("/akash/market/v1beta5/leases/list", {
         params: {
           "filters.owner": owner,
           "filters.dseq": dseq,
           "filters.state": state,
           "pagination.limit": pagination?.limit,
           "pagination.key": pagination?.key
         },
-        timeout: 10000
+        timeout: DEFAULT_HTTP_TIMEOUT_MS
       })
     );
   }

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@apps/indexer/scripts/backfill-address-reference-height.sql`:
- Around line 8-43: The DO $$ anonymous block contains COMMIT (invalid inside
DO), so convert the anonymous block into a stored procedure (e.g., CREATE
PROCEDURE backfill_address_reference_height() LANGUAGE plpgsql AS $$ ...) and
move the loop and COMMIT into that procedure where transaction control is
allowed; keep the UPDATE logic that targets "addressReference" ar and joins
"transaction" t to set ar.height from t.height for NULLs in batches, retain the
GET DIAGNOSTICS rows_updated handling and RAISE NOTICE lines, then CREATE the
procedure and invoke it with CALL backfill_address_reference_height(); this
removes the invalid COMMIT-in-DO error while preserving the batching behavior
for addressReference.height.
🧹 Nitpick comments (1)
apps/indexer/scripts/backfill-address-reference-height.sql (1)

23-27: Consider adding ORDER BY for deterministic batch selection.

The subquery selecting batch IDs has no ORDER BY, so row selection order depends on physical storage. While functionally correct, adding an explicit order (e.g., by id) makes behavior predictable and debugging easier.

♻️ Suggested improvement
        SELECT ar2.id
        FROM "addressReference" ar2
        WHERE ar2.height IS NULL
+       ORDER BY ar2.id
        LIMIT batch_size

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@apps/indexer/scripts/backfill-address-reference-height.sql`:
- Around line 20-43: The loop can spin forever if some addressReference rows
have no matching transaction; modify the UPDATE in the backfill loop (affecting
addressReference and transaction logic) to only target rows with an existing
join (e.g., use EXISTS or an INNER JOIN against "transaction" t in the WHERE so
orphan rows are not selected by the subquery), and add a safety guard by
introducing a max_iterations counter (e.g., max_iterations and batch_count) and
change the EXIT condition to EXIT WHEN rows_updated = 0 OR batch_count >=
max_iterations so the loop terminates even if orphans remain; reference symbols:
addressReference, transaction, rows_updated, batch_size, batch_count,
max_iterations.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@apps/api/src/core/lib/create-route/create-route.spec.ts`:
- Line 3: Change the root test suite to use the function's runtime name for
refactor safety: replace the hardcoded string "create-route" with
createRoute.name in the top-level describe call (the suite covering createRoute
tests) so the suite title updates automatically when the function is renamed.

@baktun14 baktun14 merged commit 2d922ba into main Jan 26, 2026
67 of 68 checks passed
@baktun14 baktun14 deleted the fix/caching-api-requests branch January 26, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments