Skip to content

feat: combine data sources#465

Merged
antoncoding merged 2 commits intomasterfrom
refactor/envio-monarch-api-naming
Mar 22, 2026
Merged

feat: combine data sources#465
antoncoding merged 2 commits intomasterfrom
refactor/envio-monarch-api-naming

Conversation

@antoncoding
Copy link
Copy Markdown
Owner

@antoncoding antoncoding commented Mar 22, 2026

Summary by CodeRabbit

Release Notes

  • Refactor

    • Consolidated admin stats and market data to use a shared Monarch GraphQL endpoint as the primary data source, replacing separate internal infrastructure.
  • Documentation

    • Updated technical documentation to reflect market-detail and admin transaction data sourcing from the shared Monarch API.
    • Updated dashboard UI messaging to indicate reliance on the experimental shared Monarch API with potential cost implications.

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented Mar 22, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
monarch Ready Ready Preview, Comment Mar 22, 2026 8:37am

Request Review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 22, 2026

📝 Walkthrough

Walkthrough

Removed the proxied /api/admin/monarch-indexer route and migrated data sources to use the shared Monarch GraphQL API directly. Updated market-detail hooks, transaction fetching, and data source exports to reference Monarch API functions instead of Envio and proxied Indexer variants.

Changes

Cohort / File(s) Summary
API Route & Indexer Removal
app/api/admin/monarch-indexer/route.ts, src/data-sources/monarch-indexer/...
Removed the POST proxy route, deleted monarchIndexerFetcher, and removed the module's export surface that previously re-exported transaction functions and fetcher.
Data Source Consolidation
src/data-sources/monarch-api/index.ts, src/data-sources/monarch-api/market-detail.ts, src/data-sources/monarch-api/transactions.ts
Extended Monarch API exports for market-detail and transaction functions; renamed all fetchEnvio* functions to fetchMonarch*; switched transaction fetcher from monarchIndexerFetcher to monarchGraphqlFetcher.
Envio Fetcher Alias Removal
src/data-sources/envio/fetchers.ts
Deleted envioGraphqlFetcher alias that was re-exporting monarchGraphqlFetcher.
Market-Detail Hooks
src/hooks/useMarket*.ts, src/hooks/queries/market-detail-fallback.ts
Updated useMarketSuppliers, useMarketBorrowers, useMarketSupplies, useMarketBorrows, and useMarketLiquidations to use Monarch API as primary source; updated MarketDetailProvider type from 'envio' to 'monarch-api'.
Transaction Hook & Auth
src/hooks/useMonarchTransactions.ts, src/stores/useAdminAuth.ts
Updated imports and source references in transaction hook; clarified auth documentation in auth store.
Documentation & UI Text
app/admin/stats-v2/page.tsx, docs/TECHNICAL_OVERVIEW.md, src/features/admin-v2/...
Updated header, technical overview, and UI text to reference "shared Monarch API" and updated data-fetching paths.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • PR #322: Directly reverses and replaces the monarch-indexer API proxy introduced in that PR by removing the route and indexer module entirely.
  • PR #117: Modifies the same useMarketLiquidations hook data path and liquidation transaction fetching.
  • PR #302: Updates the same market-detail hooks (useMarketBorrowers, useMarketSuppliers) that are being switched to Monarch API here.

Suggested labels

feature request

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'feat: combine data sources' is vague and doesn't accurately reflect the main changes, which involve consolidating Monarch API as the primary data source and removing the Envio/Indexer abstraction layer. Consider a more specific title like 'refactor: consolidate to Monarch API as primary data source' or 'refactor: remove Envio/Indexer indirection' to better convey the actual scope of changes.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/envio-monarch-api-naming

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

CodeRabbit can use TruffleHog to scan for secrets in your code with verification capabilities.

Add a TruffleHog config file (e.g. trufflehog-config.yml, trufflehog.yml) to your project to customize detectors and scanning behavior. The tool runs only when a config file is present.

Copy link
Copy Markdown
Collaborator

@starksama starksama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed on latest head.

I spot-checked the actual refactor changes (Envio -> Monarch API naming/data-source consolidation), fallback provider labels, and admin-v2 data-source wiring.

Validation:

  • pnpm lint:check ✅
  • pnpm typecheck ✅ (after clearing stale local .next generated route types from deleted API file)

LGTM from my side.

@coderabbitai coderabbitai Bot added the feature request Specific feature ready to be implemented label Mar 22, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
src/data-sources/monarch-api/market-detail.ts (2)

209-215: ⚠️ Potential issue | 🟠 Major

Bound these Monarch requests.

monarchGraphqlFetcher accepts a signal, but none of these new calls pass one. If Monarch stalls, runMarketDetailFallback never reaches Morpho API or subgraph, so the hook can hang instead of failing over. Please add a per-request timeout and pass its abort signal into each fetch.

As per coding guidelines: Network fetches should use bounded request timeouts that account for fallback coverage; any fallback path used for first paint must preserve market completeness (no truncated first: 1000 fallback).

Also applies to: 237-243, 262-268, 282-288, 302-307

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/data-sources/monarch-api/market-detail.ts` around lines 209 - 215, The
monarchGraphqlFetcher calls (e.g.,
monarchGraphqlFetcher<MonarchSuppliersPageResponse>(envioSuppliersPageQuery,
{...})) are missing an AbortSignal, so stalled Monarch requests can block the
fallback; wrap each call with a per-request AbortController using a bounded
timeout (use a shared constant like MONARCH_REQUEST_TIMEOUT_MS), pass
controller.signal into the monarchGraphqlFetcher options, and clear the
timeout/abort on completion; apply the same pattern to the other similar calls
referenced (the calls around the other ranges) so the runMarketDetailFallback
path can reliably fail-over when Monarch is slow.

137-150: ⚠️ Potential issue | 🟠 Major

Stop scanning the full participant set before slicing.

fetchMonarchMarketSuppliers and fetchMonarchMarketBorrowers still walk every 1000-row page via scanAllPages() and only then slice(). On large markets, page 1 now pays for the whole participant list on the shared endpoint, which makes latency and API cost scale with total participant count. This path needs provider-boundary pagination, or an explicit fail-closed/unknown-total mode, instead of full-history scans here.

Based on learnings: Provider fallbacks for market details must page at the provider boundary or fail closed with typed source/network errors instead of fetching full history and slicing client-side or returning empty success on missing subgraph configuration.

Also applies to: 199-224, 227-254, 320-338

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/data-sources/monarch-api/market-detail.ts` around lines 137 - 150, The
current scanAllPages implementation (used by fetchMonarchMarketSuppliers and
fetchMonarchMarketBorrowers) fetches the entire participant set page-by-page
before any slicing, causing provider-side full scans; change the flow to
paginate at the provider boundary by adding a maxResults or limit parameter to
scanAllPages and to the provider fetchPage calls (and/or add a stop condition
inside scanAllPages to return as soon as collectedItems.length >=
requestedLimit) so you never request beyond what the caller needs; additionally
add a fail-closed/unknown-total mode where, if the provider cannot honor bounded
pagination, the functions
(fetchMonarchMarketSuppliers/fetchMonarchMarketBorrowers) return a typed error
indicating the provider/network limitation instead of silently returning
truncated/empty results, and keep MONARCH_SCAN_BATCH_SIZE only as the internal
page size while honoring the upstream max limit to avoid full-history scans.
src/data-sources/monarch-api/transactions.ts (1)

102-133: ⚠️ Potential issue | 🟠 Major

Do not silently truncate the ALL range at 50 pages.

Both loops stop after MAX_PAGES, but the dashboard still offers ALL. Once either collection grows past 50 * limit, the charts and totals undercount with no signal. Either page until exhaustion or throw when the cap is hit so this fails closed instead of showing partial stats.

Possible fail-closed guard
   for (let page = 0; page < MAX_PAGES; page++) {
     const supplies = await fetchSuppliesPage(frozenTimeRange, limit, suppliesOffset);
     allSupplies.push(...supplies);
     if (supplies.length < limit) break;
+    if (page === MAX_PAGES - 1) {
+      throw new Error(`Monarch supplies pagination exceeded ${MAX_PAGES} pages`);
+    }
     suppliesOffset += limit;
   }
@@
   for (let page = 0; page < MAX_PAGES; page++) {
     const withdraws = await fetchWithdrawsPage(frozenTimeRange, limit, withdrawsOffset);
     allWithdraws.push(...withdraws);
     if (withdraws.length < limit) break;
+    if (page === MAX_PAGES - 1) {
+      throw new Error(`Monarch withdraw pagination exceeded ${MAX_PAGES} pages`);
+    }
     withdrawsOffset += limit;
   }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/data-sources/monarch-api/transactions.ts` around lines 102 - 133, The
loops in fetchMonarchTransactions currently stop silently after MAX_PAGES which
undercounts "ALL" ranges; modify the paging logic in fetchMonarchTransactions so
it fails closed instead of truncating: when iterating with fetchSuppliesPage and
fetchWithdrawsPage, detect if you reached MAX_PAGES and the last page returned
length === limit (meaning more pages likely exist) and throw a descriptive error
(e.g., "Monarch transactions page cap reached") rather than returning partial
results; apply the same check for both the supplies loop and the withdraws loop
(referencing suppliesOffset/withdrawsOffset, MAX_PAGES, fetchSuppliesPage,
fetchWithdrawsPage, and fetchMonarchTransactions) so callers can handle or
surface the failure.
🧹 Nitpick comments (2)
src/hooks/useMarketBorrows.ts (1)

102-102: Incomplete dependency array in useEffect.

Same issue as useMarketSupplies.ts - missing marketId, loanAssetId, network, minAssets, pageSize from deps.

Suggested fix
-  }, [page, data, queryClient, queryFn]);
+  }, [page, data, queryClient, queryFn, marketId, loanAssetId, network, minAssets, pageSize]);
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/hooks/useMarketBorrows.ts` at line 102, The useEffect dependency array in
the hook useMarketBorrows is missing several props (marketId, loanAssetId,
network, minAssets, pageSize) which can cause stale closures; update the
dependency array used in the effect inside useMarketBorrows (the effect that
currently lists [page, data, queryClient, queryFn]) to include marketId,
loanAssetId, network, minAssets and pageSize so the effect reruns when any of
those inputs change, keeping behavior consistent with useMarketSupplies; ensure
you keep existing dependencies (page, data, queryClient, queryFn) and only add
the listed identifiers.
src/hooks/useMarketSupplies.ts (1)

102-102: Incomplete dependency array in useEffect.

The useEffect dependency array is missing marketId, loanAssetId, network, minAssets, and pageSize. Compare with useMarketBorrowers.ts line 145 which includes all deps. Works via closure but may cause stale prefetches if those values change mid-render.

Suggested fix
-  }, [page, data, queryClient, queryFn]);
+  }, [page, data, queryClient, queryFn, marketId, loanAssetId, network, minAssets, pageSize]);
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/hooks/useMarketSupplies.ts` at line 102, The useEffect in
useMarketSupplies.ts has an incomplete dependency array (currently [page, data,
queryClient, queryFn]) which can lead to stale prefetches; update the dependency
array used by the useEffect that performs prefetching to include marketId,
loanAssetId, network, minAssets, and pageSize in addition to page, data,
queryClient, and queryFn so the effect re-runs when any of those values change
(mirror the dependencies used in useMarketBorrowers.ts line ~145); ensure you
reference the same variables used inside the effect and avoid adding non-stable
references unless memoized.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/data-sources/monarch-api/transactions.ts`:
- Around line 11-12: The admin stats read path currently calls
monarchGraphqlFetcher from client pages (e.g., used by PasswordGate) — move that
logic to a server-side API/route or a server action (create a server-only
handler such as getAdminStatsServer or /api/admin/stats) that calls
monarchGraphqlFetcher using a server-only env var (do NOT use NEXT_PUBLIC_*),
applies a bounded AbortController timeout on the upstream fetch, and returns
only the necessary data to the client; update all referenced client calls (the
blocks around lines 72-94 and 102-133) to call this server endpoint instead of
invoking monarchGraphqlFetcher directly.

---

Outside diff comments:
In `@src/data-sources/monarch-api/market-detail.ts`:
- Around line 209-215: The monarchGraphqlFetcher calls (e.g.,
monarchGraphqlFetcher<MonarchSuppliersPageResponse>(envioSuppliersPageQuery,
{...})) are missing an AbortSignal, so stalled Monarch requests can block the
fallback; wrap each call with a per-request AbortController using a bounded
timeout (use a shared constant like MONARCH_REQUEST_TIMEOUT_MS), pass
controller.signal into the monarchGraphqlFetcher options, and clear the
timeout/abort on completion; apply the same pattern to the other similar calls
referenced (the calls around the other ranges) so the runMarketDetailFallback
path can reliably fail-over when Monarch is slow.
- Around line 137-150: The current scanAllPages implementation (used by
fetchMonarchMarketSuppliers and fetchMonarchMarketBorrowers) fetches the entire
participant set page-by-page before any slicing, causing provider-side full
scans; change the flow to paginate at the provider boundary by adding a
maxResults or limit parameter to scanAllPages and to the provider fetchPage
calls (and/or add a stop condition inside scanAllPages to return as soon as
collectedItems.length >= requestedLimit) so you never request beyond what the
caller needs; additionally add a fail-closed/unknown-total mode where, if the
provider cannot honor bounded pagination, the functions
(fetchMonarchMarketSuppliers/fetchMonarchMarketBorrowers) return a typed error
indicating the provider/network limitation instead of silently returning
truncated/empty results, and keep MONARCH_SCAN_BATCH_SIZE only as the internal
page size while honoring the upstream max limit to avoid full-history scans.

In `@src/data-sources/monarch-api/transactions.ts`:
- Around line 102-133: The loops in fetchMonarchTransactions currently stop
silently after MAX_PAGES which undercounts "ALL" ranges; modify the paging logic
in fetchMonarchTransactions so it fails closed instead of truncating: when
iterating with fetchSuppliesPage and fetchWithdrawsPage, detect if you reached
MAX_PAGES and the last page returned length === limit (meaning more pages likely
exist) and throw a descriptive error (e.g., "Monarch transactions page cap
reached") rather than returning partial results; apply the same check for both
the supplies loop and the withdraws loop (referencing
suppliesOffset/withdrawsOffset, MAX_PAGES, fetchSuppliesPage,
fetchWithdrawsPage, and fetchMonarchTransactions) so callers can handle or
surface the failure.

---

Nitpick comments:
In `@src/hooks/useMarketBorrows.ts`:
- Line 102: The useEffect dependency array in the hook useMarketBorrows is
missing several props (marketId, loanAssetId, network, minAssets, pageSize)
which can cause stale closures; update the dependency array used in the effect
inside useMarketBorrows (the effect that currently lists [page, data,
queryClient, queryFn]) to include marketId, loanAssetId, network, minAssets and
pageSize so the effect reruns when any of those inputs change, keeping behavior
consistent with useMarketSupplies; ensure you keep existing dependencies (page,
data, queryClient, queryFn) and only add the listed identifiers.

In `@src/hooks/useMarketSupplies.ts`:
- Line 102: The useEffect in useMarketSupplies.ts has an incomplete dependency
array (currently [page, data, queryClient, queryFn]) which can lead to stale
prefetches; update the dependency array used by the useEffect that performs
prefetching to include marketId, loanAssetId, network, minAssets, and pageSize
in addition to page, data, queryClient, and queryFn so the effect re-runs when
any of those values change (mirror the dependencies used in
useMarketBorrowers.ts line ~145); ensure you reference the same variables used
inside the effect and avoid adding non-stable references unless memoized.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 823281d6-6326-495e-b45d-37433845a850

📥 Commits

Reviewing files that changed from the base of the PR and between d7d8b9a and df8326c.

📒 Files selected for processing (19)
  • app/admin/stats-v2/page.tsx
  • app/api/admin/monarch-indexer/route.ts
  • docs/TECHNICAL_OVERVIEW.md
  • src/data-sources/envio/fetchers.ts
  • src/data-sources/monarch-api/index.ts
  • src/data-sources/monarch-api/market-detail.ts
  • src/data-sources/monarch-api/transactions.ts
  • src/data-sources/monarch-indexer/fetchers.ts
  • src/data-sources/monarch-indexer/index.ts
  • src/features/admin-v2/components/password-gate.tsx
  • src/features/admin-v2/index.ts
  • src/hooks/queries/market-detail-fallback.ts
  • src/hooks/useMarketBorrowers.ts
  • src/hooks/useMarketBorrows.ts
  • src/hooks/useMarketLiquidations.ts
  • src/hooks/useMarketSuppliers.ts
  • src/hooks/useMarketSupplies.ts
  • src/hooks/useMonarchTransactions.ts
  • src/stores/useAdminAuth.ts
💤 Files with no reviewable changes (4)
  • src/data-sources/envio/fetchers.ts
  • src/data-sources/monarch-indexer/fetchers.ts
  • src/data-sources/monarch-indexer/index.ts
  • app/api/admin/monarch-indexer/route.ts

Comment on lines +11 to 12
import { monarchGraphqlFetcher } from './fetchers';

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Keep the admin stats read path behind a server-side reader.

app/admin/stats-v2/page.tsx is still a client page, and monarchGraphqlFetcher sends this GraphQL call straight from the browser. That makes PasswordGate UI-only: the underlying admin query and shared API spend are no longer protected by the old server proxy. Please route this through a server action or route that uses a server-only key and a bounded timeout.

Based on learnings: Server-side Monarch proxy routes must use server-only API key env vars and bounded AbortController timeouts on upstream fetches; do not let Monarch GraphQL or metrics calls hang indefinitely; do not reference NEXT_PUBLIC_* secrets in server authorization headers.

Also applies to: 72-94, 102-133

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/data-sources/monarch-api/transactions.ts` around lines 11 - 12, The admin
stats read path currently calls monarchGraphqlFetcher from client pages (e.g.,
used by PasswordGate) — move that logic to a server-side API/route or a server
action (create a server-only handler such as getAdminStatsServer or
/api/admin/stats) that calls monarchGraphqlFetcher using a server-only env var
(do NOT use NEXT_PUBLIC_*), applies a bounded AbortController timeout on the
upstream fetch, and returns only the necessary data to the client; update all
referenced client calls (the blocks around lines 72-94 and 102-133) to call this
server endpoint instead of invoking monarchGraphqlFetcher directly.

@antoncoding antoncoding merged commit dca6875 into master Mar 22, 2026
4 checks passed
@antoncoding antoncoding deleted the refactor/envio-monarch-api-naming branch March 22, 2026 08:50
@coderabbitai coderabbitai Bot mentioned this pull request Apr 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request Specific feature ready to be implemented

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants