Skip to content

Document Preload Optimization#685

Merged
DZakh merged 3 commits intomainfrom
dz/preload
Aug 7, 2025
Merged

Document Preload Optimization#685
DZakh merged 3 commits intomainfrom
dz/preload

Conversation

@DZakh
Copy link
Member

@DZakh DZakh commented Aug 7, 2025

Summary by CodeRabbit

  • New Features

    • Introduced Preload Optimization for improved indexer performance, including new documentation and configuration options.
    • Added comprehensive documentation for the Effect API, enabling efficient external calls with batching and caching.
    • Enhanced querying capabilities by allowing entity retrieval by indexed or derived fields.
  • Improvements

    • Updated guides and tutorials to recommend Node.js v22 or newer.
    • Refined event handler documentation to reflect preload optimization and new best practices.
    • Added Windows-specific setup guidance and clarified Docker requirements.
  • Deprecations

    • Deprecated the Loaders feature and updated documentation to direct users to Preload Optimization.
  • Style

    • Simplified article link styling for improved readability.
  • Other

    • Updated network tier icons and supported network tables for clarity.
    • Made minor formatting and documentation corrections across multiple files.

@vercel
Copy link

vercel bot commented Aug 7, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
envio-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 7, 2025 0:18am

@DZakh DZakh requested a review from moose-code August 7, 2025 10:54
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Aug 7, 2025

Walkthrough

This update introduces and documents the new Preload Optimization feature for HyperIndex, deprecating the previous Loaders pattern. It revises multiple guides, tutorials, and configuration examples to recommend preload optimization, updates minimum Node.js requirements to v22, and enhances documentation for the Effect API and related querying capabilities. Minor formatting, sidebar, and style adjustments are also included.

Changes

Cohort / File(s) Change Summary
Preload Optimization Feature & Migration
docs/HyperIndex/Advanced/preload-optimization.md, docs/HyperIndex/Advanced/loaders.md, docs/HyperIndex/Guides/configuration-file.mdx, docs/HyperIndex/Guides/contract-state.md, docs/HyperIndex/Guides/event-handlers.mdx, docs/HyperIndex/Guides/ipfs.md, docs/HyperIndex/Tutorials/price-data.md
Added new Preload Optimization documentation and config option; deprecated Loaders; updated guides, configuration, and code examples to use preload optimization and Effect API; removed loader-based patterns.
Effect API Documentation
docs/HyperIndex/Advanced/effect-api.md, docs/HyperIndex/Guides/contract-state.md, docs/HyperIndex/Guides/event-handlers.mdx, docs/HyperIndex/Guides/ipfs.md, docs/HyperIndex/migration-guide.md, blog/2025-07-30-dev-update-july-2025.md
Introduced detailed Effect API documentation; updated guides and migration docs to reference Effect API for external calls and caching; corrected links and code snippets.
Node.js Version Updates
docs/HyperIndex-LLM/hyperindex-complete.mdx, docs/HyperIndex/Troubleshoot/common-issues.md, docs/HyperIndex/Tutorials/greeter-tutorial.md, docs/HyperIndex/Tutorials/tutorial-erc20-token-transfers.md, docs/HyperIndex/Tutorials/tutorial-indexing-fuel.md, docs/HyperIndex/Tutorials/tutorial-op-bridge-deposits.md, docs/HyperIndex/getting-started.md
Updated minimum recommended Node.js version from v18 to v22 in prerequisites and troubleshooting sections.
Sidebar and Navigation
sidebarsHyperIndex.js
Added Preload Optimization and Effect API to sidebar; reordered and commented out deprecated/migrated entries for clarity.
Network Tier and Supported Networks Documentation
docs/HyperIndex/supported-networks/megaeth-testnet.md, docs/HyperSync/hypersync-supported-networks.md, docs/HyperSync-LLM/hypersync-complete.mdx, docs/HyperSync/HyperRPC/hyperrpc-supported-networks.md
Updated network tier icons for Megaeth Testnet; removed trace support checkmark for Eth Traces; added HyperRPC supported networks section and query builder references.
General Documentation and Formatting
docs/HyperIndex/getting-started.md, docs/HyperIndex/overview.md, docs/HyperIndex/migration-guide.md, blog/2025-07-30-dev-update-july-2025.md
Improved clarity, reworded Docker/WSL notes, added Windows user guidance, and made minor formatting and link corrections.
CSS and Styling
src/css/custom.css
Simplified link styling for articles, switching from border-bottom to underline and removing hover effects.
Code and Script Formatting
scripts/update-endpoints.js
Fixed syntax, semicolons, and path errors; improved code formatting and template string readability.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Handler
    participant DB/ExternalAPI

    User->>Handler: Triggers event (Preload Optimization enabled)
    Handler->>DB/ExternalAPI: Preload Phase (batched/cached reads, external calls)
    DB/ExternalAPI-->>Handler: Returns preloaded data
    Handler->>Handler: Processing Phase (sequential, uses cached data)
    Handler->>DB/ExternalAPI: Writes/updates (if any)
    Handler-->>User: Processing complete
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested labels

codex

Suggested reviewers

  • moose-code

Poem

🐇✨
A preload hop, a loader drop,
New docs and guides to make you stop—
And ponder twice, for double run,
With Effect API, the work gets done.
Node v22, the bar is set,
HyperIndex faster yet!
—Your code rabbit, with a hop and a bet.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch dz/preload

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Comment on lines 205 to +207
article a {
color: #000000;
text-decoration: none;
border-bottom: 1px solid transparent;
transition: border-bottom-color 0.2s ease;
}

article a:hover {
border-bottom-color: #000000;
text-decoration: underline;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DenhamPreen I returned it back, since links were not visible at all.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🔭 Outside diff range comments (5)
docs/HyperSync/hypersync-supported-networks.md (1)

50-51: Known-broken Chainweb Testnet entries should be removed

Long-term-memory note (2025-07-29) confirms both Chainweb Testnet 20 (5920) and 21 (5921) return 404 on all HyperSync endpoints. Keeping them in the “supported” list is misleading documentation. Please drop these two rows or add a deprecation notice.

docs/HyperSync/HyperRPC/hyperrpc-url-endpoints.md (1)

42-44: Remove Chainweb Testnet 20 & 21 – endpoints 404

As noted in internal learnings, these RPC endpoints are still returning HTTP 404. Until fixed, they should not be listed as “supported”.

docs/HyperIndex/Guides/event-handlers.mdx (1)

484-489: await inside a non-async function – will throw at runtime

The resolver passed to experimental_createEffect uses await fetch but the arrow function is not marked async, resulting in a SyntaxError.

-  ({ input }) => {
+  async ({ input }) => {
docs/HyperIndex/Guides/ipfs.md (2)

145-151: Environment-variable naming drifts from later snippet

Here the fallback list uses process.env.PINATA_IPFS_GATEWAY, while the “Best Practices” section (Lines 252-257) switches to process.env.PAID_IPFS_GATEWAY.

This inconsistency will trip up copy-paste adoption.

-...(process.env.PINATA_IPFS_GATEWAY ? [process.env.PINATA_IPFS_GATEWAY] : []),
+...(process.env.PAID_IPFS_GATEWAY ? [process.env.PAID_IPFS_GATEWAY] : []),

and update the earlier explanatory text accordingly.


252-257: Keep env-var name consistent with previous example

See earlier comment on Lines 145-151. Align the variable name in both code blocks to avoid mismatches.

♻️ Duplicate comments (3)
docs/HyperIndex/Tutorials/tutorial-op-bridge-deposits.md (1)

22-22: Same Node 22 prerequisite concern as noted in greeter tutorial

See prior comment for verification steps.

docs/HyperIndex/Tutorials/tutorial-indexing-fuel.md (1)

28-28: Same Node 22 prerequisite concern as noted in greeter tutorial

See prior comment for verification steps.

docs/HyperIndex/Tutorials/tutorial-erc20-token-transfers.md (1)

22-22: Same Node 22 prerequisite concern as noted in greeter tutorial

See prior comment for verification steps.

🧹 Nitpick comments (15)
docs/HyperIndex-LLM/hyperindex-complete.mdx (1)

67-69: Reduce copy-paste duplication in repeated “Prerequisites” sections.

The identical three-line list now appears six times. Consider extracting it into an MDX fragment (<Prereqs />) or an imported markdown partial so that future version bumps require one edit instead of six.

Also applies to: 1980-1982, 2177-2179, 2367-2369, 2687-2689

docs/HyperIndex/Troubleshoot/common-issues.md (1)

75-77: Small UX nit – keep nvm install + use on one line

Readers often copy-paste; collapsing into a single line reduces friction:

-nvm install 22
-nvm use 22
+nvm install 22 && nvm use 22
docs/HyperIndex/migration-guide.md (1)

202-202: Missing period after “etc.”

LanguageTool is right – add the period for correct Am-Eng punctuation.

-...to a queue etc for fetching external data
+...to a queue, etc. for fetching external data
docs/HyperIndex/getting-started.md (2)

12-14: Node 22 note – mention “current” vs “LTS”

Node 22 is current but not yet LTS. A quick parenthetical keeps newcomers from wondering:

-**[Node.js](https://nodejs.org/en/download/current)** _(v22 or newer recommended)_
+**[Node.js](https://nodejs.org/en/download/current)** _(v22 (current) or newer recommended)_

16-21: Retain Yarn caveat for docs build system

Per team learning, the docs build still requires Yarn even though indexer examples use pnpm. Add a short reminder so Windows users installing WSL don’t trip:

> **Note:** Docker is required only if you plan to run your indexer locally. ...
+> **Docs build:** Commands such as `yarn build-llm` still rely on Yarn; pnpm will not work for documentation tasks.
src/css/custom.css (1)

205-208: Link styling change – hover state removed

The previous border-transition gave a clear affordance. With only a permanent underline, the hover state now offers no feedback. Consider re-adding a subtle color change for accessibility:

 article a {
   color: #000000;
   text-decoration: underline;
+  transition: color .15s ease;
 }
 
 article a:hover {
-  /* previously had border-bottom change – now nothing */
+  color: var(--ifm-color-primary);
 }
docs/HyperIndex/Guides/configuration-file.mdx (1)

205-214: Section looks good – minor clarity suggestion only

The new “Preload Optimization” option is clearly documented and the warning links are helpful. Consider explicitly stating that preload_handlers can co-exist with the other performance flags (e.g. unordered_multichain_mode) just to pre-empt configuration questions beginners often have.
Otherwise LGTM.

docs/HyperIndex/Advanced/effect-api.md (1)

21-30: Use S.schema(…) for the output definition to stay consistent with all other Sury examples

Everywhere else in the documentation the object shape is wrapped in S.schema. Keeping the pattern consistent avoids confusion and guarantees correct type inference.

-  output: {
-      description: S.string,
-      value: S.bigint,
-    },
+  output: S.schema({
+      description: S.string,
+      value: S.bigint,
+  }),
docs/HyperIndex/Advanced/preload-optimization.md (1)

156-166: Grammar / wording nit – replace “table run nature” and tighten sentence

The current phrasing is slightly awkward and appears to contain a typo.

-As mentioned above, the Preload Phase gives a lot of benefits for the event processing, but also it means that you must be aware of its table run nature:
+As mentioned above, the Preload Phase offers major performance benefits, but you must be aware of its **double-run** nature:
docs/HyperIndex/Guides/contract-state.md (1)

18-20: Avoid mixing “optimisation” (UK) and “optimization” (US) within the same doc

The bullet here uses the UK spelling while the remainder of the guide and linked docs use US spelling. Pick one variant project-wide for consistency.

-3. Learn about [Preload Optimisation](/docs/HyperIndex/preload-optimization) ...
+3. Learn about [Preload Optimization](/docs/HyperIndex/preload-optimization) ...
docs/HyperIndex/Advanced/loaders.md (1)

40-42: Heading punctuation hurts automatic anchor generation

## How It Works? ends with a question mark; Docusaurus strips the character, producing the anchor #how-it-works.
Your link on Line 42 already targets #how-it-works, so the heading should drop the “?” for consistency with other docs.

-## How It Works?
+## How It Works
docs/HyperIndex/Guides/event-handlers.mdx (1)

303-319: Minor wording polish

“Very large getWhere queries might cause memory overflows.” – the intensifier can be dropped for concise style.

- Very large `getWhere` queries might cause memory overflows.
+ Large `getWhere` queries might cause memory overflows.
docs/HyperIndex/Guides/ipfs.md (3)

123-127: Duplicate “Preload Optimization” disclaimer – consider consolidating

The warning about handlers running twice (Lines 123-126) is repeated almost verbatim later (Lines 293-295). Re-using the same call-out twice in one guide adds vertical noise without new information.

Diff suggestion:

-**Important!** Preload optimization makes your handlers run **twice**. …
-
-Learn how Preload Optimization works in a [dedicated guide](/docs/HyperIndex/preload-optimization). …
+> **Important – Preload optimisation runs handlers twice.**  
+> See the [dedicated guide](/docs/HyperIndex/preload-optimization) for details and the “double-run footgun”.

Then drop the later repetition.


132-141: Spelling / library name likely incorrect

The comment states:

// It uses Sury library to define the schema

Is the library actually called “Sury”?
Everywhere else in the project the runtime-schema package is referred to as S from “Suru” (or occasionally surya/zod). If “Sury” is a typo, fix it to avoid confusing readers.


273-277: Link still points to deprecated Loaders docs

The guidance now favours the Effect API, but the sentence still directs readers to
/docs/HyperIndex/loaders#persistence.

Recommend switching to the upcoming persistence section of the Effect API (or adding a momentary redirect) to prevent dead-end navigation.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 54951ed and 09b4fbd.

📒 Files selected for processing (22)
  • docs/HyperIndex-LLM/hyperindex-complete.mdx (6 hunks)
  • docs/HyperIndex/Advanced/effect-api.md (1 hunks)
  • docs/HyperIndex/Advanced/loaders.md (3 hunks)
  • docs/HyperIndex/Advanced/preload-optimization.md (1 hunks)
  • docs/HyperIndex/Guides/configuration-file.mdx (2 hunks)
  • docs/HyperIndex/Guides/contract-state.md (6 hunks)
  • docs/HyperIndex/Guides/event-handlers.mdx (6 hunks)
  • docs/HyperIndex/Guides/ipfs.md (3 hunks)
  • docs/HyperIndex/Troubleshoot/common-issues.md (1 hunks)
  • docs/HyperIndex/Tutorials/greeter-tutorial.md (1 hunks)
  • docs/HyperIndex/Tutorials/price-data.md (1 hunks)
  • docs/HyperIndex/Tutorials/tutorial-erc20-token-transfers.md (1 hunks)
  • docs/HyperIndex/Tutorials/tutorial-indexing-fuel.md (1 hunks)
  • docs/HyperIndex/Tutorials/tutorial-op-bridge-deposits.md (1 hunks)
  • docs/HyperIndex/getting-started.md (1 hunks)
  • docs/HyperIndex/migration-guide.md (3 hunks)
  • docs/HyperIndex/overview.md (0 hunks)
  • docs/HyperIndex/supported-networks/megaeth-testnet.md (1 hunks)
  • docs/HyperSync/HyperRPC/hyperrpc-url-endpoints.md (1 hunks)
  • docs/HyperSync/hypersync-supported-networks.md (2 hunks)
  • sidebarsHyperIndex.js (1 hunks)
  • src/css/custom.css (1 hunks)
💤 Files with no reviewable changes (1)
  • docs/HyperIndex/overview.md
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: the envio documentation build system requires yarn and does not work with pnpm, despite other parts ...
Learnt from: DenhamPreen
PR: enviodev/docs#673
File: LLM_DOCS_README.md:24-32
Timestamp: 2025-07-29T14:32:31.665Z
Learning: The Envio documentation build system requires Yarn and does not work with pnpm, despite other parts of the repository potentially using pnpm/pnpx. The LLM documentation commands (yarn build-llm, yarn start-llm, yarn consolidate-docs) must remain using yarn to function properly.

Applied to files:

  • docs/HyperIndex/Tutorials/tutorial-op-bridge-deposits.md
  • docs/HyperIndex/Tutorials/greeter-tutorial.md
  • docs/HyperIndex/Tutorials/tutorial-erc20-token-transfers.md
  • docs/HyperIndex/Tutorials/tutorial-indexing-fuel.md
  • docs/HyperIndex-LLM/hyperindex-complete.mdx
  • docs/HyperIndex/Troubleshoot/common-issues.md
  • docs/HyperIndex/getting-started.md
📚 Learning: chainweb testnet 20 (id: 5920) and chainweb testnet 21 (id: 5921) have non-functional hypersync endp...
Learnt from: DenhamPreen
PR: enviodev/docs#673
File: docs/HyperIndex/supported-networks/chainweb-testnet-21.md:14-16
Timestamp: 2025-07-29T14:30:03.677Z
Learning: Chainweb Testnet 20 (ID: 5920) and Chainweb Testnet 21 (ID: 5921) have non-functional HyperSync endpoints that return HTTP 404, indicating these networks are not properly supported and should be removed from documentation.

Applied to files:

  • docs/HyperIndex/supported-networks/megaeth-testnet.md
  • docs/HyperSync/hypersync-supported-networks.md
  • docs/HyperSync/HyperRPC/hyperrpc-url-endpoints.md
🪛 markdownlint-cli2 (0.17.2)
docs/HyperSync/hypersync-supported-networks.md

56-56: Bare URL used

(MD034, no-bare-urls)


56-56: Bare URL used

(MD034, no-bare-urls)


77-77: Bare URL used

(MD034, no-bare-urls)


77-77: Bare URL used

(MD034, no-bare-urls)

docs/HyperSync/HyperRPC/hyperrpc-url-endpoints.md

48-48: Bare URL used

(MD034, no-bare-urls)


48-48: Bare URL used

(MD034, no-bare-urls)

🪛 LanguageTool
docs/HyperIndex/Guides/contract-state.md

[uncategorized] ~113-~113: Do not mix variants of the same word (‘optimization’ and ‘optimisation’) within a single text.
Context: ... retrieved data Important! Preload optimization makes your handlers run twice. So i...

(EN_WORD_COHERENCY)


[uncategorized] ~115-~115: Do not mix variants of the same word (‘optimization’ and ‘optimisation’) within a single text.
Context: ...erIndex/effect-api). Learn how Preload Optimization works in a [dedicated guide](/docs/Hype...

(EN_WORD_COHERENCY)


[uncategorized] ~309-~309: Do not mix variants of the same word (‘optimization’ and ‘optimisation’) within a single text.
Context: ...le RPC request 2. Learn about Preload Optimization to make your indexer thousands of tim...

(EN_WORD_COHERENCY)

docs/HyperIndex/Advanced/preload-optimization.md

[style] ~158-~158: Consider using a synonym to be more concise.
Context: ...entioned above, the Preload Phase gives a lot of benefits for the event processing, but ...

(A_LOT_OF)

docs/HyperIndex/migration-guide.md

[style] ~202-~202: In American English, abbreviations like “etc.” require a period.
Context: ...in data too or send messages to a queue etc for fetching external data, you can fur...

(ETC_PERIOD)

docs/HyperIndex/Guides/event-handlers.mdx

[style] ~317-~317: As an alternative to the over-used intensifier ‘very’, consider replacing this phrase.
Context: ...) directive - Potential Memory Issues: Very large getWhere queries might cause memory o...

(EN_WEAK_ADJECTIVE)

🔇 Additional comments (15)
docs/HyperIndex/supported-networks/megaeth-testnet.md (1)

20-23: Tier icon update looks good

The change from “STONE 🪨” → “SILVER 🥈” is consistent with the parallel update in the HyperSync tables. No further action required.

docs/HyperSync/hypersync-supported-networks.md (2)

56-56: “Eth Traces” row now claims no trace support – verify naming or column value

The product name implies trace availability, yet the “Supports Traces” cell is now blank. Either:

  1. The service no longer exposes trace data – in which case the network should be renamed (e.g., “Ethereum Replay”) to avoid confusion; or
  2. Traces are still supported – restore the ✔️ to keep the table accurate.

Please confirm and update accordingly.


77-77: Megaeth Testnet tier change acknowledged

Icon switch to 🥈 brings HyperSync docs in line with HyperIndex. Looks correct.

docs/HyperSync/HyperRPC/hyperrpc-url-endpoints.md (1)

48-48: Inconsistent “Eth Traces” description vs. capabilities

Same ambiguity as in the HyperSync table: a network branded “Eth Traces” with the “Supports Traces” column empty is confusing. Either rename the row or reinstate the checkmark once functionality is confirmed.

docs/HyperIndex-LLM/hyperindex-complete.mdx (1)

67-67: Node 22 is not yet an LTS release – confirm this upgrade path before publishing.

Bumping the minimum to “v22 or newer” may surprise users who stay on the latest LTS (currently v20). Unless HyperIndex truly needs a Node-22-only feature, consider phrasing this as “v22 (current) — or the latest LTS” and add a short justification/link.
Otherwise, please double-check that build, runtime, and CI pipelines are already pinned to v22 to avoid breakages.

Also applies to: 1980-1980, 2177-2177, 2367-2367, 2687-2687, 7662-7662

docs/HyperIndex/Tutorials/greeter-tutorial.md (1)

30-30: Verify CLI compatibility with Node.js 22 across platforms

The package.json engines field ("node": ">=18.0") does permit Node 22, but you’ll still need to confirm the CLI and tutorial flow actually work on v22 before updating the docs. Please test envio@latest and all tutorial commands (pnpx envio init, pnpm dev, etc.) on a clean Node 22 installation across macOS, Linux, and Windows to ensure no ESM/CJS or other build issues arise.

Relevant location:

  • docs/HyperIndex/Tutorials/greeter-tutorial.md, line 30
docs/HyperIndex/migration-guide.md (1)

202-203: Validate new absolute link to Effect API

Changed link points to /docs/HyperIndex/effect-api. Ensure:

  1. The file actually lives at docs/HyperIndex/Advanced/effect-api.md.
  2. Docusaurus slug matches (slug: /effect-api) or readers will land on a 404.
sidebarsHyperIndex.js (1)

100-118: Ensure new sidebar entries resolve

Advanced/preload-optimization, Advanced/effect-api, and Guides/ipfs are added here. Run yarn build to confirm:

  • MD files exist with matching id/slug.
  • No duplicate sidebar IDs (previously Guides/ipfs only existed under Guides).
docs/HyperIndex/Advanced/loaders.md (2)

4-4: Sidebar label looks good

The new “(Deprecated)” suffix is clear and matches the page intent.


10-12: Please verify the slug in internal links

/docs/HyperIndex/preload-optimization is used three times here.
The new guide lives at docs/HyperIndex/Advanced/preload-optimization.md, whose default slug will render as /docs/HyperIndex/Advanced/preload-optimization unless front-matter overrides it.

Broken links in the sidebar break the build in strict-link mode – double-check the generated URL or add an explicit slug: front-matter entry in the target file.

docs/HyperIndex/Guides/event-handlers.mdx (4)

156-170: Cross-doc links need a quick sanity check

All bullets reference /docs/HyperIndex/preload-optimization#….
If the new preload guide lives under Advanced, the real path will be /docs/HyperIndex/Advanced/preload-optimization. Please confirm to avoid 404s.


288-301: Sample getWhere usage is clear

The example accurately demonstrates the new by-field query API. No issues spotted.


423-455: context.isPreload section reads well

The snippet clearly shows how to bypass heavyweight logic in the preload pass.


463-466: External-call guidance aligns with preload semantics

Good clarification that Effect API turns the double run into parallelised I/O.

docs/HyperIndex/Guides/ipfs.md (1)

42-57: Confirm preload_handlers placement & schema support

preload_handlers: true is introduced at the root of config.yaml.
If the configuration schema has not been bumped accordingly (e.g. via a version: 2 key or a documented minimum CLI version), this flag will be ignored and silently fall back to the default. Please verify that:

  1. The latest CLI / runtime you target already recognises this key at the root level.
  2. Older readers of the docs will not break when copy-pasting the snippet.

If the schema change is still behind a CLI release, consider adding a note or gating the option under an explicit version stanza.

@moose-code
Copy link
Member

Trying to think how we surface the "double run footgun" better.

its hidden under advanced section now. Its well documented but garunteed to catch people out.

Also would users now if they were say writing to kafka etc ... put this in effects api?

Also curious if there are really expensive string / math operations, running this twice might be slow, but in general not the case and its opt out so fine!

@moose-code
Copy link
Member

// To imitate the behavior of the loader,
// we can use context.isPreload to make next code run only once.
// Note: This is not required, but might be useful for CPU-intensive operations.
if (context.isPreload) {
return;
}

You are too smart, understand now

Copy link
Member

@moose-code moose-code left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really solid changes! Documented really nicely! Only thought is how we make users more aware of double run footgun. Have a feeling its unexpected and will trip lots of users up so needs to be super clear.

Maybe in codegen comments on envio init, the comment has a BEWARE etc explaining it briefly in handlers

@DZakh
Copy link
Member Author

DZakh commented Aug 7, 2025

Maybe in codegen comments on envio init, the comment has a BEWARE etc explaining it briefly in handlers

This sounds like a good suggestion.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🔭 Outside diff range comments (2)
docs/HyperIndex/Guides/ipfs.md (1)

145-151: Unify IPFS gateway env var and endpoint lists

Both code examples in docs/HyperIndex/Guides/ipfs.md use different environment-variable names and mismatched endpoint arrays, which can confuse readers. Update them to:

  • Use the same env var: PINATA_IPFS_GATEWAY
  • Include (or remove) the dedicated Pinata gateway URL consistently

Affected locations:

  • docs/HyperIndex/Guides/ipfs.md, lines 145–151
  • docs/HyperIndex/Guides/ipfs.md, lines 252–257

Suggested diff:

--- docs/HyperIndex/Guides/ipfs.md
@@ -145,7 +145,11 @@ const endpoints = [
-  ...(process.env.PINATA_IPFS_GATEWAY ? [process.env.PINATA_IPFS_GATEWAY] : []),
+  // Optional paid gateway (set in .env)
+  ...(process.env.PINATA_IPFS_GATEWAY ? [process.env.PINATA_IPFS_GATEWAY] : []),
+  // Dedicated Pinata gateway fallback
+  "https://gateway.pinata.cloud/ipfs",
   "https://cloudflare-ipfs.com/ipfs",
   "https://ipfs.io/ipfs",
 ];

@@ -252,7 +256,8 @@ const endpoints = [
-  ...(process.env.PAID_IPFS_GATEWAY ? [process.env.PAID_IPFS_GATEWAY] : []),
+  // Optional paid gateway (set in .env)
+  ...(process.env.PINATA_IPFS_GATEWAY ? [process.env.PINATA_IPFS_GATEWAY] : []),
   "https://gateway.pinata.cloud/ipfs",
   "https://cloudflare-ipfs.com/ipfs",
   "https://ipfs.io/ipfs",

If you’d rather omit the hard-coded Pinata endpoint, remove "https://gateway.pinata.cloud/ipfs" from both snippets—but keep the env var name consistent.

docs/HyperSync-LLM/hypersync-complete.mdx (1)

2045-2053: Network table inconsistencies & outdated endpoints

  1. External learning (PR #673) notes that Chainweb Testnet 20 (ID 5920) and Chainweb Testnet 21 (ID 5921) endpoints return 404. They are still listed here—please remove or annotate as unsupported.
  2. Eth Traces row (ID 1) duplicates the ID already used by “Ethereum Mainnet”, which will break copy-paste examples and tooling that key on network-id uniqueness.
  3. Trace-support column is blank for Eth Traces, yet its name implies trace availability. Clarify or rename.

Fixing these prevents users from hitting dead endpoints or ambiguous network IDs.

♻️ Duplicate comments (1)
docs/HyperSync-LLM/hypersync-complete.mdx (1)

368-374: Duplicate CTA – see earlier comment

Same bullet as Line 99—please remove or consolidate to avoid redundancy.

🧹 Nitpick comments (5)
docs/HyperIndex/Guides/ipfs.md (3)

123-126: Surface the “double-run footgun” earlier with a call-out block

You already mention the issue here, but it still appears deep in the guide. Most readers stop at the first working code sample (Line 42). Consider adding a short :::warning admonition right after the configuration YAML so users see it before they write their first handler.

No code change needed here; move or duplicate the warning block near Line 45.

Also applies to: 293-296


132-133: Fix typo: “Sury” → “S” (or correct library name)

-// It uses Sury library to define the schema
+// Schema defined with `S` (Surya) helper

Keeps terminology consistent with the import import { …, S } from "envio";.


45-45: Bare URL inside YAML trips markdown-lint

Line 45 is flagged by MD034. Wrapping the URL in angle brackets silences the linter without altering rendered output.

-    end_block: 12299114 # Optional: limit blocks for development
+    end_block: 12299114 # Optional: limit blocks for development

(Only needed if your CI pipeline enforces markdown rules.)

docs/HyperSync-LLM/hypersync-complete.mdx (2)

96-102: Redundant bullet in “Next Steps” section

The bullet “Build queries visually with our Intuitive Query Builder” is re-introduced later (Line 372). Repeating the same call-to-action twice in close proximity is noisy and may confuse readers.

- - **Build queries visually** with our Intuitive Query Builder

Consider keeping it in one “Next Steps” list only (preferably the first) and removing the duplicate.


932-935: Nice addition – but link missing

Great to surface the Intuitive Query Builder here. A direct link (or relative MDX reference) would greatly improve usability:

- Need help building queries? Try our **Intuitive Query Builder** to construct queries visually …
+ Need help building queries? Try our **[Intuitive Query Builder](../query-builder/overview)** to construct queries visually …

(Replace path with actual file/slug.)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 24f5921 and 61c37bd.

📒 Files selected for processing (6)
  • blog/2025-07-30-dev-update-july-2025.md (4 hunks)
  • docs/HyperIndex/Guides/ipfs.md (4 hunks)
  • docs/HyperSync-LLM/hypersync-complete.mdx (7 hunks)
  • docs/HyperSync/HyperRPC/hyperrpc-supported-networks.md (1 hunks)
  • scripts/update-endpoints.js (14 hunks)
  • sidebarsHyperIndex.js (1 hunks)
✅ Files skipped from review due to trivial changes (3)
  • blog/2025-07-30-dev-update-july-2025.md
  • docs/HyperSync/HyperRPC/hyperrpc-supported-networks.md
  • scripts/update-endpoints.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • sidebarsHyperIndex.js
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: chainweb testnet 20 (id: 5920) and chainweb testnet 21 (id: 5921) have non-functional hypersync endp...
Learnt from: DenhamPreen
PR: enviodev/docs#673
File: docs/HyperIndex/supported-networks/chainweb-testnet-21.md:14-16
Timestamp: 2025-07-29T14:30:03.677Z
Learning: Chainweb Testnet 20 (ID: 5920) and Chainweb Testnet 21 (ID: 5921) have non-functional HyperSync endpoints that return HTTP 404, indicating these networks are not properly supported and should be removed from documentation.

Applied to files:

  • docs/HyperSync-LLM/hypersync-complete.mdx
🪛 markdownlint-cli2 (0.17.2)
docs/HyperIndex/Guides/ipfs.md

45-45: Bare URL used

(MD034, no-bare-urls)


45-45: Bare URL used

(MD034, no-bare-urls)

Copy link
Collaborator

@JonoPrest JonoPrest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing!

@DZakh DZakh merged commit 8766814 into main Aug 7, 2025
3 checks passed
@DZakh DZakh deleted the dz/preload branch August 7, 2025 15:19
@DenhamPreen DenhamPreen mentioned this pull request Aug 11, 2025
@DenhamPreen DenhamPreen mentioned this pull request Sep 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants