Skip to content

Improve hypersync docs #588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
moose-code merged 1 commit into from
Mar 12, 2025
Merged

Improve hypersync docs #588

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/HyperIndex/overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,15 @@ slug: /overview

![Sync Process](../../static/img/sync.gif)

:::info HyperIndex & HyperSync

**HyperIndex** is Envio's full-featured blockchain indexing framework that transforms on-chain events into structured, queryable databases with GraphQL APIs.

**HyperSync** is the high-performance data engine that powers HyperIndex. It provides the raw blockchain data access layer, delivering up to 2000x faster performance than traditional RPC endpoints.

While HyperIndex gives you a complete indexing solution with schema management and event handling, HyperSync can be used directly for custom data pipelines and specialized applications.
:::

---

## Key Features
Expand Down
125 changes: 86 additions & 39 deletions docs/HyperSync/overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,71 +1,110 @@
---
id: overview
title: Overview
title: HyperSync Overview
sidebar_label: Overview
slug: /overview
---

<!-- TODO-update: num of chains needs to be updated regularly -->
# HyperSync: High-Performance Blockchain Data Access

HyperSync is a highly specialized data node built in Rust aimed at massively improving data retrieval speeds while also providing flexibility. It serves as a real-time, accelerated data query layer with a low-level API that can be used via Python, Rust, NodeJs or Go clients.
HyperSync supports more than [70+ EVM](/docs/HyperIndex/hypersync) chains and Fuel, and we are rapidly adding new networks.
## What is HyperSync?

HyperSync is an ideal solution for indexers, block explorers, data analysts, bridges, and other applications or use cases focused on performance.
HyperSync is a purpose-built, high-performance data retrieval layer that gives developers unprecedented access to blockchain data. Built from the ground up in Rust, HyperSync serves as an alternative to traditional JSON-RPC endpoints, offering dramatically faster queries and more flexible data access patterns.

Easily analyze things like:
- Every ERC20 `Transfer` event for _any address_ on Base.
- Every tx `to` or `from` a specific address.
- Which address has spent the most on `gas` in the last 10k blocks?
- Plus many more, it's built to be extremely flexible and performant.
:::info HyperSync & HyperIndex

HyperSync can be thought of as a more performant and efficient data source alternative than JSON-RPC. Retrieve millions of blocks, logs, transactions, and traces on multiple chains in seconds.
**HyperSync** is Envio's high-performance blockchain data engine that serves as a direct replacement for traditional RPC endpoints, delivering up to 2000x faster data access.

E.g. In 10 seconds HyperSync can:
**HyperIndex** is built on top of HyperSync, providing a complete indexing framework with schema management, event handling, and GraphQL APIs.

- Scan 200 m blocks on the Arbitrum;
- Retrieve and decode every `PoolCreated` log emitted by the Uniswap v3 Factory.
- That's 20 million blocks per second.
Use HyperSync directly when you need raw blockchain data at maximum speed, or use HyperIndex when you need a full-featured indexing solution.
:::

Think of it as a "1000x speed-up" alternative to RPC.
## The Problem HyperSync Solves

<iframe width="560" height="315" src="https://www.youtube.com/embed/iu_469ELotw" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
Traditional blockchain data access through JSON-RPC faces several limitations:

- **Speed constraints**: Retrieving large amounts of historical data can take days
- **Query flexibility**: Complex data analysis requires many separate calls
- **Cost inefficiency**: Expensive for data-intensive applications

## Key Benefits

- **Exceptional Performance**: Retrieve and process blockchain data up to 1000x faster than traditional RPC methods
- **Comprehensive Coverage**: Access data across [70+ EVM chains](/docs/HyperSync/hypersync-supported-networks) and Fuel, with new networks added regularly
- **Flexible Query Capabilities**: Filter, select, and process exactly the data you need with powerful query options
- **Cost Efficiency**: Dramatically reduce infrastructure costs for data-intensive applications
- **Simple Integration**: Client libraries available for Python, Rust, Node.js, and Go

## Performance Benchmarks

HyperSync delivers transformative performance compared to traditional methods:

| Task | Traditional RPC | HyperSync | Improvement |
| ------------------------------------------------ | --------------- | --------- | ------------- |
| Scan Arbitrum blockchain for sparse log data | Hours/Days | 2 seconds | ~2000x faster |
| Fetch all Uniswap v3 PoolCreated events ethereum | Hours | Seconds | ~500x faster |

## Use Cases

HyperSync powers a wide range of blockchain applications, enabling developers to build tools that would be impractical with traditional data access methods:

### General Applications

- **Indexers**: Build high-performance data indexers with minimal infrastructure
- **Data Analytics**: Perform complex on-chain analysis in seconds instead of days
- **Block Explorers**: Create responsive explorers with comprehensive data access
- **Monitoring Tools**: Track blockchain activity with near real-time updates
- **Cross-chain Applications**: Access unified data across multiple networks
- **ETL Pipelines**: Create pipelines to extract and save data fast

### Powered by HyperSync

#### [HyperIndex](https://docs.envio.dev/docs/HyperIndex/overview)

- **100x faster blockchain indexing** across 70+ EVM chains and Fuel
- **Powers 100 plus applications** like v4.xyz analytics

#### [ChainDensity.xyz](https://chaindensity.xyz)

- **Fast transaction/event density analysis** for any address
- **Generates insights in seconds** that would take hours with traditional methods

#### [Scope.sh](https://scope.sh)

- **Ultra-fast Account Abstraction (AA) focused block explorer**
- **Fast historical data retrieval** with minimal latency

## Getting Started with HyperSync

HyperSync offers simple yet powerful client libraries. Here's a basic example using the Python client:

```python
import hypersync
from hypersync import (
LogSelection,
LogField,
DataType,
FieldSelection,
ColumnMapping,
TransactionField,
)
import asyncio


async def collect_events():
# choose network
# Initialize client with your network of choice
client = hypersync.HypersyncClient(
hypersync.ClientConfig(
url="https://arbitrum.hypersync.xyz",
# use secret bearer token for access
# See https://docs.envio.dev/docs/HyperSync/api-tokens
bearer_token="ea52c5da-4114-42ec-82df-8e73baad52ef",
bearer_token="your-token-here", # See https://docs.envio.dev/docs/HyperSync/api-tokens
)
)

# Define your query - this example finds all Uniswap v3 pool creation events
query = hypersync.Query(
from_block=0,
logs=[
LogSelection(
address=[
"0x1F98431c8aD98523631AE4a59f267346ea31F984"
], # uniswap factory
topics=[
[
"0x783cca1c0412dd0d695e784568c96da2e9c22ff989357a2e8b1d9b2b4e6b7118"
]
], # PoolCreated log
address=["0x1F98431c8aD98523631AE4a59f267346ea31F984"], # Uniswap v3 factory
topics=[["0x783cca1c0412dd0d695e784568c96da2e9c22ff989357a2e8b1d9b2b4e6b7118"]], # PoolCreated event
)
],
field_selection=FieldSelection(
Expand All @@ -77,25 +116,33 @@ async def collect_events():
LogField.DATA,
LogField.TRANSACTION_HASH,
],
transaction=[
TransactionField.BLOCK_NUMBER,
],
transaction=[TransactionField.BLOCK_NUMBER],
),
)

# Configure output format with automatic event decoding
config = hypersync.StreamConfig(
hex_output=hypersync.HexOutput.PREFIXED,
event_signature="PoolCreated(address indexed token0, address indexed token1, uint24 indexed fee, int24 tickSpacing, address pool)",
)

# Collect data to a local Parquet file
await client.collect_parquet("data", query, config)


asyncio.run(collect_events())
```

## See HyperSync in Action

<iframe width="560" height="315" src="https://www.youtube.com/embed/iu_469ELotw" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

## Next Steps

- [Get an API Token](/docs/HyperSync/api-tokens) to access HyperSync services
- [View Supported Networks](/docs/HyperSync/hypersync-supported-networks) to see available chains
- [Check Client Documentation](/docs/HyperSync/hypersync-clients) for language-specific guides
- [Join our Discord](https://discord.gg/Q9qt8gZ2fX) for support and updates

:::note
Docs under construction! We appreciate your patience until we get there. Until then, we are happy to answer all questions in our [Discord](https://discord.gg/Q9qt8gZ2fX).
Our documentation is continuously improving! If you have questions or need assistance, please reach out in our [Discord community](https://discord.gg/Q9qt8gZ2fX).
:::

---