Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 24 additions & 20 deletions sdks/javascript.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,18 @@ icon: "js"
</CardGroup>

<Note>
These docs cover **`scrapegraph-js` ≥ 2.0.1**. The v2 SDK is **ESM-only** and requires **Node ≥ 22**. Earlier `0.x`/`1.x` releases expose a different, deprecated API.
These docs cover **`scrapegraph-js` ≥ 2.1.0**. The v2 SDK is **ESM-only** and requires **Node ≥ 22**. Earlier `0.x`/`1.x` releases expose a different, deprecated API.
</Note>

<Warning>
**Breaking in 2.1.0 (types only):** all exported TypeScript types and Zod schemas dropped the `Api` prefix and now match `scrapegraph-py` 1:1 (`ApiScrapeRequest` → `ScrapeRequest`, `ApiFetchConfig` → `FetchConfig`, `apiScrapeRequestSchema` → `scrapeRequestSchema`, etc.). Monitor input types are also renamed: `ApiMonitorCreateInput` → `MonitorCreateRequest`, `ApiMonitorUpdateInput` → `MonitorUpdateRequest`, `ApiMonitorActivityParams` → `MonitorActivityRequest`. `ApiResult<T>` is the only type that keeps the prefix. Runtime JS code is unchanged — only TypeScript consumers need to rename imports.
</Warning>

## Installation

```bash
# npm
npm i scrapegraph-js@latest # pins a version >= 2.0.1
npm i scrapegraph-js@latest # pins a version >= 2.1.0

# pnpm
pnpm add scrapegraph-js@latest
Expand Down Expand Up @@ -117,12 +121,12 @@ const res = await sgai.scrape({

#### Parameters

| Parameter | Type | Required | Description |
| ------------- | --------------- | -------- | ------------------------------------------------------------- |
| `url` | `string` | Yes | URL to scrape |
| `formats` | `FormatEntry[]` | No | Defaults to `[{ type: "markdown" }]` |
| `contentType` | `string` | No | Override detected content type (e.g. `"application/pdf"`) |
| `fetchConfig` | `FetchConfig` | No | Fetch configuration |
| Parameter | Type | Required | Description |
| ------------- | ---------------- | -------- | ------------------------------------------------------------- |
| `url` | `string` | Yes | URL to scrape |
| `formats` | `FormatConfig[]` | No | Defaults to `[{ type: "markdown" }]` |
| `contentType` | `string` | No | Override detected content type (e.g. `"application/pdf"`) |
| `fetchConfig` | `FetchConfig` | No | Fetch configuration |

**Formats:**
- `markdown` — Clean markdown (modes: `normal`, `reader`, `prune`)
Expand Down Expand Up @@ -290,18 +294,18 @@ await sgai.crawl.delete(crawlId);

#### `crawl.start()` parameters

| Parameter | Type | Required | Description |
| ------------------ | --------------- | -------- | -------------------------------------------------------- |
| `url` | `string` | Yes | Starting URL |
| `formats` | `FormatEntry[]` | No | Defaults to `[{ type: "markdown" }]` |
| `maxDepth` | `number` | No | Maximum crawl depth. Default: `2` |
| `maxPages` | `number` | No | Maximum pages (1–1000). Default: `50` |
| `maxLinksPerPage` | `number` | No | Links followed per page. Default: `10` |
| `allowExternal` | `boolean` | No | Allow crossing domains. Default: `false` |
| `includePatterns` | `string[]` | No | URL patterns to include |
| `excludePatterns` | `string[]` | No | URL patterns to exclude |
| `contentTypes` | `string[]` | No | Allowed content types |
| `fetchConfig` | `FetchConfig` | No | Fetch configuration |
| Parameter | Type | Required | Description |
| ------------------ | ---------------- | -------- | -------------------------------------------------------- |
| `url` | `string` | Yes | Starting URL |
| `formats` | `FormatConfig[]` | No | Defaults to `[{ type: "markdown" }]` |
| `maxDepth` | `number` | No | Maximum crawl depth. Default: `2` |
| `maxPages` | `number` | No | Maximum pages (1–1000). Default: `50` |
| `maxLinksPerPage` | `number` | No | Links followed per page. Default: `10` |
| `allowExternal` | `boolean` | No | Allow crossing domains. Default: `false` |
| `includePatterns` | `string[]` | No | URL patterns to include |
| `excludePatterns` | `string[]` | No | URL patterns to exclude |
| `contentTypes` | `string[]` | No | Allowed content types |
| `fetchConfig` | `FetchConfig` | No | Fetch configuration |

### `sgai.monitor.*`

Expand Down
2 changes: 1 addition & 1 deletion services/crawl.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ asyncio.run(main())

### Official SDKs
- [Python SDK](/sdks/python)
- [JavaScript SDK](/sdks/javascript) (`scrapegraph-js` ≥ 2.0.1, Node ≥ 22)
- [JavaScript SDK](/sdks/javascript) (`scrapegraph-js` ≥ 2.1.0, Node ≥ 22)

### AI Framework Integrations
- [LangChain Integration](/integrations/langchain)
Expand Down
2 changes: 1 addition & 1 deletion services/extract.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ asyncio.run(main())

### Official SDKs
- [Python SDK](/sdks/python)
- [JavaScript SDK](/sdks/javascript) (`scrapegraph-js` ≥ 2.0.1, Node ≥ 22)
- [JavaScript SDK](/sdks/javascript) (`scrapegraph-js` ≥ 2.1.0, Node ≥ 22)

### AI Framework Integrations
- [LangChain Integration](/integrations/langchain)
Expand Down
2 changes: 1 addition & 1 deletion services/monitor.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ asyncio.run(main())

### Official SDKs
- [Python SDK](/sdks/python)
- [JavaScript SDK](/sdks/javascript) (`scrapegraph-js` ≥ 2.0.1, Node ≥ 22)
- [JavaScript SDK](/sdks/javascript) (`scrapegraph-js` ≥ 2.1.0, Node ≥ 22)

## Support & Resources

Expand Down
2 changes: 1 addition & 1 deletion services/scrape.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ asyncio.run(main())

### Official SDKs
- [Python SDK](/sdks/python) — perfect for automation and data processing
- [JavaScript SDK](/sdks/javascript) — ideal for web applications and Node.js (`scrapegraph-js` ≥ 2.0.1, Node ≥ 22)
- [JavaScript SDK](/sdks/javascript) — ideal for web applications and Node.js (`scrapegraph-js` ≥ 2.1.0, Node ≥ 22)

### AI Framework Integrations
- [LangChain Integration](/integrations/langchain) — use Scrape in your content pipelines
Expand Down
2 changes: 1 addition & 1 deletion services/search.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ asyncio.run(main())

### Official SDKs
- [Python SDK](/sdks/python)
- [JavaScript SDK](/sdks/javascript) (`scrapegraph-js` ≥ 2.0.1, Node ≥ 22)
- [JavaScript SDK](/sdks/javascript) (`scrapegraph-js` ≥ 2.1.0, Node ≥ 22)

### AI Framework Integrations
- [LangChain Integration](/integrations/langchain)
Expand Down
2 changes: 1 addition & 1 deletion transition-from-v1-to-v2.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ Exact paths and payloads are listed under each service (for example [Scrape](/se
1. Log in at [scrapegraphai.com/login](https://scrapegraphai.com/login)
2. Start from [Introduction](/introduction)
3. Follow [Installation](/install)
4. Upgrade packages: `pip install -U scrapegraph-py` / `npm i scrapegraph-js@latest` (requires **`scrapegraph-js` ≥ 2.0.1** and **Node ≥ 22**)
4. Upgrade packages: `pip install -U scrapegraph-py` / `npm i scrapegraph-js@latest` (requires **`scrapegraph-js` ≥ 2.1.0** and **Node ≥ 22**)

## SDK migration guides (detailed changelogs)

Expand Down