Skip to content

updated docs#13

Merged
mdehsan873 merged 1 commit intomainfrom
doc-smartcrawlerfix
Jul 3, 2025
Merged

updated docs#13
mdehsan873 merged 1 commit intomainfrom
doc-smartcrawlerfix

Conversation

@mdehsan873
Copy link
Copy Markdown
Contributor

No description provided.

@mdehsan873 mdehsan873 merged commit 574f77f into main Jul 3, 2025
1 check passed
VinciGit00 added a commit that referenced this pull request Apr 1, 2026
- Rename smart-scraper β†’ extract, search-scraper β†’ search
- Remove commands dropped from v2: agentic-scraper, generate-schema, sitemap, validate
- Update scrape with --format flag (markdown, html, screenshot, branding)
- Update crawl with v2 polling model (max-pages, max-depth, max-links-per-page, allow-external)
- Update history with v2 service names (scrape, extract, search, monitor, crawl)
- Update all examples, JSON mode docs, and AI agent skill docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
VinciGit00 added a commit that referenced this pull request Apr 9, 2026
- Rename smart-scraper β†’ extract, search-scraper β†’ search
- Remove dropped commands: agentic-scraper, generate-schema, sitemap, validate
- Replace --stealth boolean with --mode fetch mode enum
- Update scrape with --format flag (markdown, html, screenshot, branding)
- Update crawl with v2 polling model and new options
- Update env variables to SGAI_API_URL, SGAI_TIMEOUT_S
- Update response field names (remainingCredits, markdown)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
VinciGit00 added a commit that referenced this pull request Apr 14, 2026
Align CLI docs with just-scrape PR #13 (v2 migration):
- search: add --location-geo-code, --time-range, --format, --nationality flags
- scrape: add links, images, summary, json formats, multi-format, --html-mode, --scrolls, --prompt/--schema
- crawl: add --format flag
- credits: fix jq path to camelCase remainingCredits
- ai-agent-skill: update CLAUDE.md snippet with new formats and flags

Align MCP docs with scrapegraph-mcp PR #16 (v2 migration):
- Update available tools from 8 v1 tools to 16 v2 tools
- Add crawl_stop, crawl_resume, credits, sgai_history, monitor_* tools
- Remove sitemap, agentic_scrapper references

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
VinciGit00 added a commit that referenced this pull request Apr 14, 2026
Rewrite all JavaScript code examples to match the new v2 SDK API from
ScrapeGraphAI/scrapegraph-js#13. Key changes:
- Replace factory pattern (scrapegraphai({ apiKey })) with direct imports
- All functions use (apiKey, params) signature
- scrape() uses formats array instead of single format string
- Return type is ApiResult<T> with status check, not throw-on-error
- crawl.status() renamed to crawl.get(), crawl.delete() added
- monitor.create() uses formats array, not prompt
- Restore generateSchema and checkHealth in docs
- Schema params use JSON objects, not Zod instances
- history is now history.list() and history.get()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
VinciGit00 added a commit that referenced this pull request Apr 14, 2026
- Add Monitor command section (create, list, get, update, delete, pause, resume)
- Fix extract: --mode is HTML mode (normal/reader/prune), not fetch mode
- Fix search: --location-geo-code β†’ --country
- Fix fetch modes: auto|fast|js + separate --stealth flag
- Add schema to history services list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
VinciGit00 added a commit that referenced this pull request Apr 15, 2026
Brings the CLI docs in line with the CLI changes in
ScrapeGraphAI/just-scrape#14 (which pulls in
scrapegraph-js v2 PR #13 head 096c110):

- Document the full `just-scrape monitor` action set, including the new
  `monitor activity --id <id> [--limit] [--cursor]` for paginated tick history
- Replace stale `-m direct+stealth` / `-m js+stealth` with real CLI syntax
  (`-m js --stealth`, fetch modes: auto/fast/js)
- Env vars: `SGAI_TIMEOUT_S` β†’ `SGAI_TIMEOUT`, default base URL now
  `https://api.scrapegraphai.com/api/v2`, document `SGAI_DEBUG`
- Credits example uses `.remaining` (v2 response shape)
- Add `schema` to the history services list
- Fix `--location-geo-code` β†’ `--country` in search example
- Add monitor usage examples (webhook, activity, jq filter for changes)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant