node-spider-man

Features

• Transform the page into JSON format response according to the rules
• Use fetchQueue to manage the fetchRequests in the order
• Support sync and async mode.
  • sync: Execute each fetchTask after the previous one finished
  • async: Execute each fetchTask after the delayFetch time
• Support queue finished callback
• Support to PAUSE or CONTINUE the spider's work(>=0.3.0)

How to use

npm install node-spider-man --save

var spiderMan = require('node-spider-man');

Demo

The demo shows the experiement that we can grab what we want from the internet using node-spider-man

Lisence @Apache-2.0

Copyright to @2012-2016 Staydan.com