Skip to content
Guamaso edited this page Sep 24, 2014 · 1 revision

(Needs to be cleaned up)

Would like it to accept a couple of arguments that will keep it abstracted from the data source and parsers. I can point the scraper to a data source, like a URL, and tell it to use the function in the file X.js for that URL. Just in case the data on that page changes, it should be passed an array of possible files to use. If no errors are returned by that file then it will stop and move on to the next step in the process.

The data source can be a URL or a file path.

Note that these files will not be publicly accessible, they will run on a server using some sort of cron process.

Clone this wiki locally