Skip to content
This repository was archived by the owner on Mar 14, 2019. It is now read-only.

1.2.0 FileWorker support for scaling and throttling ressources

Open
Overdue by 11 year(s)
Due by May 31, 2014
Last updated Apr 9, 2015
50% complete

The FileWorker architecture is based on workers / instances pulling tasks as their resources allow it.
Reasons:

  1. When scaling Meteor we will run on multiple instances using oplog to keep data in sync - but storing uploaded files in TempStore should be available to all instances (we can loose instances in some arcs)
  2. The concept of transform streams allows for heavy analyze of data / files - so we would like to be able to isolate those not crashing the main servers

We could scale by having multiple Meteor instances and use gridFS for TempStore (uploaded files)
Tasks will be retrieved by using a request pattern via db - ensuring one worker pr. task.

  • Rework the cfs-worker implement better throttling

List view

    There are no open issues in this milestone

    Add issues to milestones to help organize your work for a particular release or project. Find and add issues with no milestones in this repo.