Skip to content
This repository was archived by the owner on Mar 14, 2019. It is now read-only.

1.1.0 TempStore support for Storage Adapter Api

Open
Overdue by 11 year(s)
Due by April 17, 2014
Last updated May 30, 2014
100% complete

Reasons:

  1. Some OS's reset the temporary folder - removing all uploaded data (this is expected)
  2. Not all deployments allow access to the os temp folder
  3. We prepare for the fileWorker implementation - which allows us to scale

The user can assign a storage adapter to the TempStore,
The cfs-worker will have the TempStore default to gridfs if cfs-gridfs is added to the project - but it will not overwrite user configured settings.

We may have a collection to track file uploads at some point - The TempStore is event based and allows others to listen for events.

That said we have to have a way of tracking temporary files and status of upload, why a collection could be nice. The file workers could observe on this, or listen to events - again a flexible approach.

The TempStore presents a streaming api that allows data to be:

  • uploaded in chunks
  • passed directly from server api
  • synchronized from a store

API

Write stream
FS.TempStore.createWriteStream(fileObj);
FS.TempStore.createWriteStream(fileObj, storeName);
FS.TempStore.createWriteStream(fileObj, chunkNumber);

Read stream
FS.TempStore.createReadStream(fileObj);

Remove
FS.TempStore.removeFile(fileObj);

List chunks / parts
FS.TempStore.listParts(fileObj)

File exists
FS.TempStore.exists(fileObj)

Events
The TempStore is a node js EventEmitter and emits following events:

start - callback( fileObj, chunkNumber )
progress - callback( fileObj, chunk, chunkCount )
remove - callback( fileObj, filePath )

ready events
uploaded - callback( fileObj )
stored - callback( fileObj )
synchronized - callback( fileObj, storeName )

ready - callback( fileObj, options )
options can be:

  • undefined - direct stored from server api

  • string - name of Store

  • number - sum of chunks

TODO:
We should have a collection to track files making our own temporary storage, removing files when we intent to and not depend on one type of storage.

  • Convert the current implementation to use the FS.Store.FileSystem SA
  • Track files in a collection, and use storageId instead of fileId
    uploadId is used by S3 but we do more than just upload, we have direct api and SA synchronization
  • A file could be stored twice, make sure the latest win, eg. the old may not emit ready events
  • The tracking collection should have a expireAt on each file - This way we make sure to remove old data

List view

    There are no open issues in this milestone

    Add issues to milestones to help organize your work for a particular release or project. Find and add issues with no milestones in this repo.