Separate the reusable data processors from the data layers

[kloudkl]

This PR will resolve #148 and take over #196.

[kloudkl]

Welcome comments on the API.

[jeffdonahue]

SaveNextBatch (capitalize B, and change other occurrences as well)

[jeffdonahue]

This looks great! Could you update imagenet.prototxt with how you anticipate that looking with the new data source/processing/sink API?

[sguada]

@kloudkl I'm also working in #148, will create a PR soon, and then we can compare both.

I think data_sinks, should be separated of data_sources.

@Yangqing I don't know how protobuf handles changing enum types over time. I'm not sure if we should use enum type for the different kinds of data_sources and data_processing. What would happen if we add a new one. Would the new protobuf compatible with the old one?

[jeffdonahue]

@sguada, I believe the enum type will work fine (I'm also using it for the new LayerTypes in #219), we just can't change any of the ID numbers once we release. So we should always use the "next available" ID number in the enum (increment by 1 the largest ID ever used in the enum) and never reuse a number even if we decide to retire a particular data source or processing type.

[sguada]

I would add param with type: leveldb. In that case is clear that only one type is possible

[kloudkl]

Agreed.

[kloudkl]

There is no need to re-invent the data layers into data source. Only the data processors need to be extracted. A base class InputLayer will be added to simplify integration of the processors with the data layers. There will also be OutputLayer correspondingly.

[kloudkl]

Solved by #954.