Refactor data layers to avoid duplication of data transformation code#954
Refactor data layers to avoid duplication of data transformation code#954
Conversation
|
@GeenuX Thanks for your initial commit, could you also update the ImageDataLayer to use the same DataTransformer? |
|
Thanks for working on this @GeenuX, it will be great to have the madness that is the current set of DataLayers cleaned up. Since commenting on @sguada's #569 I've changed my mind a bit...I think each of the transformations actually could be done as layers with little or no extra cost in terms of memory/time. The key would be to use @longjon's The current design looks good and would definitely be an improvement over the current state as well, but is slightly less flexible than having each transformation be a layer. We could modify this to use layers later but might end up having to make proto-breaking changes, which isn't ideal. I'll leave it up to @GeenuX to decide if/how to take this suggestion. |
Start the refactoring of the datalayers to avoid data transformation code duplication. So far, only DataLayer has been done.
3ff84d9 to
f6ffd8e
Compare
|
I've refactored the ImageDataLayer as well. I agree, it would be nice to have the transformations defined as separate layers. I'm not sure |
Refactor data layers to avoid duplication of data transformation code
|
@GeenuX I agree this is a worthwhile improvement in itself. Thanks for your further work to improve the data layers and make data processing more sane. |
|
@sguada I will let you decide how to carry on in #569. While the DataTransformer is a step in the right direction, it would be better still to have transformation layers along the lines of #954 (comment). I will follow-up by refactoring |
|
Don't panic, but this breaks model definitions in dev as pointed out by @jeffdonahue.
Sorry for merging before correcting these issues. |
|
Sorry about that! Thanks for the fix! |
Refactor data layers to avoid duplication of data transformation code
Refactor data layers to avoid duplication of data transformation code
As discussed in #941, I started refactoring the data layers to avoid the current code duplication of data
transformations.
(crop, mirror, mean file...)
Could @shelhamer or someone else could have a look at it before I go about refactoring the other
data layers?