Caffemodel protobuf snapshots with shared weights shouldn't save multiple copies#2955
Caffemodel protobuf snapshots with shared weights shouldn't save multiple copies#2955danielgordon10 wants to merge 1 commit intoBVLC:masterfrom
Conversation
So here I actually feel like the HDF5 code is less organized than the ToProto code. Through the principle of code reuse, we would want the parent serializer to call the child serializer rather than re-implement the logic. So there comes a question - do we want to be able to serialize a blob on its own? In the Proto case, that is possible. In the HDF5 case, it is not currently written that way. So I could certainly pull the logic that writes blobs/layers into the net. But then we should probably also delete the ToProto methods in Layer and in Blob. And personally, I feel they might be useful in the future. |
|
Yes, I see now that there's already asymmetry in the protobuf vs. HDF5 serialization pathways, and I could see an argument for a bit of reorganization there in one direction or the other. However, the most immediate thing is to fix the duplicate storage bug, so I'd be happy to merge the most straightforward patch that does that, and revisit other issues later. You don't have to create a new PR to rewrite history; just |
|
What changes would you like me to make that would qualify as the "most straightforward patch"? |
|
Ping |
|
So what's the deal here? Is this dead? |
Sorry, I had to delete the other branch #2946 because of the squashing. Continue discussion here.
It does seem feasible to me to write separate diffs for each shared param but not need to write the data. The data among shared parameters will always be the same, but based on the path through those parameters, the diffse could be different on different layers. Looking at the HDF5 writer, a similar precaution is taken. The comment in ToHDF5 on line 977 appears to support this argument.
This change is