You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is an uncomplete PR to support multiclass loss in caffe.
I doubt this will be ever merged but I wanted to share some modifications I made to memoryDataLayer to accept multiple labels and to softmax to handle mutliclass problems.
A good discussion about multilabel, multiclass and multitask problems in caffe is here
The main news this PR introduce are:
a new method for MemoryDataLayer called addMatVectorMultilabel that accepts a vector < vector < int > > as label for each data Mat. So a Mat can be easily associated with multiple classes. The resulting label blob will have the label values on the channel dimension (or more exactly in the dimension 1since the N-dimension support).
extended SoftmaxLossLayer to let it to compute the loss on different labels and on slices of bottom[0]. The number of slices are specified with the prototxt parameter slice and must match bottom[1]->shape(softmax_axis_)
extended SoftmaxLayer to let it to compute the softmax independently on slices of bottom[0], so the SoftmaxLossLayer can compute the loss for each slice comparing it with each corresponding label.
The idea is to feed the network with MemoryDataLayer and addMatVectorMultilabel, then attaching a SoftmaxLossLayer specifying the parameter slice to be equal to the size of the vector of the labels added. It is important to set the output of bottom[0] of the SoftmaxLossLayer to be a multiple of the number of classes and of the slices.
For example let's say we want to add a vector of 3 labels, and each labels can have 4 different classes, the input for addMatVectorMultilabel could be a vector like '{{0,0,0,1}, {1,0,0,0}, {0,1,0,0}}.
Now after some conv layer we attach an Inner_product layer with num_output: 12 (3*4 = 12). Then we attach to the InnerProductLayer layer a SoftmaxLossLayer specifying slice: 3. So the softmax will be applied independently to each slice of 4 elements of the output of the InnerProductLayer and the loss calculated for each softmax.
Actually the cpu version of SoftmaxLayer and SoftmaxLossLayer is working, but the gpu is not, I wrote some modification on the gpu part of the SoftmaxLayer but some index is wrong ( the test doesn't pass when batch_size>2.
I don't know when I will have time to fix that, if someone wants to contribute, any help is appreciated :)
@mtamburrano I also found a definition of multiclass, multilabel and multi-task classification here. According to this reference, softmax is inherently a multiclass classifier.
I think your modification is more related to multi-task classification, which can be implemented by setting inner_num_ in caffe. In the above example, you can use ReshapeLayer to reshape (12,) into (4, 3) before forwarding into SoftmaxWithLossLayer, and set softmax_axis_ to the 1st axis. Then inner_num_ will be automatically set to 3 and multi-task classification is executed.
Basically, multilabel classification can also be done in this way, but one needs to duplicate the inner_product output (4-D in your example) number_of_labels (number_of_tasks) times. This needs write a new DuplicateLayer or simply use SplitLayer + ConcatLayer. However, either method will consume more memory space. So in PR #3268, I did some modification in SoftmaxWithLossLayer to support multilabel classification without more memory requirement.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is an uncomplete PR to support multiclass loss in caffe.
I doubt this will be ever merged but I wanted to share some modifications I made to memoryDataLayer to accept multiple labels and to softmax to handle mutliclass problems.
A good discussion about multilabel, multiclass and multitask problems in caffe is here
The main news this PR introduce are:
MemoryDataLayercalledaddMatVectorMultilabelthat accepts avector < vector < int > >as label for each data Mat. So a Mat can be easily associated with multiple classes. The resulting label blob will have the label values on the channel dimension (or more exactly in the dimension1since the N-dimension support).SoftmaxLossLayerto let it to compute the loss on different labels and on slices ofbottom[0]. The number of slices are specified with the prototxt parametersliceand must matchbottom[1]->shape(softmax_axis_)SoftmaxLayerto let it to compute the softmax independently on slices ofbottom[0], so theSoftmaxLossLayercan compute the loss for each slice comparing it with each corresponding label.The idea is to feed the network with
MemoryDataLayerandaddMatVectorMultilabel, then attaching aSoftmaxLossLayerspecifying the parametersliceto be equal to the size of the vector of the labels added. It is important to set the output ofbottom[0]of theSoftmaxLossLayerto be a multiple of the number of classes and of the slices.For example let's say we want to add a vector of 3 labels, and each labels can have 4 different classes, the input for
addMatVectorMultilabelcould be a vector like '{{0,0,0,1}, {1,0,0,0}, {0,1,0,0}}.Now after some conv layer we attach an
Inner_productlayer withnum_output: 12(3*4 = 12). Then we attach to theInnerProductLayerlayer aSoftmaxLossLayerspecifyingslice: 3. So the softmax will be applied independently to each slice of 4 elements of the output of theInnerProductLayerand the loss calculated for each softmax.Actually the cpu version of
SoftmaxLayerandSoftmaxLossLayeris working, but the gpu is not, I wrote some modification on the gpu part of theSoftmaxLayerbut some index is wrong ( the test doesn't pass whenbatch_size>2.I don't know when I will have time to fix that, if someone wants to contribute, any help is appreciated :)