Skip to content

Comments

Fixed a bug in AccuracyLayer.#6066

Closed
ryanflower wants to merge 1 commit intoBVLC:masterfrom
ryanflower:master
Closed

Fixed a bug in AccuracyLayer.#6066
ryanflower wants to merge 1 commit intoBVLC:masterfrom
ryanflower:master

Conversation

@ryanflower
Copy link

gpu_diff of bottom[0] used as temporary buffer needs to be set back to
zeros. Otherwise the AccurarcyLayer may propagate back wrong gradient
values in training.

gpu_diff of bottom[0] used as temporary buffer needs to be set back to
zeros. Otherwise the AccurarcyLayer may propagate back wrong gradient
values in training.
@shaibagon
Copy link
Member

This is not a good fix: if the prediction layer is shared with other layers (e.g., "SoftmaxWithLoss") resetting diff might reset the gradients coming from the loss layer.

Please see PR #5987 for a fix for this issue.

@shaibagon shaibagon closed this Nov 22, 2017
@ryanflower
Copy link
Author

I see your point, though I think split layers are automatically added to avoid such cases.

A better fix is to use bottom[0]->diff().use_count() to check whether the diff memory is shared with other layers. If so, initialize and use an internal buffer; otherwise reuse bottom[0]->diff() and reset it to zeros after usage.

@shaibagon
Copy link
Member

@ryanflower this is an interesting point you are making. Can you please move this discussion to the open PR #5987?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants