Skip to content

Comments

slightly relax batch norm check#4785

Merged
shelhamer merged 1 commit intoBVLC:masterfrom
bwilbertz:slightly_relax_batch_norm_check
Sep 30, 2016
Merged

slightly relax batch norm check#4785
shelhamer merged 1 commit intoBVLC:masterfrom
bwilbertz:slightly_relax_batch_norm_check

Conversation

@bwilbertz
Copy link
Contributor

As already raised in #4704, the current LayerSetup method of the BatchNormLayer expects to have no LayerParamter configured at all, but also adds 3 LayerParameter itself (cf. batch_norm_layer.cpp).
This all is done to avoid any unintenional backward modification of the global stats in the BatchNormLayer and therefore forcing lr_mult = 0.f.

As a consequence, a valid (upgraded) net, which was passed once into Net<Dtype>(net_param) and read back via ToProto(...) cannot be passed a second time into Net without being upgraded again.

This PR slightly relaxes the checks in LayerSetUp, to just ensure lr_mult == 0.f or adding it, if the LayerParameter is missing (as it was introduced in #4704).

@shelhamer
Copy link
Member

Thanks for issuing a patch for this @bwilbertz and sorry I couldn't handle it immediately. I'll try to check and merge today.

Note that the CI failures seem to be some bogus issue with CUDA packages on the Travis machines: https://travis-ci.org/BVLC/caffe/jobs/163831672 We can hopefully resolve that shortly. @lukeyeager have you seen this issue w/ Travis?

@lukeyeager
Copy link
Contributor

@lukeyeager have you seen this issue w/ Travis?

Yeah we're working on it - sorry.

@shelhamer shelhamer merged commit a7f950b into BVLC:master Sep 30, 2016
@shelhamer
Copy link
Member

Thanks again for this @bwilbertz -- merged after checking it myself since the CI issue is unrelated.

@lukeyeager thanks for looking into the CI problem!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants