Add an optional switch to control edge behavior of average pooling

[oscarriddle]

Because the commits on old thread #6296 got messed up, I try to create this new thread based on a clean branch. Hi @Noiredd, would you please help close that thread, it will be very helpful.

1. Add an optional boolean switch named "dynamic_ave_pool_size" in the caffe.proto, and default=false.
2. Pass the handle of layer_param_.pooling_param() into Forward_cpu() and Backward_cpu() functions in pooling_layer.cpp, and the AvePoolForward() function in pooling_layer.cu (The AvePoolBackward() seems correct already).
3. If the dynamic_ave_pool_size==true, the variable "pool_size" would be updated immediately after the 4 coordinates been updated.

I just tested the Forward_cpu() version, it seems good. I will test the GPU forward version later on, but need some help on the Backward_cpu().

Thanks,
Xiaolun Cao

[Noiredd]

The only thing this PR is missing is a test case for this functionality. As it is, it passes the tests because the default value of the flag is false, so it's disabled and never tested.

Also, I left a couple minor comments in the review.

[Noiredd]

How about, instead of reading the param every time, just add a class member variable to hold the value of the flag, and just set it once during LayerSetUp?

[Noiredd]

if (pool_param.dynamic_ave_pool_size()) will suffice. And if you decide to follow my advice from the comment above, it will get even simpler (if (member_var_)).

[Noiredd]

Like above: if (dynamic_ave_pool_size) would be cleaner.

[Noiredd]

My last comment and the most opinion-based: is this the best name for this flag? How about valid_ave_pooling?
Also, while it's good that you've explained what this does, I think that the comment should make it more clear that valid pooling ignores the padding elements in the calculation.

[oscarriddle]

Hi Noiredd,

Thanks for help reviewing my fix. Now the code has been updated accordingly.
However, I'm not quite sure about why the version 1fddd65 can pass the travis CI, but after I modified several comment lines, and new version 14f04c9 and ad291f5 become failed. I'm still trying to address the issue.
The new fix is test passed under my project. I think I can help write a caffe test for this if you think that would be much better.

Thanks,
Xiaolun Cao

[oscarriddle]

E: Failed to fetch http://dl.bintray.com/apache/cassandra/dists/39x/main/binary-amd64/Packages 502 Bad Gateway [IP: 52.10.151.113 80] E: Failed to fetch http://dl.bintray.com/apache/cassandra/dists/39x/main/binary-i386/Packages 502 Bad Gateway [IP: 52.10.151.113 80] E: Some index files failed to download. They have been ignored, or old ones used instead.

Seems something happened to Travis CI server, perhaps I need to try again later.

[Noiredd]

Thanks for the fixes. One other thing: now that you use a member variable, we don't need to declare PoolingParameter pool_param in forward/backward methods.

As I said, this passes tests because it is actually never tested - every time any test case instantiates a Pooling Layer, it does so with its default setting, which is without valid_ave_pooling. For this reason this needs an explicit test for this particular functionality.

Those random Travis errors are obviously not related to the code, so let's not worry about them for now. If need be, I will restart the tests.

[oscarriddle]

The param declaration within Forward/Backward are removed now, and new ver. passed the travis.

Yes, I suggest this needs specific tests, and I'm looking into the existed tests and learning to create one. Besides, I think this feature shall be both tested under the CPU and GPU backend.

Thanks,

[oscarriddle]

I created an explicit test named "TestForwardAveValid", it will turn on the member variable valid_ave_pooling and compare the outputs.

I noticed the comments in the cuDNN version tests, says cuDNN doesn't support padding. So I'm not quite sure about the test method in cuDNN version.

If you would help take a review, it'll be much appreciated.

Thanks,

[Noiredd]

cuDNN doesn't support padding

Must be some old information, I'm quite sure we can use padded pooling with cuDNN.
I think we should test the padded version too, as without it there is no difference between valid/invalid pooling.

Another thing is that the blob that you're using is quite trivial - filled with a constant value. Are you sure this will be a solid test?
Also, what do you think about testing the backward pass on this as well?

[oscarriddle]

Sorry, I was busy working on other projects afterwards.

The test blob that used in this test is imitating the existed average pool test, which is "TestForwardAve".

I'm not 100% sure about the meaning of gradient test that in test_pooling_layer.cpp, like "TestGradientAve" corresponds to "TestForwardAve".

I think my previous code modification shall be safe enough, however add a gradient test would also be a good idea. Would you help check the actual mechanism and sufficiency of "TestGradientAve".

Thanks,