Obtain a low top-5 accuracy based on caffe reference imagenet model

[shuye-cheung]

I try caffe reference imageNet model to test the validation set of ImageNet 2012 competition. The top one accuracy is 56.89% and the top five accuracy is 70.24%. Yet the Alex' paper published in NIPS 2012 reported that the top 1 accuracy is 59.3% and the top 5 accuracy is 81.8%. This is a big gap (70.24% vs 81.8%). I doubt that if caffe reference model is not trained completely or I have done some mistakes? I'm looking forward to receiving your attention and reply.

[sguada]

That seems low. Are you sure the top-5 evaluation is correct?

Did you try to evaluate 10 crops (corners and center+ mirror), then average
them to classify the validation images?

On Tuesday, June 24, 2014, shuye-cheung notifications@github.com wrote:

I try caffe reference imageNet model to test the validation set of
ImageNet 2012 competition. The top one accuracy is 56.89% and the top five
accuracy is 70.24%. Yet the Alex' paper published in NIPS 2012 reported
that the top 1 accuracy is 59.3% and the top 5 accuracy is 81.8%. This is a
big gap (70.24% vs 81.8%). I doubt that if caffe reference model is not
trained completely or I have done some mistakes? I'm looking forward to
receiving your attention and reply.

—
Reply to this email directly or view it on GitHub
#535.

Sergio

[sguada]

@shuye-cheung I just computed top-1 and top-5 accuracy (using #531 ) of the caffe_reference model and got 57.4% and 80.4% respectively. So check your code or how are you evaluating the network.

[shelhamer]

@sguada could you add the top-1 and top-5 scores to the CaffeNet
description on the pretrained models page of the docs?

Le mercredi 25 juin 2014, Sergio Guadarrama notifications@github.com a
écrit :

@shuye-cheung https://github.com/shuye-cheung I just computed top-1 and
top-5 accuracy (using #531 #531 ) of
the caffe_reference model and got 57.4% and 80.4% respectively. So check
your code or how are you evaluating the network.

—
Reply to this email directly or view it on GitHub
#535 (comment).

Evan Shelhamer

[shuye-cheung]

@sguada Thanks for your prompt help. It's a bug in my code that lead to the wrong top-5 accuracy rather than the caffe_reference model. I have fixed the bug in my code. Now the top-5 accuracy has got 80.4%.

[jamt9000]

@sguada is that centre patches only?

[hiwonjoon]

@sguada Is there any built-in features or easy way to make 10 crops((4corners+1center)*2) and do accuracy evaluation? Or, do I need to implement this with my own in c++ or python?

Thanks!