Compile a binary to run all tests at once.

[erictzeng]

Fixes #174.

This pull requests adds a new make target for a test_all.testbin. Running this binary will run all tests. This is preferable to the current for-loop method of running each binary individually, since it will report a single summary once all tests complete.

This required a few minor changes to the test_caffe_main header.

I am by no means a Makefile expert, so I am very much open to comments on my Makefile edits.

[kloudkl]

Some users may still want to run the runtest target in the old way. By "adds a new make target for a test_all.testbin" you might mean to do something as follows.

runtestall: $(TEST_ALL_BIN)
    $(TEST_ALL_BIN)

[erictzeng]

I'm not against keeping the old runtest around, but as I see it there isn't really a reason to. The single binary seems like a nicer way to run all tests, and people can always run the individual binaries or pass the --gtest-filter flag to test specific things.

[shelhamer]

I think running all the tests with a full report is the right test target, and runtestall should replace runtest instead of adding a new target. Running individual tests is prone to accidentally missing a failure.

While in the development loop, if a particular test is failing it could be re-run alone by filtering as Eric suggests.

To that end, the --gtest-filter flag should be documented. @erictzeng how about adding a skeleton "Developer Guidelines" page to the docs in this PR? We can polish it before master to include lint and git advice too.

[shelhamer]

@erictzeng see development.md added in 857c6ac.

[Yangqing]

Good to know the --gtest-filter flag - I didn't know that and went for the (hacky) separation. This seems a much nicer solution.

[kloudkl]

I used to run the tests for the feature in development with "build/src/caffe/test/test_feature.testbin". Of course the flag is much more flexible.

[shelhamer]

Hey @erictzeng, how about committing a sentence or two explaining how to run tests and filter them to docs/development.md and we'll merge?

Thanks!

[erictzeng]

Requested documentation has been added.

[shelhamer]

Check this: CommonTest.TestBrewMode fails.

[erictzeng]

Ah, this was due to a test assuming that Caffe would start in CPU mode. It fails when the test before it switches Caffe to GPU mode.

I changed that test to explicitly set CPU mode before running, and a cursory glance at the remaining tests indicates that they are more explicit in setting Caffe's mode, but we should ensure future tests don't make any assumptions of this sort.

[shelhamer]

Thanks @erictzeng!