[vote] terminology for training process

[pluskid]

In the current Python binding, the terminology for some training related terms is a bit inconsistent. Specifically, here we should determine the usage of the term for one mini-batch and one full pass through the data.

Current status is: round and iteration mean the same thing -- full path through the data. epoch means a single mini-batch. This will affect parameter and callback names, like start_round, epoch_callback, etc. We should choose a unified (and standard) terminology here before it's getting too late to change names.

My personal opinion is that epoch, if we ever want to use this name, should mean one full pass through the data. I do not find any "standard" reference for this. But looking around, almost all materials I found epoch means this. Some examples:

• One of Leon Bottou's paper on SGD
• Caffe: they do not have variable or function with epoch in it, but from here, I believe epoch means 1 full data pass
• In Theano, epoch is the outer loop, and batch is the inner loop
• In Chainer, epoch is the outer loop
• ...

Possible vote options (please add more if needed)

1. epoch for full data pass, iteration for a single minibatch
2. data_pass for full data pass, iteration for a single minibatch
3. round for full data pass, iteration for a single minibatch
4. epoch for full data pass, batch for a single minibatch

[tqchen]

How about epoch and batch, like theano? As batch makes things quite clear it is minibatch iteration

[pluskid]

@tqchen yes, that also sounds quite sensible to me.

[tqchen]

@antinucleon @mli @hjk41 @jermainewang

[XericZephyr]

As a heavily caffe user, the terminology in mxnet is quite confusing to me. The most confusing one is the 'epoch' which here means a mini-batch actually. I suggest using epoch for a full-pass iteration instead which I think is also accepted in main stream.

[tqchen]

I vote for 4

[shengwa]

I vote for 4.
epoch for one full data pass and batch for one minibatch pass, it is quite clear.

[pluskid]

+1 for 1 or 4.

[antinucleon]

+1 for 4

[hetong007]

+1 for 4

[yeqinglee]

+1 for 4

[Wakeupbuddy]

vote for 1 and 4

[tqchen]

Seems 4 it is then. @antinucleon @hetong007 Can you push the change on python and R side respectively?

[tqchen]

python side changed by #415