Add wmt translation example

[patrickvonplaten]

PR adds translation example for T5.
It uses the sacrebleu BLEU scorer.

I adapted the README.md a bit so that users are aware that models in official paper were attained with finetuned T5 @craffel

[LysandreJik]

Cool, that looks clean. I guess we don't need all the usual arguments (data parallel, fp16 and others) since this is an evaluation script and not a training script?

It doesn't seem to run on GPU, does it take a long time to compute? Shouldn't we try to cast to a GPU if there's one available?

[patrickvonplaten]

long

Yeah we should definitely try to run it on a GPU - will take a look at that :-)

[patrickvonplaten]

Not sure whether we need fp16 and multi-gpu training. I think single GPU training is enough and t5 + wmt does not take much memory. But happy to take a look into it if you guys think it's worth it :-) @thomwolf @LysandreJik @julien-c

[patrickvonplaten]

Code quality test fails because of unpinned isort library (see #3449)

[thomwolf]

Looks good to me

[craffel]

1. Suggest changing "the T5 Model t5-base" to "the multitask pre-trained checkpoint for t5-base
2. If you want to be more specific than "slightly", it looks like they are about 1.5 BLEU points lower than they would be with fine-tuning.

[codecov-io]

Codecov Report

Merging #3428 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3428   +/-   ##
=======================================
  Coverage   52.51%   52.51%           
=======================================
  Files         100      100           
  Lines       17051    17051           
=======================================
  Hits         8954     8954           
  Misses       8097     8097

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b4fb94f...713524e. Read the comment docs.

[LysandreJik]

LGTM!