add translation literals for various Indic languages (Bengali, Gujarati, Punjabi, Tamil)

[rpm000]

We introduce translation literals for four indic languages (Bengali, Gujarati, Punjabi, Tamil). This allows multilingual evaluations to be run over these languages.

[rpm000]

@NathanHB , @hynky1999 : adding some translation literals for indic languages. This is my first PR in lighteval, any feedback/comments would be very much appreciated :)

[clefourrier]

Hi! It's looking good, so let's see if tests pass. Just to be sure, did you make sure the words you provided make sense in the examples provided in the doc?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[rpm000]

Hi @clefourrier , thanks for fixing the failing tests 🙏

Just to be sure, did you make sure the words you provided make sense in the examples provided in the doc?

Yep, we did make sure to check that. Just noting that these words were not professionally translated but machine translated (with some sanity checking from speakers from each language). Would you like me to add a comment highlighting this ?

[clefourrier]

Hi! We usually avoid machine translations in lighteval in order to get expressions as fluent as possible - I'll let @NathanHB decide whether we want to merge.

[NathanHB]

having the words sanity checked by native speakers is good enough imo !

[rpm000]

having the words sanity checked by native speakers is good enough imo !

Excited to have my first contribution to lighteval :) ! Very excited to hopefully add more in future (if there is a list of tasks/things that need to be fixed somewhere, would be happy to start working towards those too :) )