Enable loading data sets from files for custom tasks

[davebiagioni]

Purpose: Allow custom tasks to load datasets from local files, not only from the Hugging Face Hub. Useful for offline / air‑gapped / otherwise restricted environments

Changes

• Config: Add optional hf_data_files to LightevalTaskConfig.
• Loader: Forward hf_data_files as data_files to datasets.load_dataset in LightevalTask.download_dataset_worker.
• Examples: Add examples/custom_tasks_templates/custom_yourbench_task_from_files.py.
• Docs: Update docs/source/adding-a-custom-task.mdx with file-based usage.

Checklist

• Tests pass locally
• Pre-commit hooks pass locally
• Added/updated documentation

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[NathanHB]

Looking great ! few nits and good to merge if tests pass :)

[davebiagioni]

@NathanHB review comments addressed. thanks!

[davebiagioni]

@NathanHB wondering about your preferred way to keep approved but unmerged branches up to date with main. should i just hold off until we're closer to merging, or update as we go? i don't want to trigger your CI/CD pipeline more than is needed 😄 thanks again!

[NathanHB]

hey @davebiagioni thanks for asking :)
I usually just merge with, run tests and merge as soon as they are green, we had low bandwidth recently so forgot to merge your but merging today !

[davebiagioni]

Thanks @NathanHB !🙏

[NathanHB]

thank you, and happy new year :)