Refactor CLI [9/N]: Replace HfArgumentParser from transformers with local

[albertvillanova]

Replace HfArgumentParser from transformers with local.

This PR updates the import statements in trl/scripts/utils.py to use local versions of argument parsing utilities instead of those from the external transformers library. This change helps improving latency within the codebase.

The latency of import trl.scripts.utils changes from 7.5s to <0.5s after the merge of:

• Refactor CLI [7/N]: Move patching to compat and import transformers conditionally #5208
• Refactor CLI [8/N]: Refactor scripts/utils with delayed imports #5209
• Refactor CLI [9/N]: Replace HfArgumentParser from transformers with local #5210

See upstream issue:

• Lazy loading is not working properly transformers#44273

Dependency management:

• Replaced imports of HfArgumentParser, DataClass, and DataClassType from the transformers library with imports from the local trl.scripts._hf_argparser module.

---

Note

Medium Risk
Although intended as a drop-in copy, vendoring core CLI argument parsing code can subtly change parsing behavior and may drift from upstream fixes, impacting script configuration/flag handling.

Overview
Introduces a vendored copy of transformers’ hf_argparser as trl/scripts/_hf_argparser.py, with yaml imported lazily in parse_yaml_file to avoid the upstream import-time latency.

Updates trl/scripts/utils.py to import HfArgumentParser/DataClass/DataClassType from the local module instead of transformers, keeping CLI parsing behavior local and reducing startup overhead.

^{Written by Cursor Bugbot for commit 2e15b40. This will update automatically on new commits. Configure here.}

[qgallouedec]

does it mean that when huggingface/transformers#44273 is solved, we could just revert this commit?

[albertvillanova]

Thanks for your review, @qgallouedec.

Not exactly, because we still support older versions of transformers where the import latency is not fixed.

Instead, we could make a conditional import depending on the transformers version.
What do you think?

[qgallouedec]

I'd recommend at least a comment explaining that it's copied from transformers and modified because of the latency (you already did it) +

• a comment saying that this can be removed when min transformers version contains the fix of Lazy loading is not working properly transformers#44273 OR
• a conditional import (maybe over-engineered here?)

up to you