e2e example with HF model + MoE + torchao nvfp4

[vkuzo]

Summary:

Small e2e script for hf moe model + torchao nvfp4 inference.

There are a couple of workarounds needed, they are noted
with TODO comments and will be addressed in future PRs.

Test Plan:

python scripts/prototype/test_nvfp4_moe.py --recipe bf16
python scripts/prototype/test_nvfp4_moe.py --recipe nvfp4
// outputs nearly identical

// nvfp4
| Tasks  |Version|Filter|n-shot|    Metric     |   |Value |   |Stderr|
|--------|------:|------|-----:|---------------|---|-----:|---|------|
|wikitext|      2|none  |     0|bits_per_byte  |↓  |0.6024|±  |   N/A|
|        |       |none  |     0|byte_perplexity|↓  |1.5183|±  |   N/A|
|        |       |none  |     0|word_perplexity|↓  |9.3277|±  |   N/A|

[vkuzo]

Stack from ghstack (oldest at bottom):

• install debug expert token counters on nvfp4 moe test script #4322
• hook up hf + moe + nvfp4 script to lm_eval #4321
• wire up nvfp4 bmm #4320
• -> e2e example with HF model + MoE + torchao nvfp4 #4319

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/4319

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Rolling out OSDC (ARC) runners on pull & trunk workflows in PyTorch main

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[jerryzh168]

I remember seeing similar scripts of testing quantizing a model and check performance / accuracy somewhere else before as well, should we centralize these?