[BugFix] Fix MultiOneHot.to_numpy() returning scalar instead of array

[jashshah999]

Summary

MultiOneHot.to_numpy() inherits from OneHot.to_numpy(), which calls argmax(-1) over the entire concatenated one-hot vector. For a MultiOneHot((11, 3)) spec with a 14-element vector like [0,0,1,0,0,0,0,0,0,0,0,1,0,0], this returns a single scalar 2 instead of the correct array [2, 0].

This also breaks gym MultiDiscrete environment interaction since GymWrapper maps those to MultiOneHot.

Changes

• Add to_numpy override to MultiOneHot that delegates to to_categorical (which already correctly splits by nvec and applies per-segment argmax)
• Add parametrized tests covering multiple nvec configs and batch shapes

Fixes #2633

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3589

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Cancelled Job

As of commit 5c52391 with merge base e2c8a8d ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Workflow failed! Resource not accessible by integration
• Unit-tests on Linux / tests-olddeps (3.10, 11.8) / linux-job (gh)
  RuntimeError: Command docker exec -t f1415fd3efb173f4b306796d51501c90bc8488fd4ff7e2d20faf517be9c2e8b9 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[vmoens]

Wondering if getting a categorical value out of a one-hot when calling to_numpy() is the expected behavior?
Is it documented as such?
What happens for OneHot for instance, do we get a numpy scalar?

[vmoens]

Ok read your comment on the PR, LGTM then!