ci: Don't require secrets in medium e2e test (backport #530)

[mergify]

This removes any usage of secrets in the medium e2e

---

This is an automatic backport of pull request #530 done by Mergify.

[mergify]

Cherry-pick of da7b24a has failed:

On branch mergify/bp/release-v0.7/pr-530
Your branch is up to date with 'origin/release-v0.7'.

You are currently cherry-picking commit da7b24a.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   .github/workflows/e2e-nvidia-l4-x1.yml

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally