Add support for bind mounts under /tmp with hermetic tmp

[fmeum]

This is achieved by rewriting the user-specified mounts to mounts onto subdirectories of the hermetic sandbox tmp directory.

Fixes #20527

[fmeum]

@lberki Happy to implement any other kind of fix if you can think of one that doesn't require disabling hermetic tmp in this case.

[fmeum]

@lberki Both tests pass with --noincompatible_sandbox_hermetic_tmp.

[lberki]

Disabling hermetic /tmp isn't a very good idea because it wouldn't let us delete the code path supporting --noincompatible_sandbox_hermetic_tmp. I have three ideas:

1. Error out with a mount point under /tmp
2. Change the order of mounts: if a mount point is under /tmp, mount it after we mount the hermetic /tmp here
3. Rewrite the mount target so that it's under the directory that we eventually mount under /tmp

(2) and (3) contain some amount of DWIM, which could come back to bite us: for example, I could imagine someone creative mounting /foo under /tmp/foo, then /tmp/foo under /bar, which would not work with the reshuffling proposed.

I think the only theoretically perfect solution is either (1) or to apply some cleverness to rewrite or reorder the mount specifications such that they do they right thing. But I don't like cleverness so my preferred option is (1).

[fmeum]

I also favor 1, but what about the other code paths that disable the feature, e.g. a tmpfs mount under /tmp/foo? We also have to continue to support --sandbox_add_mount_pair=/tmp - can we even get rid of the alternate code path?

[lberki]

Why do we have to continue supporting --sandbox_add_mount_pair=/tmp ?

[fmeum]

Why do we have to continue supporting --sandbox_add_mount_pair=/tmp ?

There are cases where you would want your test to be able to communicate with some external process (such as Docker) via a socket in /tmp. It would be mildly surprising if --sandbox_add_mount_pair would work for essentially any path except for /tmp (or anything underneath).

[lberki]

Fair enough. Mind taking a stab at implementing the clever reordering / rewriting I mentioned in one of the above comments then? It doesn't look impossible or fundamentally difficult, it was just cleverness I had hoped we could live without.

[fmeum]

Sure, I will give it a try.

[lberki]

Thanks, let me know when you need my eyes (I'm OOO from Friday until the end of this year, though)

[fmeum]

@lberki I implemented the rewriting and my test cases pass now. Are there any more scenarios you would like me to add tests for?

[fmeum]

@bazel-io fork 7.0.1

[lberki]

Looks very reasonable. I have one nit, but I could even be convinced that that one is not important.

[iancha1992]

@fmeum @lberki the tests keep failing. Can you please take a look?
cc: @bazelbuild/triage

[lberki]

For the Open JDK 11 / Mac OS tests (//src/test/py/bazel:py_test), this change needs to be cherrry-picked to 7.0.1: d9dc4fd

for the rest, I don't know -- maybe @meteorcloudy or @fweikert or @Wyverald do?

[Wyverald]

The test failures are due to our CI machine upgrades and will be gone when we rebase this PR on top of master. (No need to do that; we're already doing the import internally.)

[iancha1992]

@bazel-io fork 7.1.0