env bug: infinite recursion when setting environment variables

[milahu]

upstream issue from 2015 https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/1421760

these work as expected ...

$ ./coreutils/target/debug/env python
>>> print("hello")
hello

$ ./coreutils/target/debug/env A=1 python
>>> import os; print(os.environ["A"])
1

#! ./coreutils/target/debug/env python
# test.py
print("hello") # hello

but when i combine shebang with env A=1, i get infinite recursion

#! ./coreutils/target/debug/env A=1 python
# test.py
import os; print(os.environ["A"])

warning: this is a fork bomb → use systemd-run to limit the number of forks

$ systemd-run --scope -p TasksMax=10  --user strace -f --trace execve ./test.py 2>&1 | grep -F 'execve("./test.py"'
execve("./test.py", ["./test.py"], 0x7ffd648f0898 /* 131 vars */) = 0
[pid 220313] execve("./test.py", ["./test.py"], 0x5632208a67b0 /* 132 vars */) = 0
[pid 220314] execve("./test.py", ["./test.py"], 0x7ffd14507d78 /* 132 vars */) = 0
[pid 220315] execve("./test.py", ["./test.py"], 0x7ffecc08f328 /* 132 vars */) = 0
[pid 220316] execve("./test.py", ["./test.py"], 0x7ffc15a32f08 /* 132 vars */) = 0
[pid 220317] execve("./test.py", ["./test.py"], 0x7ffe70b3ef28 /* 132 vars */) = 0
[pid 220318] execve("./test.py", ["./test.py"], 0x7ffd2260e9b8 /* 132 vars */) = 0
[pid 220319] execve("./test.py", ["./test.py"], 0x7ffdd4e2c858 /* 132 vars */) = 0
[pid 220320] execve("./test.py", ["./test.py"], 0x7ffdb17304b8 /* 132 vars */) = 0

[tertsdiepraam]

Interesting find! So, if I understand correctly, both uutils and GNU have the same problem here?

For GNU, there is at least a workaround: -S. This works on my machine:

#! /usr/bin/env -S A=1 python
# test.py
import os; print(os.environ["A"])

But uutils env does not support this yet. The reason this is necessary is that A=1 python are passed to env as one argument (although it is not clear to me why that leads to infinite recursion).

Sidenote: If you wanted to post an upstream bug report to GNU coreutils, you are in the wrong place. This repo is a reimplementation of the GNU coreutils. GNU bug reports are created from their mailing list and posted to https://debbugs.gnu.org/cgi/pkgreport.cgi?pkg=coreutils. See https://www.gnu.org/software/coreutils/coreutils.html for more info.

[milahu]

For GNU, there is at least a workaround: -S

weird ...

But uutils env does not support this yet.

#1326

both uutils and GNU have the same problem here?

yes. let's hope that no one depends on this bug ; )

The reason this is necessary is that A=1 python are passed to env as one argument

no. A=1 is correctly parsed as "set env A to 1"

see my example

$ ./coreutils/target/debug/env A=1 python
>>> import os; print(os.environ["A"])
1

so it's the combination of shebang and env that triggers the bug

This repo is a reimplementation of the GNU coreutils.

yes. i feel it's easier to fix the bug here first, and then send a patch to upstream

[tertsdiepraam]

The reason this is necessary is that A=1 python are passed to env as one argument

no. A=1 is correctly parsed as "set env A to 1"

Yeah, sorry, I meant "are passed as one argument in a shebang" :). It is indeed correct when not used as a shebang. See section 23.2.2 here: https://www.gnu.org/software/coreutils/manual/html_node/env-invocation.html

[milahu]

aaah, so it's a limitation of shebang
https://en.wikipedia.org/wiki/Shebang_(Unix)#Character_interpretation

#! /usr/bin/printf A=1 '(%s) ' 1 2 3 4

prints

A=1 '(./test.sh) ' 1 2 3 4

because A=1 '(%s) ' 1 2 3 4 is parsed as one string
which makes the env -S feature necessary to pass arguments

#! /usr/bin/env -S printf '(%s) ' 1 2 3 4

prints

(1) (2) (3) (4) (./test.sh) 

so it's surprising that argv[1] == "A=1 python" leads to infinite recursion
while this fails correctly

#! /usr/bin/env hello world

with

/usr/bin/env: ‘hello world’: No such file or directory
/usr/bin/env: use -[v]S to pass options in shebang lines

[thomasqueirozb]

I don't think this is actually a bug and env is working as intended. I know that what I just said sounds wrong, but hear me out first.

test.sh consists of:

#! /path/to/env A=1 echo

When running ./test.sh, argv will become the following:

["/path/to/env", "A=1 echo", "./test.sh"]

Alright, everything is working as intended until now. Now is where it all falls apart.

When env parses argv[1] it notices the = sign and splits it into 2 parts, the first being A and the second being 1 echo. Then it sets A to 1 echo. Since there are no longer any arguments with an =, argv[2] (./test.sh) is the command that is going to be run.

So basically env has correctly set A to 1 echo and then runs ./test.sh again, starting the fork bomb. This is all intended behavior. Weird but intended. If it were modified, there would be no way to set a variable with spaces in it using env.

[milahu]

If it were modified, there would be no way to set a variable with spaces in it using env.

spaces can be quoted ...

$ env '-S printf "a %q z\n" 1 2 3 4'
a 1 z
a 2 z
a 3 z
a 4 z

$ ./target/debug/env '-S printf "a %q z\n" 1 2 3 4'
error: Found argument '-S' which wasn't expected, or isn't valid in this context

... but not escaped

$ env '-S printf a\ %q\ z\n 1 2 3 4'
env: invalid sequence '\ ' in -S

$ env '-S printf a\\ %q\\ z\\n 1 2 3 4'
a\printf: warning: ignoring excess arguments, starting with ‘%q\\’

So basically env has correctly set A to 1 echo and then runs ./test.sh again, starting the fork bomb. This is all intended behavior. Weird but intended.

another difference:
gnu env is using "tail call recursion" which runs forever
uutils env is using "stack recursion" which runs until OOM

compare these:

#! /usr/bin/env A=1 python
# test.py
import os; print(os.environ["A"])

#! ./coreutils/target/debug/env A=1 python
# test.py
import os; print(os.environ["A"])

systemd-run --scope -p TasksMax=10  --user strace -f --trace execve ./test.py 2>&1 | grep -F 'execve("./test.py"'

/usr/bin/env keeps running, because numTasks is always below 10
./coreutils/target/debug/env stops after 10 forks

[chrisdebian]

Just reading through this and wondering if it still needs to be open?

[milahu]

the main issue is fixed by

For GNU, there is at least a workaround: -S

bit this is still an issue:

another difference:
gnu env is using "tail call recursion" which runs forever
uutils env is using "stack recursion" which runs until OOM

possible solution: os.execve

[ChrisDryden]

The issue of using fork instead of exec has been fixed since this PR: #8365 and the -S flag is now supported. The infinite recursion is matching GNU behaviour, so I believe this issue can be resolved.

[milahu]

yes! thanks