startup_regtest: configure clnrest and generate rune

[endothermicdev]

This configures the startup_regtest.sh nodes to enable clnrest + websocket while also providing output on their connection info (an unrestricted rune is generated as well.) Also uses the snap installed bitcoind path if appropriate.

Format of the start_ln output is:
Commands:
l1-cli, l1-log,
l2-cli, l2-log,
bt-cli, stop_ln, fund_nodes
Node Info:
l1 rest: https://127.0.0.1:3110 rune: "lgkWSp0PQK-pkbElLpjcNVQnX7yfEjouJaJHjLuW8w89OA=="
l2 rest: https://127.0.0.1:3111 rune: "sUqGqkevGYG2r1e_JUiz8Me00GhtTv5-IuGk4o9Beyc9OQ=="

Changelog-None

[endothermicdev]

rebased on #6857 and should now check if clnrest is running before generating runes and providing connection urls.

[ShahanaFarooqui]

lgtm.

The only issue I found is that it generates a new rune each time we start the node. Even if the node already has a rune. We can use showrunes to get the already generated rune but it looks an overkill for the script as it is just for developer testing and runs in /tmp.

[ShahanaFarooqui]

ACK 1dfccbb

[tonyaldon]

I just tried to run start_ln with the new changes.

Everything works well with clnrest python dependencies installed but not without the dependencies:

$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 665564
lightningd: Config file /tmp/l1-regtest/config line 6: clnrest-port=3110: unknown option
[1]+  Exit 1                  test -f "/tmp/l$i-$network/lightningd-$network.pid" || $EATMYDATA "$LIGHTNINGD" "--network=$network" "--lightning-dir=/tmp/l$i-$network" "--bitcoin-datadir=$PATH_TO_BITCOIN" "--database-upgrade=true"
[1] 665603
WARNING: eatmydata not found: install it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes
lightningd: Config file /tmp/l2-regtest/config line 6: clnrest-port=3111: unknown option
[1]+  Exit 1                  test -f "/tmp/l$i-$network/lightningd-$network.pid" || $EATMYDATA "$LIGHTNINGD" "--network=$network" "--lightning-dir=/tmp/l$i-$network" "--bitcoin-datadir=$PATH_TO_BITCOIN" "--database-upgrade=true"
grep: /tmp/l5-regtest/log: No such file or directory
grep: /tmp/l5-regtest/log: No such file or directory
timed out parsing log /tmp/l5-regtest/log

In that second case, it seems that the node doesn't start because after disabling clnrest the option clnrest-port is not defined but present in the config file:

lightningd: Config file /tmp/l1-regtest/config line 6: clnrest-port=3110: unknown option

[endothermicdev]

That makes sense. I've updated it to test if clnrest loads before writing the config file. If any dependencies are missing I would expect it to fail there and we can just skip it. I'll squash these commits if this solution works for you.

[tonyaldon]

Now it also works without clnrest dependencies.

But depending on how long it takes for the last node to be running, I got this message in the terminal:

grep: /tmp/l3-regtest/log: No such file or directory
grep: /tmp/l3-regtest/log: No such file or directory

This is due to the following call to clnrest_status in start_ln

wait_for_lightningd "$nodes"
active_status=$(clnrest_status "/tmp/$i-$network/log")

using $i variable set at the end of the for loop in wait_for_lightningd.

If I understand correctly, what you want to do in wait_for_lightningd is to test that the last started node is running. If this the case, I think you can remove wait_for_lightningd and use this patch in start_ln instead:

-	wait_for_lightningd "$nodes"
-	active_status=$(clnrest_status "/tmp/l$i-$network/log")
+	while ! $LCLI --lightning-dir=/tmp/l"$nodes"-"$network" getinfo > /tmp/null 2>&1; do sleep 1; done
+	active_status=$(clnrest_status "/tmp/l$nodes-$network/log")

[niftynei]

Rebased on latest; addressed last comment from @tonyaldon re: index for $i being out of scope/wrong.

ACK 714de43