itest-flake: revocation test, process fails to exit

[Roasbeef]

Somewhat rare, likely a missing select statement (quit chan) on a channel send/receive somewhere?

=== RUN   TestLightningNetworkDaemon/31-of-82/bitcoind/invoice_update_subscription
    test_harness.go:88: Failed: (revoked uncooperative close retribution zero value remote output): exited with error: 
        *errors.errorString unable to restart Dave's node: process did not exit
        /home/travis/gopath/src/github.com/lightningnetwork/lnd/lntest/itest/lnd_test.go:7010 (0x1403bf0)
        	testRevokedCloseRetributionZeroValueRemoteOutput: t.Fatalf("unable to restart Dave's node: %v", err)
        /home/travis/gopath/src/github.com/lightningnetwork/lnd/lntest/itest/test_harness.go:112 (0x1397cae)
        	(*harnessTest).RunTestCase: testCase.test(h.lndHarness, h)
        /home/travis/gopath/src/github.com/lightningnetwork/lnd/lntest/itest/lnd_test.go:11702 (0x144a6cf)
        	TestLightningNetworkDaemon.func4: ht.RunTestCase(testCase)
        /home/travis/.gimme/versions/go1.16.3.linux.amd64/src/testing/testing.go:1193 (0x52232f)
        	tRunner: fn(t)
        /home/travis/.gimme/versions/go1.16.3.linux.amd64/src/runtime/asm_amd64.s:1371 (0x472001)
        	goexit: BYTE	$0x90	// NOP
=== RUN   TestLightningNetworkDaemon/32-of-82/bitcoind/multi-hop_htlc_error_propagation
=== RUN   TestLightningNetworkDaemon/01-of-82/bitcoind/test_multi-hop_htlc/committype=anchors
=== RUN   TestLightningNetworkDaemon/01-of-82/bitcoind/test_multi-hop_htlc/committype=anchors/local_force_close_immediate_expiry
=== RUN   TestLightningNetworkDaemon/71-of-82/bitcoind/sendpayment_amp_invoice/external_payaddr
=== RUN   TestLightningNetworkDaemon/33-of-82/bitcoind/reject_onward_htlc
    test_harness.go:88: Failed: (revoked uncooperative close retribution zero value remote output): exited with error: 
        *errors.errorString unable to shutdown Dave: process did not exit
        /home/travis/gopath/src/github.com/lightningnetwork/lnd/lntest/itest/lnd_test.go:604 (0x13e209d)
        	shutdownAndAssert: t.Fatalf("unable to shutdown %v: %v", node.Name(), err)
        /home/travis/.gimme/versions/go1.16.3.linux.amd64/src/runtime/panic.go:613 (0x437345)
        	Goexit: done := runOpenDeferFrame(gp, d)
        /home/travis/.gimme/versions/go1.16.3.linux.amd64/src/testing/testing.go:741 (0x52043c)
        	(*common).FailNow: runtime.Goexit()
        /home/travis/.gimme/versions/go1.16.3.linux.amd64/src/testing/testing.go:815 (0x520e13)
        	(*common).Fatalf: c.FailNow()
        /home/travis/gopath/src/github.com/lightningnetwork/lnd/lntest/itest/test_harness.go:88 (0x1397b65)
        	(*harnessTest).Fatalf: h.t.Fatalf("Failed: (%v): exited with error: \n"+
        /home/travis/gopath/src/github.com/lightningnetwork/lnd/lntest/itest/lnd_test.go:7010 (0x1403bf0)
        	testRevokedCloseRetributionZeroValueRemoteOutput: t.Fatalf("unable to restart Dave's node: %v", err)
        /home/travis/gopath/src/github.com/lightningnetwork/lnd/lntest/itest/test_harness.go:112 (0x1397cae)
        	(*harnessTest).RunTestCase: testCase.test(h.lndHarness, h)
        /home/travis/gopath/src/github.com/lightningnetwork/lnd/lntest/itest/lnd_test.go:11702 (0x144a6cf)
        	TestLightningNetworkDaemon.func4: ht.RunTestCase(testCase)
        /home/travis/.gimme/versions/go1.16.3.linux.amd64/src/testing/testing.go:1193 (0x52232f)
        	tRunner: fn(t)
        /home/travis/.gimme/versions/go1.16.3.linux.amd64/src/runtime/asm_amd64.s:1371 (0x472001)
        	goexit: BYTE	$0x90	// NOP
=== RUN   TestLightningNetworkDaemon/34-of-82/bitcoind/node_announcement
=== RUN   TestLightningNetworkDaemon/72-of-82/bitcoind/send_multi_path_payment
=== RUN   TestLightningNetworkDaemon/35-of-82/bitcoind/node_sign_verify
=== RUN   TestLightningNetworkDaemon/01-of-82/bitcoind/test_multi-hop_htlc/committype=anchors/receiver_chain_claim
=== RUN   TestLightningNetworkDaemon/36-of-82/bitcoind/derive_shared_key
=== RUN   TestLightningNetworkDaemon/37-of-82/bitcoind/async_payments_benchmark
    lnd_test.go:11679: 
        	Error Trace:	lnd_test.go:11679
        	            				panic.go:613
        	            				testing.go:741
        	            				testing.go:815
        	            				test_harness.go:88
        	            				lnd_test.go:604
        	            				panic.go:613
        	            				testing.go:741
        	            				testing.go:815
        	            				test_harness.go:88
        	            				lnd_test.go:7010
        	            				test_harness.go:112
        	            				lnd_test.go:11702
        	Error:      	Received unexpected error:
        	            	process did not exit
        	Test:       	TestLightningNetworkDaemon/47-of-82/bitcoind/revoked_uncooperative_close_retribution_zero_value_remote_output
=== CONT  TestLightningNetworkDaemon
    lnd_test.go:11710: Failure time: 2021-07-01 23:56:49.320
--- FAIL: TestLightningNetworkDaemon (469.17s)
    --- PASS: TestLightningNetworkDaemon/41-of-82/bitcoind/switch_offline_delivery_persistence (53.82s)
    --- PASS: TestLightningNetworkDaemon/42-of-82/bitcoind/switch_offline_delivery_outgoing_offline (37.56s)
    --- PASS: TestLightningNetworkDaemon/43-of-82/bitcoind/revoked_uncooperative_close_retribution (29.90s)
    --- PASS: TestLightningNetworkDaemon/44-of-82/bitcoind/failing_link (22.20s)
    --- PASS: TestLightningNetworkDaemon/45-of-82/bitcoind/garbage_collect_link_nodes (101.64s)
    --- PASS: TestLightningNetworkDaemon/46-of-82/bitcoind/abandonchannel (19.06s)
    --- FAIL: TestLightningNetworkDaemon/47-of-82/bitcoind/revoked_uncooperative_close_retribution_zero_value_remote_output (202.46s)

[yyforyongyu]

I think that all the process did not exit flakes are related to deadlocks, which would be fixed by #5642

[guggero]

Haven't seen this in a while. Assume fixed by #5642.