[bug]: btc locked in chain of sweep transactions that have not been broadcast

[mrfelton]

Background

Hi, I seem to have some bitcoin that is encumbered in transactions that have not been broadcast from our node and I'd like to understand why, and how the locked BTC can be unlocked?

The following sweep transaction (found in lncli listchaintxns) was never broadcast. It is related to a remote channel force closure:

{
            "tx_hash":  "a123546ceec63298521b2bf22e0495f5551e401b2110a8a4ec8e555213581bff",
            "amount":  "147",
            "num_confirmations":  0,
            "block_hash":  "",
            "block_height":  0,
            "time_stamp":  "1702627098",
            "total_fees":  "0",
            "dest_addresses":  [
                "bc1pzxkzwk4y06hvkxyw4ha468666255c4tn3ftklfplnxe2cpr23drs48y79r"
            ],
            "output_details":  [
                {
                    "output_type":  "SCRIPT_TYPE_WITNESS_V1_TAPROOT",
                    "address":  "bc1pzxkzwk4y06hvkxyw4ha468666255c4tn3ftklfplnxe2cpr23drs48y79r",
                    "pk_script":  "512011ac275aa47eaecb188eadfb5d1f5ad2a94c55738a576fa43f99b2ac046a8b47",
                    "output_index":  "0",
                    "amount":  "24960489",
                    "is_our_address":  true
                }
            ],
            "raw_tx_hex":  "02000000000102b61d9accd88f48003168bbe3b78de5df60363bc157b1b938012c109e0557f7a50100000000000000003588a04f920fb4e2ae3cd4cf4000d4777d0bf69e4025efff3cdef1899c61831300000000000000000001e9dd7c010000000022512011ac275aa47eaecb188eadfb5d1f5ad2a94c55738a576fa43f99b2ac046a8b47024830450221009e76c627d8fdc290360a10e5e17eb2f588d74861771c23629c17a61231841586022011349ae811d3478298aa9d745085f4ed9894e64abfb8cf33ce85120dc13ed34f01282102f0a13731bc87f9476c645644fb922cd9dcbf02067968a32a7d8cc56c09f4c52bac736460b26801402fa73766df0fcc2ac7c0e4da1ddb10576fe0a9783e2b9e15864af492090405ea6bf0111297df2ad8f9cd56007f60cabb52bf20e14204262046713b199ec3b1d715880c00",
            "label":  "0:sweep",
            "previous_outpoints":  [
                {
                    "outpoint":  "a5f757059e102c0138b9b157c13b3660dfe58db7e3bb683100488fd8cc9a1db6:1",
                    "is_our_output":  false
                },
                {
                    "outpoint":  "1383619c89f1de3cffef25409ef60b7d77d40040cfd43caee2b40f924fa08835:0",
                    "is_our_output":  true
                }
            ]
        },

Another sweep transaction of ours spends the above utxo, and was also never broadcast:

{
            "tx_hash":  "2b44db45c5655f8f28043a7b92f72b6829f0b35f047df1b4f23432c7d7822f2b",
            "amount":  "-7787",
            "num_confirmations":  0,
            "block_hash":  "",
            "block_height":  0,
            "time_stamp":  "1703307606",
            "total_fees":  "7787",
            "dest_addresses":  [
                "bc1peqpew6n5helm4tu23wjj9uat3unwuh8krksv9rxvhj58qqzpe8asdtnj8s"
            ],
            "output_details":  [
                {
                    "output_type":  "SCRIPT_TYPE_WITNESS_V1_TAPROOT",
                    "address":  "bc1peqpew6n5helm4tu23wjj9uat3unwuh8krksv9rxvhj58qqzpe8asdtnj8s",
                    "pk_script":  "5120c803976a74be7fbaaf8a8ba522f3ab8f26ee5cf61da0c28cccbca8700041c9fb",
                    "output_index":  "0",
                    "amount":  "24952702",
                    "is_our_address":  true
                }
            ],
            "raw_tx_hex":  "02000000000101ff1b581352558eeca4a810211b401e55f595042ef22b1b529832c6ee6c5423a1000000000000000000017ebf7c0100000000225120c803976a74be7fbaaf8a8ba522f3ab8f26ee5cf61da0c28cccbca8700041c9fb0140951a111240a2c3113cdc36d60c9b24d492450473201481b93c16e58b91d61b834ce05b24be4e96fda575b4499fa8a34eb16763daa95fb44f29c8d781ff105825f68c0c00",
            "label":  "0:sweep",
            "previous_outpoints":  [
                {
                    "outpoint":  "a123546ceec63298521b2bf22e0495f5551e401b2110a8a4ec8e555213581bff:0",
                    "is_our_output":  true
                }
            ]
        },

I'm not sure I fully understand what happened here, but as I understand it, the channel partner initiated a force closure (https://mempool.space/tx/a5f757059e102c0138b9b157c13b3660dfe58db7e3bb683100488fd8cc9a1db6), and some time after they did CPFP to finalise the closure (CPFP: https://mempool.space/tx/5de3df72bf2fe1b23dffca4e3ceef1d6dd80288464f8145b5a363689727b8a31).

We attempted sweep from that channel close but that was never broadcast for some reason (perhaps because the remote party already swept it), and then it seems we have attempted to spend the unbroadcast sweep output in another sweep that was also never broadcast.

The result seems to be that we have 2 chained sweep transactions on our node that have never been broadcast, and 0.24952702 BTC that is not available to the wallet as it seems to be encumbered in those two transactions that have not been broadcast.

I'm unable to rebroadcast either of the two transactions manually. The first fails to broadcast because the fee rate is too low. The second fails to broadcast because it spends an output from the first, which has not been broadcast.

Can someone help me understand what actually happened here, and how it can be resolved? Please let me know any other information I can provide to help debug, Thanks!

Your environment

• version of lnd: 0.17

[mrfelton]

Here are the logs relating to the first unboradcast sweep:

Note the last log entry: publishAttempts(10) exceeds MaxSweepAttempts(10), removed.

And here are logs relating to the second unbroadcast sweep:

[ziggie1984]

I'm unable to rebroadcast either of the two transactions manually. The first fails to broadcast because the fee rate is too low. The second fails to broadcast because it spends an output from the first, which has not been broadcast.

Can you check which of the two tx is in your mempool:
a123546ceec63298521b2bf22e0495f5551e401b2110a8a4ec8e555213581bffor dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282

[mrfelton]

a123546ceec63298521b2bf22e0495f5551e401b2110a8a4ec8e555213581bff is in my bitcoind node's mempool.

dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282 is not.

fwiw, my mempool size is set to 1200 mb.

[ziggie1984]

Can you publish dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282 via bitcoin-ci cmd and see whether it RBFs your other one ?

[mrfelton]

I looked up the hex for that transaction here https://mempool.space/api/tx/dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282/hex and tried to publish it using my bitcoind node, but I get an error as follows:

bitcoin-cli sendrawtransaction "$(cat transaction.txt)"
error code: -26
error message:
insufficient fee, rejecting replacement dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282, less fees than conflicting txs; 0.0000259 < 0.0000797

[ziggie1984]

Ok I can explain it to you, somehow you lightning node/ bitcoind backend did not get the RBF tx of your output in time and you were able to create CPFP tx from the original a123546ceec63298521b2bf22e0495f5551e401b2110a8a4ec8e555213581bff which increase the RBF requirements for dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282 . But all other bitcoin nodes have already dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282 which means your tx package will never get broadcasted.

Things you can do, set the min_relay_fee of the bitcoind node to something like 2(startup setting), than your tx should be purged from the mempool and dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282 should be able to get into it. After that you still need to restart lnd so that it is aware of all new unconfirmed tx. Lnd 18 will have the option removetx which lets you remove those tx via rpc call.

I think the newest version of bitcoind supports also package relay or ?

Maybe you don't even have to restart it, just try the relay fee setting first and see how your lnd utxos change (the unconfirmed ones)

[mrfelton]

That makes sense.

I set minrelaytxfee=0.00001 in my bitcoind conf and restarted the bitcoin node.

I then manually broadcast the missing RBF transaction. I actually broadcast ee357f4af46d9bf4370111f615e9029373e6ba38f3d6c2aa1b158012a3eed0fc rather than dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282 as I noticed that they did a fresh RFB on that a few hours ago.

I then checked the bitcoind mempool and found that dad763247ca651e70b62139862c6d8a68dda19eb4296f87e2fec952fb0427282 was in there, and my orphaned transactions a123546ceec63298521b2bf22e0495f5551e401b2110a8a4ec8e555213581bff and 2b44db45c5655f8f28043a7b92f72b6829f0b35f047df1b4f23432c7d7822f2b were no longer there.

LND did not pick up the changes to the mempool or adjust it's understanding of the updated UTXO set automatically, but after restarting LND it then had the correct/expected state and I no longer have the 0.25 BTC encumbered in those orphaned/unbroadcast double spend's

Thank you for you help!

Without my intervention here, presumably, this would have automatically self healed once the remote parties sweep transaction finally confirms?

[ziggie1984]

Without my intervention here, presumably, this would have automatically self healed once the remote parties sweep transaction finally confirms?

Exactly, because we do not monitor the mempool for RBFs currently. But it could have taken a long time until it would have been confirmed.

But maybe we are planning to stop sweeping the anchors when feerates are that high see also #7939

I will close the issue, feel free to open a discussion if you still have some open questions.