LND rejects a valid commitment

[pm47]

Background

eclair-lnd channel, an htlc has been failed right after a disconnection, and reconnection lnd rejects a sig which seems valid.

Note the lnd node definitely seems to be having issues (see huge delays between sig and rev below), but that doesn't explain the disagreement over the commitment

Your environment

• version of lnd: bitrefill node 024a2e265cd66066b78a788ae615acdc84b5b0dec9efac36d7ac87513015eaf6ed

Steps to reproduce

A                B
  |<-----add-------| htlc for 9 satoshi
  |<-----sig-------|
  |------rev------>|
  |------sig------>|
  |                | an hour elapses (?)
  |<-----rev-------|
  |------fail----->| we fail immediately
  |------sig------>|
  |                | 10 minutes
  |------fee------>|
  |                | 10 minutes
  |------fee------>|
  |                |
  |     (...)      | 9 update fee in total, one every 10 minutes
  |                |
  |------fee------>|
  |                |
  |<------rev------| eventually we get the revocation two hours later
  |-------sig----->| we sign the fee update
  |<------sig------| we receive a sig with the htlc removed
  |-------rev----->|
  |                |
  |        X       | connection reset
  |                |
  |--reestablish-->|
  |<-reestablish---|
  |-----fee x9---->| we resend all the fees
  |------sig------>| and the sig
  |------rev------>| and the rev
  |<-----err-------| error: rejected commitment

Expected behaviour

lnd accepts the sig.

Actual behaviour

lnd rejects the signature, and provides the tx it expected, which does include the htlc that was previously removed (and which removal was acked by lnd). OTOH this tx has the correct feerate.

This looks like everything from the fail until the disconnection was lost on lnd's side, but shouldn't the state have been persisted to disk?

[alexbosworth]

I see something that seems like a related issue seemingly on LND <> LND as well

This happens when I'm doing HTLCs which are failing back to me. The outcome is a local force close of the channel

lnd.log.403.gz:2018-10-30 01:12:14.720 [ERR] HSWC: ChannelLink(525316:2046:1) Failing link: ChannelPoint(108f1caacc284fb6e34f47d96d0a85645b3c4094efe3686794c6ca7177334ea0:1): received error from peer: rejected commitment: commit_height=1473, invalid_commit_sig=3045022100ddd7b40d5c52a22615c92a8ce4691aa32277cb17ee349c1d41ba04496241ee5502206bcf71ee8fe6dbde51a5ab7cfd34c11c65572eebeb73c2a575f6c64fa154338e, commit_tx=0200000001a04e337771cac6946768e3ef94403c5b64850a6dd9474fe3b64f28ccaa1c8f10010000000031ab5280040679010000000000220020a08ab5f981b2f79da8143aea28d90c722688b3ec766831059e3ba1a11977e1acf1490200000000002200203ec1bcbeff5d4fdb4fe4874220a9cadf8d9fd9e5cb0405667fa4b6a54beb7128f1490200000000002200206cbeb1872db9d48960cb7f2d1c9166551d51326c48d27cba40e9e52024fd19ea440f0300000000001600146cdae41ff2bf52a17e80613cd91ee8abf20c3cfda0a3c720, sig_hash=03489a05926139c4eae84d9cf7b8d1ee2fc6af4b443d4039488025718b882d1c
lnd.log.403.gz:2018-10-30 01:30:21.398 [ERR] HSWC: ChannelLink(547300:2128:1) Failing link: ChannelPoint(1c0930b4a28c44ac54890d7d206e1b04a96202307dea519317329abc528afd5e:1): unable to accept new commitment: rejected commitment: commit_height=22, invalid_commit_sig=3044022033986a4da9cdaf9288301d8cd65322185528f5094864bf8ffffc12ee2f1ad92802202841b11a0a1db2b8d868ec863f629e5b02f788d341ab587936bef82ec21abe87, commit_tx=02000000015efd8a52bc9a32179351ea7d300262a9041b6e207d0d8954ac448ca2b430091c01000000001fb9348001df7a1e0000000000220020d932afd696882abc40dbff4443a577b8136142b56eb9863471998c84de69d320653ffe20, sig_hash=78acc8ae0155afc6c92cc4253fcca761bd47caabeb9496179950da890dbb04d3
lnd.log.403.gz:2018-10-30 01:45:43.873 [ERR] HSWC: ChannelLink(543977:2048:1) Failing link: ChannelPoint(7ac2645c3f92e689f2256a17fb642bd5dd35800a65577a5072604b3fdea61f78:1): unable to accept new commitment: rejected commitment: commit_height=228, invalid_commit_sig=30450221008c862fa025f756218c46dcdde4056d3ffa83453de910b92abc65a3cfa2959781022037269b1c13ce07c448dac4cbe3ad3646487f3eef933971807968ff4ccb31659d, commit_tx=0200000001781fa6de3f4b6072507a57650a8035ddd52b64fb176a25f289e6923f5c64c27a0100000000c7af81800247641800000000002200207e2d4756ad022d5b5651716c1d57b266f08dee8dc0ead9e0083b2f7c5f93c352189b240000000000160014fdbbe928f3e7f96ecb9748ceea36ec879a750d7cd2178020, sig_hash=c2a2b139441db2b5935ccaba50a4153b9361454857a51aa1f1a9ec33af8d414d
lnd.log.403.gz:2018-10-30 01:45:43.875 [ERR] HSWC: ChannelLink(543964:1299:0) Failing link: ChannelPoint(7b9bec1fc2dd0e44963310d6448e17d1eb705200a02293f8dae43b4c0f311ac0:0): unable to accept new commitment: rejected commitment: commit_height=200, invalid_commit_sig=304402204026226955bdfb5c00008b88f2e7991c9b22d52d09ea27c2cce0490275bdcadc02206382464b445d9a6f7b76023ff527c98a39e1c0fc3fbd254aa785f97787779fad, commit_tx=0200000001c01a310f4c3be4daf89322a0005270ebd1178e44d6103396440eddc21fec9b7b000000000027ebad8002688c030000000000220020c306135cfeeaaae8b687094711af6b8bcbeb5d60d1dbf0610a7ac42da484fcd14fa80b0000000000160014503cb3b154e3f18e00bb084fa418ed32b41c168585b97220, sig_hash=cc6a3b94002210c9548a4213daadb49f5fefa24e2072bb9e06d578b74ab6a34b
lnd.log.403.gz:2018-10-30 01:45:43.876 [ERR] HSWC: ChannelLink(544578:984:1) Failing link: ChannelPoint(40e02f51185b6dc57c8cdc6a1648b46f7716faf23882c0d3b090db5e3636c2bc:1): unable to accept new commitment: rejected commitment: commit_height=148, invalid_commit_sig=304402201aa93c3b219237f9dc373a9308bcd22ff1b3453f63d288df1bacb4ee72af2e2c0220222574ff26e1c9f4efda47193b2f22a8c175ca1a1fdfe6d647c1279c7d1d0752, commit_tx=0200000001bcc236365edb90b0d3c08238f2fa16776fb448166adc8c7cc56d5b18512fe040010000000056a75280019f414c0000000000220020e733cad7bf7505c772f58b6daa1a7714828b785528d8c054c8a891f14d997cd116c92e20, sig_hash=8e7e9596b21877cb41f0f2a300dec62eec4d921d5f13045caf8751b5e0d56c79

[juscamarena]

Here are all the cases I see of this in the last few weeks: https://paste.ubuntu.com/p/KvMW9pWGqV/

[Roasbeef]

This happens when I'm doing HTLCs which are failing back to me

What do you mean by this?

[alexbosworth]

The HTLCs do not succeed, they resolve in timeouts

[halseth]

Could be that the nodes disagree on the commitment view somehow, and therefore validate the signature against different commitments.

It is hard to tell from these errors though. Ideally we should get Trace logs from the two nodes when this happens, as they print the commitments before signing/validating. Alternatively we could promote these to Debug until this is sorted out.

[halseth]

@alexbosworth are you paying through a channel where you control both sides of the channel? In that case wallet trace logs (debuglevel --level=LNWL=trace) should tell us what we need to know.

[alexbosworth]

No this hasn’t happened to me in a situation where I control both sides

On Thu, Nov 1, 2018 at 4:40 AM Johan T. Halseth ***@***.***> wrote: @alexbosworth <https://github.com/alexbosworth> are you paying through a channel where you control both sides of the channel? In that case wallet trace logs (debuglevel --level=LNWL=trace) should tell us what we need to know. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1908 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAYiIdx_SxXi2eBaVifhCHr5_QgU8vPRks5uqt2zgaJpZM4WpmM_> .

-- Sent from my iPhone

[Talkless]

I have upgraded from vanilla 0.5 to today's master, and my lnd force-closed channel with Joltfun LND node after about an hour of running:

Nov 03 13:25:56 odroid-hc1 lnd[31982]: 2018-11-03 13:25:56.246 [ERR] HSWC: ChannelLink(537688:1737:0) Failing link: ChannelPoint(...): unable to accept new commitment: rejected commitment: commit_height=662, invalid_commit_sig=..., sig_hash=...
Nov 03 13:25:56 odroid-hc1 lnd[31982]: 2018-11-03 13:25:56.246 [ERR] HSWC: ChannelLink(537688:1737:0) link failed, exiting htlcManager