Rewrite integration test harness ensuring interop with CLN, Eclair, and LND

[tnull]

While we have integration tests with CLN and LND, they so far cover only very basic flows.

We should create a more sophisticated test harness (probably using proptest?) that also adds Eclair interop coverage covers more and more error prone scenarios, such as:

a) peer disconnection and reconnection
b) splicing
c) channel closing
d) fee rate disagreements

etc.

[febyeji]

Hi, I've been working on this issue and wanted to share a quick update. So far I've covered:

• An ExternalNode trait that provides a common interface for CLN, LND, and Eclair (connect, disconnect, open/close channel, create/pay invoice, keysend, etc.)
• Shared test scenarios in scenarios.rs that are generic over ExternalNode implementation side
• Add Eclair test coverages
• A single docker-compose.yml with profiles (cln, lnd, eclair) replacing the separate compose files

For the remaining items, I ran into some constraints and would appreciate any guidance:

• Splicing: Our current Eclair Docker image is release-0.8.0, but it looks like splicing is introduced in v0.10.0. CLN v23.08 does have experimental support behind the experimental-splicing flag, and LND doesn't support it yet. Would it be okay if I introduce version updates in the upcoming PR?
• Fee rate disagreements: I'm thinking the most straightforward approach would be to inject divergent FeeEstimator values across nodes. Would this be the right direction, or is there a more realistic scenario you'd prefer to cover?

Would love to hear any thoughts or suggestions on how to approach these two.

[tnull]

Cool, that sounds pretty great, feel free to open a draft PR when you're ready for a first look!

• An ExternalNode trait that provides a common interface for CLN, LND, and Eclair (connect, disconnect, open/close channel, create/pay invoice, keysend, etc.)

How will this handle that different implementations won't support all features? E.g., LND doesn't support Splicing and BOLT12 tests so far, so wouldn't implementing the trait lead to a lot of unimplemented! methods? I guess that's fine in test code, but we'd still need to select which subset of tests run for which implementation, no?

• Splicing: Our current Eclair Docker image is release-0.8.0, but it looks like splicing is introduced in v0.10.0. CLN v23.08 does have experimental support behind the experimental-splicing flag, and LND doesn't support it yet. Would it be okay if I introduce version updates in the upcoming PR?

I think we should right away use the most recent docker images. IIUC, for Eclair it's mostly an issue of tagging, as https://hub.docker.com/r/acinq/eclair/tags shows that latest has been updated just a few days ago, while 0.8 is 3 years old. We should just use latest which is what I did in the draft over at #785. For CLN we should just run it experimental-splicing and other 'experimental' flags required (eg. onion message forwarding might be one).

• Fee rate disagreements: I'm thinking the most straightforward approach would be to inject divergent FeeEstimator values across nodes. Would this be the right direction, or is there a more realistic scenario you'd prefer to cover?

Ugh, no, let's not go down that road. Rather, we should use the test cases to identify why the fee rate disagreements happen and fix them rather than patching them up/working around them in tests. I believe @tankyleo wanted to look into making corresponding adjustments in LDK recently.

[febyeji]

Thanks for the detailed feedback!

• ExternalNode trait & feature gaps: The trait only includes common operations all three support. My current plan is that feature-specific tests (BOLT12, splicing) would live in per-implementation test files, not in the shared scenarios.
• Docker images: I'll switch to latest for Eclair and add the experimental flags for CLN (experimental-splicing, etc.) in the upcoming PR. I'll base the image setup on what you have in Add Eclair integration tests #785.
• Fee rate disagreements: I'll drop the divergent FeeEstimator idea. I'll use the tests to surface the actual fee rate mismatches and document what we find.

[tnull]

• Docker images: I'll switch to latest for Eclair and add the experimental flags for CLN (experimental-splicing, etc.) in the upcoming PR. I'll base the image setup on what you have in Add Eclair integration tests #785.

Yeah, feel free to take this as a starting point, but I'm happy to close this draft as soon as the superseding PR is open!

[febyeji]

Eclair latest (0.14.0-SNAPSHOT) requires Bitcoin Core 29+, but our test infra uses blockstream/bitcoind:27.2. Should we bump bitcoind to 29 (need to verify CLN/LND compat), or stay with release-0.8.0 for now?

[tnull]

Eclair latest (0.14.0-SNAPSHOT) requires Bitcoin Core 29+, but our test infra uses blockstream/bitcoind:27.2. Should we bump bitcoind to 29 (need to verify CLN/LND compat), or stay with release-0.8.0 for now?

Yes, please bump to at least 29 then, if possible we should even bump to 30.2 (see latest tag here: https://hub.docker.com/r/blockstream/bitcoind/tags)