Add closedchannels RPC

[rorp]

This PR allows to access the historic channel data without relying on third party services like LN explorers.

[t-bast]

Can you explain why you think this is useful?

Please note that the API is strictly for managing your node and we don't want to end up maintaining too many unused APIs. Exotic use-cases / analysis should run directly on the DB (and ideally on the read-only replicated DB to avoid impacting the running node).

[rorp]

Can you explain why you think this is useful?

First of all, it's going to make me as famous as Rusty Russell is (well, sorta :)) via Bitcoin Optech Newsletter: https://bitcoinops.org/en/newsletters/2023/04/05/#core-lightning-5967

Second, in addition to providing more control over the node to the users, it helps to retain the users' privacy.

Eg.: every time I receive a "channel closed" notification on my Nostr alarm bot, the closed channel disappears from the list that returns the channels API call, and the only way to find out which peer closed the channel, what was the cause, what was the balance, etc is looking at the channel with a Lightning explorer like 1ML or Amboss, leaking my private data in the process. You cannot see the exact balance though. However, I have all the data I need right here at my fingertips, but I have no way to read them.

Please note that the API is strictly for managing your node and we don't want to end up maintaining too many unused APIs.
Exotic use-cases / analysis should run directly on the DB (and ideally on the read-only replicated DB to avoid impacting the running node).

What is so exotic about knowing what happened with my own money? It's an essential part of managing my node. As I'm a node op, I know that very well.

The code and the data already exist. Just a few lines of code are needed to put them together.

Also, it's not the first time you suggest to access the database directly. There are 2 problems though:

1. I use Sqlite backend. Eclair locks eclair.sqlite exclusively, other processes cannot access it. No luck here...
2. The channel database contains raw data blobs. Their structure is undocumented and will change without notice (as it has happened before). And this is the REAL problem.

Although JSON format isn't documented either, one can use their natural (or artificial for some) intelligence to figure out the structure.

I use Python for my small automation scripts. How should I write a script that tells if the channel force closed using this data?

sqlite> select * from local_channels where is_closed=1 limit 1;
K? ????5?R??z?	?1 ?N?] ????||1|1681668108044||||1681668156943

With this PR I can do

channel = json.loads('{ closed channel JSON }')
if not channel['data']['mutualClosePublished']:
    print("force closed")

[t-bast]

Fair enough! The list of closed channels is an always-growing list that will eventually be huge, so I think we shouldn't allow listing everything at once, this is too footgunny. What about adding pagination and making the count parameter mandatory?

[codecov-commenter]

Codecov Report

Merging #2642 (f82a2c5) into master (e7b4631) will increase coverage by 0.00%.
The diff coverage is 70.00%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@           Coverage Diff           @@
##           master    #2642   +/-   ##
=======================================
  Coverage   85.88%   85.88%           
=======================================
  Files         214      214           
  Lines       17591    17613   +22     
  Branches      728      738   +10     
=======================================
+ Hits        15108    15127   +19     
- Misses       2483     2486    +3     

Impacted Files	Coverage Δ
...r-core/src/main/scala/fr/acinq/eclair/Eclair.scala	55.76% <0.00%> (-0.79%)	⬇️
.../main/scala/fr/acinq/eclair/db/DualDatabases.scala	10.94% <0.00%> (-0.17%)	⬇️
...a/fr/acinq/eclair/db/sqlite/SqliteChannelsDb.scala	97.89% <88.88%> (-0.95%)	⬇️
...ain/scala/fr/acinq/eclair/db/pg/PgChannelsDb.scala	99.28% <100.00%> (+0.03%)	⬆️

... and 6 files with indirect coverage changes

[t-bast]

Let's forget about the performance for nodes with too much historical data, it's probably fine for now. We can optimize that later by moving closed channels to their own table.

Needs a rebase!

[t-bast]

Getting close, a few comments on the changes in the DB files, but otherwise looks good to me.

[t-bast]

LGTM, thanks for your patience 🙏

[rorp]

We did it! https://bitcoinops.org/en/newsletters/2023/06/07/#eclair-2642

[t-bast]

Congrats 😄