routing: proportional attempt cost

[joostjager]

In #2802 probability based routing was introduced. One of the control parameters introduced is the payment attempt cost.

The payment attempt cost is a virtual cost that is assigned to a failed payment attempt. This parameter influences the trade off between cheap and reliable routes. Specifically it adds an extra term to the pathfinding cost function of attempt_cost / probability.

Probability estimation is based on past failures and successes. The impact of failures on probability estimates diminishes relatively quickly (default "half life time" is one hour), but success experiences keep having a constant effect on probability until a failure happens.

For a presentation about mission control, see https://twitter.com/joostjgr/status/1186177262238031872.

For nodes that aren't continuously sending payments and use default mission control parameters, it is likely that failures don't have much impact on mission control performance beyond the currently active payment. For those nodes, recorded information about previously successful routes is the main driver for pathfinding.

The probability estimate is 60% (by default) for unknown hops (or hops with older failures recorded) and 95% for previously successful hops.

Suppose:

• There is a route S through a success hop with routing fee 0.1%
• There is a route U though an unknown hop with routing fee 0.01%
• We are sending 4M sats

What should the attempt cost c be set to for pathfinding to choose route S?
4M * 0.1% + c / 0.95 = 4M * 0.01% + c / 0.6
Solving for c results in an attempt cost = 5863 sats.

So we need to set a pretty high attempt cost to not get distracted by unknown routes. In the reality of the Lightning Network, these unknown routes do often fail and cause a lengthy payment process. Good liquidity isn't the cheapest liquidity generally.

The current default attempt cost is set at 100 sat. For the scenario described above with a relatively large gap in fees and a mid-size payment amount, the default setting would effectively disable probability based routing and lead to a lot of exploration even though successful routes may be known.

One thing that becomes clear is that an absolute attempt cost is unlikely to work for every payment amount. This PR therefore introduces an additional control parameter that allows setting a proportional attempt cost as well. The goal of this is to improve the default pathfinding performance in particular for large payment amounts.

[joostjager]

An idea brought up by @halseth in the past is to bump up the payment attempt cost. So similar to what is sometimes done with RBF, start with a low payment attempt cost (go for the cheapest route), but if that turns out to be difficult, start bumping the attempt cost up to move towards reliable routes.

[cfromknecht]

Solid change, definitely makes sense to begin exposing this level control as the average channel/payment size becomes larger! Diff is otherwise straightforward 👍

An idea brought up by @halseth in the past is to bump up the payment attempt cost. So similar to what is sometimes done with RBF, start with a low payment attempt cost (go for the cheapest route), but if that turns out to be difficult, start bumping the attempt cost up to move towards reliable routes.

this does seem like a good direction to explore. it does seem to add a good amount of complexity compared to your proposed changes. it would be interesting if the fee ratcheting could explore the fixed and proportional coefficients simultaneously...

[cfromknecht]

should this match the default value?

[joostjager]

It seems that we use other values than the default in this file. Probably to demonstrate what it looks like with a non-default value.

[halseth]

Good change! Definitely sounds like this is a good first step, then we can explore the more complex dynamic attempt cost at a later point.