Automatic InstantSend locks for "simple" transactions by gladcow · Pull Request #2140 · dashpay/dash

gladcow · 2018-06-21T17:23:46Z

All "simple" transactions (now they are transactions with less than MAX_INPUTS_FOR_AUTO_IX inputs) are locked after creation in wallet (GUI or RPC calls) or after receiving by other node in CInv message.

Such automatic locks are impossible with special InstantSend fees, so these fees are removed (this change requires protocol bump, only new nodes will accept InstantSend with small fee).

Transactions created and sent with rawtransaction RPC API are not locked by sending node (but they will be locked by other nodes after receiving this transaction).

Also, `locktransaction' RPC call was added. It allows any node to lock any transaction.

PastaPastaPasta · 2018-06-21T18:44:24Z

@gladcow Can you go into a little bit of detail about what this does? What does a 'lock' in this context mean?

thephez · 2018-06-21T18:54:04Z

If I'm understanding correctly, it:

Removes the IS fee 🎉
Automatically attempts to IS lock any tx with less than a certain number of inputs. And allows the txlock to be requested by any node that receives a compatible tx (not just the sending one)

If (2) is correct, it means that wallets wouldn't have to be aware of IS to send IS txs (because a receiving node could request the txlock). 😲

gladcow · 2018-06-21T19:09:29Z

@paulied , all usual transactions become InstantSend transactions even if they are created by the wallet that doesn't support InstantSend, for example.

@thephez, yes, you are right.)

PastaPastaPasta · 2018-06-21T19:10:39Z

Are quorums able to do that so quickly at this stage? Our network could get up to 22 tx/seconds. My understanding was that IS resulted in huge bandwidth, and as such could not be on by default.

thephez · 2018-06-21T19:19:36Z

@paulied There is some info on Discord. Summarizing - basically previous refactoring has been working toward this, intensive testing required, and it probably will not scale to all txs (but future enhancements could make it even more scalable).

UdjinM6 · 2018-06-21T19:40:32Z

To address concerns about bandwidth - we tested 2MB blocks filled with IS txes on testnet and network handled it just fine. We'll test it again with these changes too of course, but in general, if you are running mainnet full node you should probably be prepared for bandwidth growth anyway.

Antti-Kaikkonen · 2018-06-21T23:47:26Z

Are there any scenarios where the receiver wouldn't be able to lock the received transaction? If the transaction depends on non-locked transactions then can the receiver just lock all the dependencies? I can imagine that this wouldn't work if the dependencies contain too recent coinbase transactions.

thephez · 2018-06-22T01:12:05Z

@Antti-Kaikkonen

Are there any scenarios where the receiver wouldn't be able to lock the received transaction?

Sure, but a valid transaction would just continue to be a normal transaction (instead of getting "upgraded" to an IS one by the receiver). Similar to how a failed IS request today fails-back to being a regular tx (as long as it meets all the normal tx rules).

Antti-Kaikkonen · 2018-06-22T09:39:40Z

Sure, but a valid transaction would just continue to be a normal transaction

I think that this update is great but it would be even better if the receiver could somehow ensure that all transactions can be locked. Are there any plans to achieve this? For example IS only addresses were discussed at some point.

gladcow · 2018-06-22T10:16:56Z

@Antti-Kaikkonen:I think it is better to create special issue with feature request to discuss such opportunity.

UdjinM6

I'm pretty sure that with changes like that you are effectively removing non-simple IS txes which is probably far too much. Also, introducing another "type" of IS txes should probably be done with some kind of activation because otherwise you'll fail at both validation, voting and verifying your votes on older nodes which opens an attack vector for splitting the network I think.

UdjinM6 · 2018-06-22T12:16:07Z

UdjinM6 · 2018-06-22T12:16:42Z

UdjinM6 · 2018-06-22T12:21:24Z

Trying to lock blockchain tx makes no sense imo (and will most likely fail).

UdjinM6 · 2018-06-22T12:23:31Z

You can't just drop it like that, should check if there is enough fee OR it's a "simple" tx.

UdjinM6 · 2018-06-22T12:35:25Z

Same, should modify this to handle simple txes correctly instead of dropping this.

UdjinM6 · 2018-06-22T12:36:23Z

This should stay.

UdjinM6 · 2018-06-22T12:36:56Z

This should stay.

krishnawhite · 2018-06-26T09:28:02Z

Would this change make the network vulnerable to a denial of service attack where the attacker spams the network with low-fee IS transactions in order to fill the mempool and prevent legitimate transactions?

My understanding is that if the mempool becomes full, under normal circumstances there is a fee mechanism which allows the lowest fee non-IS transactions to be dropped in favour of higher fee transactions - dropping non-IS transactions from the mempool is OK because they are not considered to be confirmed until inclusion in a block. However, InstantSend transactions cannot be dropped from the mempool regardless of the fee paid, because they are considered to be guaranteed as soon as the lock has been issued (at which point they are still in the mempool).

Therefore filling the mempool with low-fee IS transactions would prevent any further transactions - even new transactions that pay a high fee would not be able to displace existing low fee transactions as they are considered locked.

At present, there is a higher fee associated with IS transactions, which makes it costly for an attacker to fill the mempool with IS transactions and keep it full as blocks come in. However this proposed change lowers the IS fee and therefore lowers the cost of the attack.

A very crude approximation of the cost involved goes something like this:

Masternode RAM requirement = 1GB = 1,000,000,000 bytes (mempool capacity).
Assume a transaction size of 250 bytes.
Assume a transaction fee of $0.0006 (the current median).

1,000,000,000 / 250 = 4,000,000 transactions max in the mempool.
4,000,000 * $0.0006 = $2400 to fill the mempool with IS-locked spam transactions.

2MB = 2,000,000 bytes per block.
2,000,000 / 250 = 8000 transactions per block.
8000 * $0.0006 = $4.80 every 2.5 minutes, or $2764 per day to sustain the attack.

So the total cost of this attack is in the region of $2400 initially + $2764 per day unless I am missing something important.

gladcow · 2018-06-26T12:08:34Z

@krishnawhite , you are right, now I'm going to check the special mempool conditions to prevent this spam attack.)

PastaPastaPasta · 2018-06-26T12:10:06Z

@krishnawhite another aspect of this to take into account, is how much dash someone needs to hold in order to execute this attack.

Minimum send amount - 546 duffs. based on this, in order to fill the mempool (assuming 4,000,000 txs is accurate) would be 2,184,000,000 duffs, which is 21.84 Dash.

Now, every 2.5 ish minutes you need to refill after a block goes through, about 8000 txs. So for one block that is 4,368,000 duffs per block. Now, since inputs to an IS need 6 confirmations we multiply that number by six. 26208000 duffs. Or just 0.262 Dash.

Add them together and you find someone would need about 22 dash in holdings to be able to in theory pull off this attack.

A potential fix and mitigation, is to set a minimum value for txs that can be confirmed via IS on a low fee

UdjinM6

Getting closer :) 👍

See inline comments.

UdjinM6 · 2018-07-16T18:54:28Z

This will lead to failed "normal" ixes during migration because ranks are going to be messed up for new nodes vs old nodes. Should be reverted imo.

UdjinM6 · 2018-07-16T18:58:43Z

Trx is a bit uncommon abbreviation for transaction. Also, this should probably be a regular non-static function in CTxLockRequest instead, smth like bool CTxLockRequest::IsSimple().

UdjinM6 · 2018-07-16T19:38:22Z

I don't see how trying to lock txes on the receiver side only can be useful tbh, you need the whole network to lock tx to make sure no double-spent tx will ever be mined. This means that you need masternodes to vote on this tx and every other node (or at least vast majority of them) should get these votes and lock tx too. I don't think you can do this via rpc, because once tx is relayed you can't re-relay ix for it and force masternodes to vote. This rpc won't do anything imo and should be removed.

UdjinM6 · 2018-07-16T19:52:05Z

This should probably be a private member of CTxLockRequest.

UdjinM6 · 2018-07-16T19:53:16Z

This should probably be a private member of CInstantSend.

UdjinM6 · 2018-07-17T20:25:06Z

Thinking about this piece of code a bit more and... it probably won't do anything actually. Instead, I think you should process ix and simple txes identically. One way would be to change (strCommand == NetMsgType::TXLOCKREQUEST) conditions into (strCommand == NetMsgType::TXLOCKREQUEST || (strCommand == NetMsgType::TX && CInstantSend::IsTrxSimple(tx) && CInstantSend::CanAutoLock())) here (lines 2001+), a bit above (lines 1949+) and for rejected txes down below (on lines 2131+). This would also require smth like txLockRequest.tx = ptx; on lines 1931+.

However, I'm not 100% sure if these changes would be enough either. We should probably adjust p2p-instantsend.py to include some tests for simple txes (success for simple txes, failure for non-simple regular txes) to be a bit more confident that changes in this PR actually work.

Agree, adding tests now.

sidhujag · 2018-07-31T06:13:58Z

If this means what I think it means it would require 5 or 6 signature checks per transaction (this is partly why it should have higher fees). This will reduce the overall throughput that each mempool node is capable of processing. I would recommend IS or "locking" be opt-in with the higher fee and don't do it under the hood for them, downstream this breaks some other implementation specific stuff that relies on stuff not being locked.

gladcow · 2018-07-31T07:22:50Z

@sidhujag , can you give more details on the stuff that can be broken, please?

gladcow · 2018-08-07T13:40:58Z

I've added separate tests for this feature in p2p-autoinstantsend.py. It reuses some code from p2p-instantsend.py and it is correct to move this common code in some base class, but I think it is better to do this in separate PR to not blur this PR goal.

About the identical processing for simple txes and instantsend txes in net_processing.cpp: such code requires several small hacks (for example we should initialize CTxLockRequest with simple tx pointer without deserializing it from stream), it can cause errors in the future after possible serialization format changes for example. Current code is working (as we have checked with regtest) and is more compact, so I have decided to stick with it. @UdjinM6, tell me, please, if you disagree.

UdjinM6 · 2018-08-09T09:29:24Z

I'm probably missing smth but p2p-autoinstantsend.py fails for me locally... Could you rebase this on top of the latest develop pls?

gladcow · 2018-09-24T13:08:56Z

Rebased and fixed, re-review, please.

UdjinM6 · 2018-09-24T14:50:29Z

Still doesn't quite fix it in the way I suggested #2140 (comment) and I'm not sure if fixes here are enough. Wouldn't this UdjinM6@c04a973 be a better way?

gladcow · 2018-09-24T15:03:20Z

Sorry, I've misunderstood required fixes, will add this commit now

UdjinM6

re-utACK

but would be nice if someone else would also review the latest patch with fresh eyes @nmarley @codablock

nmarley

utACK

PastaPastaPasta

minor formatting, see inline

PastaPastaPasta · 2018-09-25T01:10:19Z

+bool CInstantSend::CanAutoLock()
+{
+    if(!isAutoLockBip9Active)
+        return false;


body same line or brackets

PastaPastaPasta · 2018-09-25T01:10:27Z

+    if(!isAutoLockBip9Active)
+        return false;
+    if(!sporkManager.IsSporkActive(SPORK_16_INSTANTSEND_AUTOLOCKS))
+        return false;


bodies same line or brackets

PastaPastaPasta · 2018-09-25T01:10:34Z

 CAmount CTxLockRequest::GetMinFee() const
 {
+    if(IsSimple())
+        return CAmount();


bodies same line or brackets

codablock

utACK

UdjinM6 · 2018-09-26T15:33:05Z

Ooops... I missed @PastaPastaPasta comments somehow 🙈 Let's fix them in some future cleanup/refactoring PR.

gladcow · 2018-09-26T15:57:03Z

It's my fault, I've decided to fix it later and missed the merging. I'll add this fix in next PR.

PastaPastaPasta · 2018-09-26T16:50:06Z

No worries @UdjinM6 and @gladcow

UdjinM6 added this to the 12.4 milestone Jun 21, 2018

UdjinM6 requested changes Jun 22, 2018

View reviewed changes

gladcow changed the title ~~Automatic InstantSend locks for "simple" transactions~~ [WIP] Automatic InstantSend locks for "simple" transactions Jun 22, 2018

gladcow force-pushed the recixlock branch 3 times, most recently from 5019ab7 to 451242e Compare July 16, 2018 14:24

gladcow changed the title ~~[WIP] Automatic InstantSend locks for "simple" transactions~~ Automatic InstantSend locks for "simple" transactions Jul 16, 2018

gladcow force-pushed the recixlock branch from 451242e to 3a75b38 Compare July 16, 2018 14:30

UdjinM6 requested changes Jul 16, 2018

View reviewed changes

UdjinM6 reviewed Jul 17, 2018

View reviewed changes

gladcow changed the title ~~Automatic InstantSend locks for "simple" transactions~~ [WIP] Automatic InstantSend locks for "simple" transactions Jul 18, 2018

gladcow force-pushed the recixlock branch from 1ea841e to 0bc8f8d Compare August 6, 2018 12:08

gladcow changed the title ~~[WIP] Automatic InstantSend locks for "simple" transactions~~ Automatic InstantSend locks for "simple" transactions Aug 7, 2018

gladcow and others added 15 commits September 24, 2018 13:44

fix mempool threshod calculation

f38fa95

bump mocktime in activate_autoix_bip9

a4af6e5

set node times

fa095fd

no need to spam the node with gettransaction rpc requests that often

6ff0868

use spork active instead of leaking spork logic into tests

2858dc6

codestyle fixes

fca2b13

add test description in comments

a167a16

fix typo

57b4de3

sync test nodes more often during BIP9 activation

4df7644

Use 4th bit in BIP9 activation

b81fddc

Fix comments according codestyle guide

85764a1

Call AcceptLockRequest and Vote at the first node creating autoix lock

8f32d20

fix mempool used memory calculation

eff6197

rallback not necessary change in CWallet::CreateTransaction

e274bf4

test for stopping autolocks for full mempool

9732fb1

gladcow force-pushed the recixlock branch from aacd3e1 to 9732fb1 Compare September 24, 2018 13:07

Inject "simple autolockable" txes into txlockrequest logic

4842988

UdjinM6 approved these changes Sep 24, 2018

View reviewed changes

nmarley approved these changes Sep 25, 2018

View reviewed changes

PastaPastaPasta requested changes Sep 25, 2018

View reviewed changes

codablock approved these changes Sep 26, 2018

View reviewed changes

UdjinM6 merged commit 5454bea into dashpay:develop Sep 26, 2018

UdjinM6 mentioned this pull request Oct 4, 2018

Fix auto-IS and tests #2331

Merged

gladcow mentioned this pull request Oct 4, 2018

Trivial: Codestyle fixes in InstantSend code #2332

Merged

Conversation

gladcow commented Jun 21, 2018

Uh oh!

PastaPastaPasta commented Jun 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thephez commented Jun 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gladcow commented Jun 21, 2018

Uh oh!

PastaPastaPasta commented Jun 21, 2018

Uh oh!

thephez commented Jun 21, 2018

Uh oh!

UdjinM6 commented Jun 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Antti-Kaikkonen commented Jun 21, 2018

Uh oh!

thephez commented Jun 22, 2018

Uh oh!

Antti-Kaikkonen commented Jun 22, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gladcow commented Jun 22, 2018

Uh oh!

UdjinM6 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

krishnawhite commented Jun 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gladcow commented Jun 26, 2018

Uh oh!

PastaPastaPasta commented Jun 26, 2018

Uh oh!

UdjinM6 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

UdjinM6 Jul 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sidhujag commented Jul 31, 2018

Uh oh!

gladcow commented Jul 31, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gladcow commented Aug 7, 2018

Uh oh!

UdjinM6 commented Aug 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PastaPastaPasta commented Jun 21, 2018 •

edited

Loading

thephez commented Jun 21, 2018 •

edited

Loading

UdjinM6 commented Jun 21, 2018 •

edited

Loading

Antti-Kaikkonen commented Jun 22, 2018 •

edited

Loading

krishnawhite commented Jun 26, 2018 •

edited

Loading

UdjinM6 Jul 17, 2018 •

edited

Loading

gladcow commented Jul 31, 2018 •

edited

Loading

UdjinM6 commented Aug 9, 2018 •

edited

Loading