M-of-N-like sporks by gladcow · Pull Request #2288 · dashpay/dash

gladcow · 2018-09-12T14:31:34Z

See issue #2221

allow multiple hardcoded spork addresses (N) to be specified in chainparams/cmd-line/config (same signers for all sporks);
allow multiple messages with the same id/value signed by different spork addresses;
(de)activate any spork only when at least M similar message with the same id/value signed by different signers were received (the same M for all sporks);

"M-of-N" spork implementation requires spork signers consensus achieved with some external communication and doesn't offer any facilities for such consensus achievement. If this consensus is not achived, the spork is treated as it has default value. For example, if sporks are configured as "at least 2 signers of 5 possible" and some spork has default value 0, and 2 signers broadcast this spork value 1, while 2 other signers broadcast value 2 (and nTimeSigned field has the same value, see details later), this spork is treated as it has value 0.

Also, several possible signers require new logic for nTimeSigned field processing. In old implementation this field is used to separate old non-actual spork messages from the new spork messages (spork message with bigger nTimeSigned overwrites message with smaller one). We need similar logic in new implementation. For example, we have the same configuration like in previous example, 2-of-5-signers. At moment t1 signers 1, 2, 3 broadcast spork with value 1 and spork switches to this value. After that at moment t2 signers 4 and 5 tries to switch this spork to value 2, and fail to do this, because votes of first 3 signers don't allow any node to calculate current spork value, and spork goes to default value. We can't also simply overwrite all "old" spork messages of other signers because of the fact that in this case just one signer can "rallback" several other signer's messages.

To resolve this issue we can require all signers to use the same value of the nTimeSigned field to treat their messages as a part of the same "spork broadcast" and allow such signer's groups to overwrite messages of other signer's groups with less nTimeSigned field value if this group size is bigger or equal to the minimum configured value (M in M-of-N). It is difficult to use current time for this goal as it was before, because signer's clocks are not synchronized and signers can broadcast their messages at slightly different moments of time. I think it is better to use increasing counter instead and allow signers to set its value directly. Also it is better to allow spork signers to use some default way to calculate this counter value for their convenience and I have added such calculation, but this calculation is not a facility for achieving of signers consensus, so it doesn't resolve all possible situations and sometimes signers should set counter value explicitly. Also, I've renamed nTimeSigned field to nBroadcastID to reflect its new meaning.

nmarley · 2018-09-13T02:09:44Z

Thanks for this implementation @gladcow!

I'm not sure it makes sense to try and coordinate on nTimeSigned or rename to use a broadcast field -- or rather, I think that adds extra, un-needed complexity. The nTimeSigned should be when the signer signed the spork, and we track spork ID/Values per-spork-signer instead. The latest nTimeSigned value over-writes the value for that spork ID for that given signer only.

At moment t1 signers 1, 2, 3 broadcast spork with value 1 and spork switches to this value. After that at moment t2 signers 4 and 5 tries to switch this spork to value 2, and fail to do this, because votes of first 3 signers don't allow any node to calculate current spork value, and spork goes to default value.

I don't understand this. What do you mean by "because votes of first 3 signers don't allow any node to calculate current spork value" ? Can you explain this in another way? I don't think the intention is to require signing and broadcasting at a specific time or timestamp, but rather, sporks are updated per-signer and when the threshold number is reached where ID / Value match, then the active spork switches to that value.

gladcow · 2018-09-13T08:22:17Z

@nmarley ,

I don't understand this. What do you mean by "because votes of first 3 signers don't allow any node to calculate current spork value" ? Can you explain this in another way?

Let's consider what a some usual node has in this situation. It has spork messages from signers 1-3 with spork value 1 (and "old" times in nTimeSigned) and messages from signers 4 and 5 with spork value 2 (and "new" times in nTimeSigned). How could this node understand that current spork value is 2, not 1? The only way is to compare nTimeSigned field of all this messages.

UdjinM6 · 2018-09-13T12:02:14Z

I think I see what's going on here with switching from time to sequence id - it's basically an attempt to make sure everyone is signing the same message I guess. But I don't see why it would matter if signatures are not combined (in a crypto way like e.g. in BLS). To distinguish messages signed by different spork key holders I'd rather switch from strSporkAddress to mapSporkAddresses <int, string> e.g. [(0, "Xsomeaddr1"), (1, "Xsomeaddr2), ..]. You could then make -sporkaddr accept format like signerid:signeraddr e.g. 0:Xsomeaddr1 and instead of encoding every addr in one string in cmd line you could submit multiple -sporkaddr just like we do for -connect or -addnode. This would also require extending spork message with smth like nSignierID though but I think it should be feasible to implement upgrade logic here.

nmarley · 2018-09-13T16:12:26Z

How could this node understand that current spork value is 2, not 1? The only way is to compare nTimeSigned field of all this messages.

nTimeSigned should be different for all signers. The node uses the spork with the same ID / value for the threshold signers. If the threshold is below majority (e.g. 2 of 5), then it should honor majority signers, e.g. if 2 spork signers send 10004 / 1000, then the other 3 send 10004 / 3000, then the majority signed value should take the "active" slot.

This would also require extending spork message with smth like nSignierID though but I think it should be feasible to implement upgrade logic here.

I initially was trying to do something like this, but I don't think it's necessary (is an arbitrary unique ID field that doesn't have to be honored by all nodes and is superfluous). We can just use the sporkAddr as the unique ID instead. The spork signer is identified by reading the signature and recovering the pubkey / hash160, something like:

https://github.com/nmarley/dash/blob/a5b2df7a86da7c7a789b607be5a194ec9ae4abb2/src/spork.cpp#L431

UdjinM6 · 2018-09-13T16:23:05Z

@nmarley Ah, good point re pubkey recovery!

gladcow · 2018-09-13T17:24:46Z

nTimeSigned should be different for all signers. The node uses the spork with the same ID / value for the threshold signers. If the threshold is below majority (e.g. 2 of 5), then it should honor majority signers, e.g. if 2 spork signers send 10004 / 1000, then the other 3 send 10004 / 3000, then the majority signed value should take the "active" slot.

Looks like I can't explain it clearly.) This 3 signers and other 2 signers participate in different votings really, for example first 3 signers vote in 2017 year and last 2 signers vote in 2018. And spork should be 1000 in 2017 and should become 3000 in 2018. But usual node in 2018 has 5 messages in its memory: 3 messages from 2017 and 2 from 2018, that's why switching to 3000 in 2018 fails.

It is not attempt to simply defend the written code, it is a real problem I've encountered during testing.)
There are 2 ways to resolve this issue: special processing for nTimeSigned field and using of crypto m-of-n threshold signature like @codablock has described in #2221. In the second way the spork sending process for the signers is different from the original process, so I've decided to use the first way.

nmarley · 2018-09-13T19:54:35Z

Looks like I can't explain it clearly.) This 3 signers and other 2 signers participate in different votings really, for example first 3 signers vote in 2017 year and last 2 signers vote in 2018. And spork should be 1000 in 2017 and should become 3000 in 2018. But usual node in 2018 has 5 messages in its memory: 3 messages from 2017 and 2 from 2018, that's why switching to 3000 in 2018 fails.

Ah, ok, thanks for the explanation. I think I understand now. In this case, just make the threshold value require a majority, e.g. if 5 total spork keys exist, require at least 3 of them. That should solve the problem, right?

gladcow · 2018-09-14T08:48:25Z

@nmarley , right, it solves the problem, but I've thought that 1-of-N (or 2-of-N at least) case is important for this functionality (we can have emergency need to switch some spork value, so any additional communications among signers can be too slow). Am I wrong? What is the most possible configuration for many signers we plan to use?

UdjinM6 · 2018-09-14T12:20:02Z

I'd say let's start with implementing the simplest/smallest possible one aka 2of3 first and see how it goes.

gladcow · 2018-09-14T13:11:17Z

So, to make it clear, what should be changed:

Remove processing of small Ms, M always is greater than N/2.
Remove new processing of the nTimeSigned field, calc current spork value with majority of signers.
Get KeyID from message signature like in @nmarley implementation, not by comparing signature with all signer addresses like in current implementation (I'm using address as signer unique id already, only calculation of the KeyID is different).
Change -setsporkaddress syntax to don't set all addresses in one string.

Have I missed something?

UdjinM6 · 2018-09-14T15:10:56Z

@gladcow sounds good (assuming (4) means ", support providing multiple -sporkaddrs instead")

gladcow · 2018-09-17T12:37:44Z

Rebased and fixed, re-review, please.

PastaPastaPasta

Sorry about all the nits! See inline

PastaPastaPasta · 2018-09-17T18:07:36Z

if( -> if (

PastaPastaPasta · 2018-09-17T18:08:02Z

for( -> for (

PastaPastaPasta · 2018-09-17T18:10:09Z

bodies of an if should either be on the same line or in brackets

If an if only has a single-statement then-clause, it can appear on the same line as the if, without braces. In every other case, braces are required, and the then and else clauses must appear correctly indented on a new line.

PastaPastaPasta · 2018-09-17T18:11:16Z

same as above, same line or braces. Also if( -> if (

PastaPastaPasta · 2018-09-17T18:11:38Z

above, same line or braces

PastaPastaPasta · 2018-09-17T18:11:55Z

if( -> if (

PastaPastaPasta · 2018-09-17T18:12:23Z

if( -> if (

PastaPastaPasta · 2018-09-17T18:13:33Z

if( -> if (

)){ -> )) {

PastaPastaPasta · 2018-09-17T18:14:09Z

if( -> if (

)){ -> )) {

PastaPastaPasta · 2018-09-17T18:14:47Z

I'd prefer a better name ¯\(ツ)/¯

@PastaPastaPasta , I'm writing wrong ifs automatically, I think it is bad practice to break codestyle guide, so it is good you are pointing this errors to me, I'm sorry that I'm making them so often.)

UdjinM6

see inline comments

UdjinM6 · 2018-09-17T20:55:49Z

This should be updated to match actual test logic.

UdjinM6 · 2018-09-17T20:56:06Z

UdjinM6 · 2018-09-17T22:13:38Z

No need to set r here, could just return false instead.

UdjinM6 · 2018-09-17T22:19:39Z

Naming: nSporkID and nActiveValueRet (same in cpp)

UdjinM6 · 2018-09-17T22:31:17Z

Given the conditions in SetMinSporkKeys there can be only one item with value_data.second >= nMinSporkKeys i.e. there is no need for max_count and if, you can simply set activeValue and return true; here. Or even better, you could check this in previous loop (line 40) and drop this one completely.

UdjinM6 · 2018-09-17T22:31:48Z

naming: mapValueCounts

UdjinM6 · 2018-09-17T22:35:27Z

naming: keyIDSigner

UdjinM6 · 2018-09-17T22:36:04Z

nActiveValue

UdjinM6 · 2018-09-17T22:36:38Z

keyIDSigner

UdjinM6 · 2018-09-18T09:19:32Z

This changes the hash. Not sure if this is going to work (without being banned), will test.

Yep, I'm being banned on testnet. Should probably change GetSignatureHash() to hash everything but vchSig and use it instead of GetHash() in GetSignerKeyID() (for activated spork6 only).

UdjinM6 · 2018-09-18T10:21:16Z

Yep, I'm being banned on testnet. Should probably change GetSignatureHash() to hash everything but vchSig and use it instead of GetHash() in GetSignerKeyID() (for activated spork6 only).

UdjinM6 · 2018-09-18T10:23:34Z

Should use a (modified) GetSignatureHash() here.

Hash should include signature for this functionality because of the fact that signature is the only thing that is different in messages from different signers. If we don't include the signature the messages from other signers are not broadcasted after the message from the first signer. So it looks like I should add some update code compatible with the old version.(

I know :) But this is only true for p2p part, not for the sign/verify part which can use whatever hash it needs, that's basically the purpose of GetSignatureHash().

Ah, I see now, will try to fix it this way.)

gladcow · 2018-09-26T13:06:36Z

Rebased and fixed, re-review, please

gladcow · 2018-09-26T13:12:27Z

Contains changes from #2313 (I missed them), but also stops to serialize pubkey ids, pubkeys should be hardcoded or read from options, not from spork cache ( = from previous dashd run)

thephez · 2018-09-26T13:30:14Z

I initially was trying to do something like this, but I don't think it's necessary (is an arbitrary unique ID field that doesn't have to be honored by all nodes and is superfluous). We can just use the sporkAddr as the unique ID instead. The spork signer is identified by reading the signature and recovering the pubkey / hash160, something like:

https://github.com/nmarley/dash/blob/a5b2df7a86da7c7a789b607be5a194ec9ae4abb2/src/spork.cpp#L431

@nmarley A little late and probably not relevant at this point in time, but my understanding is that this type of pubkey recovery cannot be done with BLS. Perhaps something to think about in the future.

UdjinM6 · 2018-09-26T14:18:42Z

Pls rebase again and bump CSporkManager::SERIALIZATION_VERSION_STRING

…e synced at moment

gladcow · 2018-09-26T16:32:53Z

Rebased, serialization version is bumped.

UdjinM6

I have a couple of (small) change requests, mostly about readability, see inline commits. Otherwise looks good and seems to be working 👍 Will test a bit more.

UdjinM6 · 2018-09-26T19:44:25Z

-            if (!itActive->second.CheckSignature(sporkPubKeyID, true)) {
-                mapSporksByHash.erase(itActive->second.GetHash());
-                mapSporksActive.erase(itActive++);
+        auto signer_msg_pair = itActive->second.begin();


The name is confusing here imo, would prefer smth like itSignerPair or smth like that.

UdjinM6 · 2018-09-26T19:45:07Z

+                    continue;
+                }
+            }
+            signer_msg_pair++;


Should use pre-increment here i.e. ++signer_msg_pair; (or rather ++itSignerPair; considering the previous comment)

UdjinM6 · 2018-09-26T19:51:22Z

        LOCK(cs); // make sure to not lock this together with cs_main
        for (const auto& pair : mapSporksActive) {
-            connman.PushMessage(pfrom, CNetMsgMaker(pfrom->GetSendVersion()).Make(NetMsgType::SPORK, pair.second));
+            for (const auto& signer_spork: pair.second) {


Would prefer a bit more self-describing name e.g. signerSporkPair

UdjinM6 · 2018-09-26T19:55:36Z

+{
+    int maxKeysNumber = setSporkPubKeyIDs.size();
+    if ((minSporkKeys <= maxKeysNumber / 2) || (minSporkKeys > maxKeysNumber)) {
+        LogPrintf("CSporkManager::SetSporkAddress -- Invalid min spork signers number: %d\n", minSporkKeys);


s/SetSporkAddress /SetMinSporkKeys/

UdjinM6 · 2018-09-26T19:59:10Z

        }
+
+        CKeyID keyIDSigner;
+        if (!(spork.GetSignerKeyID(keyIDSigner, IsSporkActive(SPORK_6_NEW_SIGS))


Would prefer this rewritten via ||

UdjinM6 · 2018-09-26T20:00:47Z


    if(spork.Sign(sporkPrivKey, IsSporkActive(SPORK_6_NEW_SIGS))) {
+        CKeyID keyIDSigner;
+        if (!(spork.GetSignerKeyID(keyIDSigner, IsSporkActive(SPORK_6_NEW_SIGS) && setSporkPubKeyIDs.count(keyIDSigner)))) {


Would prefer this rewritten via ||

gladcow · 2018-09-27T15:36:31Z

Fixed the issues

UdjinM6

Slightly tested ACK

nmarley

utACK, 👍 Good work @gladcow !

Seems like an easy transition from legacy to multiple keys, by:

Adding multiple keys by default in 12.4 release (with legacy key being 1 of them) and only requiring 1-of-1
Coordinating and signing sporks with other (non-legacy) keys during that period
The next release after that can remove or replace the default spork key (since sporks now signed by multiple keys), and require majority.

Is this what you had in mind for rollout @UdjinM6 ?

UdjinM6 · 2018-09-29T21:34:47Z

Not sure I understand it right @nmarley. I was thinking of using another spork to activate new keys smth like

// add new keys to chainparams, bump min required
if (IsSporkActive(SPORK_X_MULTIKEYSPORK) {
    // use chainparams
} else {
    // ignore chainparams, use the first (i.e. legacy) key only and minkey == 1
}

So first activate SPORK_X_MULTIKEYSPORK with the legacy key and then re-sign needed sporks with multiple keys.

nmarley · 2018-09-29T23:40:13Z

Wouldn't that be more complicated? Seems to me like the method I outlined would be easier, and it doesn't require another spork (which would only be used for this purpose). Are there any downsides to that method?

UdjinM6 · 2018-09-30T11:01:20Z

@nmarley I don't think the logic for spork activation here is compatible with multiple keys + 1-of-1 case. But we could probably "downgrade" the activation code with some hardcoded 1-of-1 backwards compatible logic and a legacy spork address i.e. smth like "if signed by the legacy spork address, assume that's the right value and ignore everything else for now". Is that what you are proposing?

nmarley · 2018-09-30T15:44:38Z

@UdjinM6 ~~Yes~~ Kinda. The more I think of it, the more I don't like trying to shoehorn the legacy spork key into the multiple spork signers logic. But I was thinking something like you suggested, namely:

Look at multi signers logic first. Otherwise:
If legacy spork signed it, then consider it valid.

Something like: nmarley/dash@pr2288-rebased...spork-legacy-logic

Should be separate PR if we go that route. I think this particular PR should be good though as-is, since it introduces the main multi signers logic and there could be alternative ways to activate, as we've discussed a couple already.

nmarley · 2018-09-30T15:48:11Z

edit: Actually, not exactly this:

if signed by the legacy spork address, assume that's the right value and ignore everything else for now

More like, prefer multi signer logic first, and then fall back to legacy spork key.

UdjinM6 · 2018-09-30T17:00:49Z

Need to think more about it but I agree that new keys and migration logic is smth for another PR. Let's merge this as is first :)

gladcow changed the title ~~M-of-N-like sporks~~ [WIP] M-of-N-like sporks Sep 12, 2018

gladcow changed the title ~~[WIP] M-of-N-like sporks~~ M-of-N-like sporks Sep 13, 2018

gladcow changed the title ~~M-of-N-like sporks~~ [WIP] M-of-N-like sporks Sep 14, 2018

gladcow force-pushed the multispork branch from c2ee4cd to bfeae79 Compare September 17, 2018 12:36

gladcow changed the title ~~[WIP] M-of-N-like sporks~~ M-of-N-like sporks Sep 17, 2018

PastaPastaPasta requested changes Sep 17, 2018

View reviewed changes

UdjinM6 requested changes Sep 18, 2018

View reviewed changes

UdjinM6 reviewed Sep 18, 2018

View reviewed changes

gladcow force-pushed the multispork branch from c601917 to 6a671c1 Compare September 26, 2018 13:05

gladcow added 5 commits September 26, 2018 17:37

support set of keys to sign spork

bf41b31

several addresses support in -sporkkey

d449c00

tests for multykey sporks

47249e5

command line option -minsporkkeys

6fc3998

make spork active only after given number of signers

656fad2

gladcow added 11 commits September 26, 2018 17:37

extract keyid from signed spork message directly

d51ceae

change -sporkaddr option syntax to process several addresses

9ad4485

codestyle fixes

4adf334

fix test comments

a094966

codestyle fixes

ea19a73

simplify CSporkManager::SporkValueIsActive

985ca1d

Calc signature hash without signature field

5c0c2ce

do not restore pubkey ids from cach

58da4c5

Calc different keyids to check signature because not all sporks can b…

8aa814a

…e synced at moment

codestyle fixes

167bf9f

Fix CSporkManager::CheckAndRemove to use several keys

d184ece

gladcow force-pushed the multispork branch from 6a671c1 to d184ece Compare September 26, 2018 16:27

UdjinM6 added this to the 12.4 milestone Sep 26, 2018

UdjinM6 reviewed Sep 26, 2018

View reviewed changes

gladcow added 2 commits September 27, 2018 17:29

codestyle fixes

f60fbfb

Correct processing of not actual spork6 value with GetSignerKeyID

8dc93fd

UdjinM6 approved these changes Sep 28, 2018

View reviewed changes

UdjinM6 requested review from codablock and nmarley September 28, 2018 16:33

nmarley approved these changes Sep 29, 2018

View reviewed changes

UdjinM6 mentioned this pull request Sep 29, 2018

document spork system with doxygen comments #2301

Merged

UdjinM6 merged commit f7ab6c4 into dashpay:develop Sep 30, 2018

Conversation

gladcow commented Sep 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nmarley commented Sep 13, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gladcow commented Sep 13, 2018

Uh oh!

UdjinM6 commented Sep 13, 2018

Uh oh!

nmarley commented Sep 13, 2018

Uh oh!

UdjinM6 commented Sep 13, 2018

Uh oh!

gladcow commented Sep 13, 2018

Uh oh!

nmarley commented Sep 13, 2018

Uh oh!

gladcow commented Sep 14, 2018

Uh oh!

UdjinM6 commented Sep 14, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gladcow commented Sep 14, 2018

Uh oh!

UdjinM6 commented Sep 14, 2018

Uh oh!

gladcow commented Sep 17, 2018

Uh oh!

PastaPastaPasta left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PastaPastaPasta Sep 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

UdjinM6 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gladcow commented Sep 12, 2018 •

edited

Loading

nmarley commented Sep 13, 2018 •

edited

Loading

UdjinM6 commented Sep 14, 2018 •

edited

Loading

PastaPastaPasta Sep 17, 2018 •

edited

Loading

thephez commented Sep 26, 2018 •

edited

Loading