Skip to content

server/auth,multi: connect resp excludes revoked match requests#653

Merged
chappjc merged 5 commits into
decred:masterfrom
chappjc:pending-queue-clean
Sep 1, 2020
Merged

server/auth,multi: connect resp excludes revoked match requests#653
chappjc merged 5 commits into
decred:masterfrom
chappjc:pending-queue-clean

Conversation

@chappjc
Copy link
Copy Markdown
Member

@chappjc chappjc commented Aug 31, 2020

server/auth: 'connect' response excludes match-related requests for revoked matches

The 'connect' response includes a list of active matches. connect also triggers
sending pending requests and messages. This change excludes match-
related requests from being sent if the match in question is not included
in the list of active matches. Only pending revoke_match requests are
allowed in this case, purely as a courtesy to the client. Similarly, the connect
timeout for revoke_match requests is now 4 hours.

Test RequestWhenConnected/SendWhenConnected.

dex/msgjson: add (*Message).ExtractMatchID

client/core: missing matches are assumed revoked

Also improve logging on the most important client and server events. INF level for a trade (both sides):

Maker, sell 20 DCR

[INF] CORE[dcr]: Funding 2000000000 atom order with coins [30ae25a069befcab83baa5403011f6e938483f2194c543768d05d4c602a11a75:1] worth 2770311969
[INF] CORE: Standing order 915f1f4d did not match and is now booked.
[INF] CORE: Starting negotiation for match f55e612d42b1a37efbcadd0925e830c5dad0f628088e7e5fcb3e6cbc5ed0960a for order 915f1f4daea970686b73386d281fb9c6da2e4ce74ed8919b12412cffcafec1a3 with swap fee rate = 11, quantity = 2000000000
[INF] CORE: Broadcasted transaction with 1 swap contracts for order 915f1f4daea970686b73386d281fb9c6da2e4ce74ed8919b12412cffcafec1a3. Fee rate = 11. Receipts (dcr): [2cff5647a82a00367456770cb56041dde7eaadd6231ca83495770b5a3445a749:0]
[INF] CORE: Contract coin 2cff5647a82a00367456770cb56041dde7eaadd6231ca83495770b5a3445a749:0 (dcr), value = 2000000000, refundable at 2020-09-01 18:02:42 +0000 UTC (script = 6382012088c02034a78b5a8bb7de90a8a277908fe44024af48533218bf17892155f6e8725ff4bd8876a914a399bcd68ad4712a866f7cda1a6bbffc105bde636704c28c4e5fb17576a914436f588809e8739a3192caf710edc679ef6f23b76888ac)
...
[INF] CORE: Audited contract (btc: b20dc8f32a833f53c864e4175c2008b935692bec3025f4bda0982944d5eb9867:0) paying to mthNc66CoTwzay75UQBfST8azrXphsa4i4 for order 915f1f4daea970686b73386d281fb9c6da2e4ce74ed8919b12412cffcafec1a3...
...
[INF] CORE: Broadcasted redeem transaction spending 1 contracts for order 915f1f4daea970686b73386d281fb9c6da2e4ce74ed8919b12412cffcafec1a3, paying to d4f742143aaef48cf55bc0f437584a5b6918953add06b931d533df1f2651df6b:0 (btc)
[INF] CORE: Match f55e612d42b1a37efbcadd0925e830c5dad0f628088e7e5fcb3e6cbc5ed0960a complete: sell 2000000000 dcr
[INF] CORE: Retiring inactive order 915f1f4daea970686b73386d281fb9c6da2e4ce74ed8919b12412cffcafec1a3

Taker, buy 20 DCR

[INF] CORE[btc]: Funding 200000000 Satoshis order with coins [7606b5f128500f60f648adfa6d7f23d6aa84d34d6b3229036bb7e485c40e46e6:1] worth 700000000
[INF] CORE: Starting negotiation for match f55e612d42b1a37efbcadd0925e830c5dad0f628088e7e5fcb3e6cbc5ed0960a for order ac3fdd0a1eef8aaa989df8f754a8843c2fc0776e5379f4eebef5a4121d9b90b9 with swap fee rate = 2, quantity = 2000000000
[INF] CORE: Audited contract (dcr: 2cff5647a82a00367456770cb56041dde7eaadd6231ca83495770b5a3445a749:0) paying to SsjGK154NT8u8yCujSAHD6AeRDXUWz5vQqr for order ac3fdd0a1eef8aaa989df8f754a8843c2fc0776e5379f4eebef5a4121d9b90b9...
...
[INF] CORE: Broadcasted transaction with 1 swap contracts for order ac3fdd0a1eef8aaa989df8f754a8843c2fc0776e5379f4eebef5a4121d9b90b9. Fee rate = 2. Receipts (btc): [b20dc8f32a833f53c864e4175c2008b935692bec3025f4bda0982944d5eb9867:0]
[INF] CORE: Contract coin b20dc8f32a833f53c864e4175c2008b935692bec3025f4bda0982944d5eb9867:0 (btc), value = 200000000, refundable at 2020-09-01 06:02:42 +0000 UTC (script = 6382012088a82034a78b5a8bb7de90a8a277908fe44024af48533218bf17892155f6e8725ff4bd8876a914909216abbf35703740dc1ec35b63c57d68ce8b01670402e44d5fb17576a9140ff0fdc3f0eebee74763dd41bc8c18fa9c1f692e6888ac)
...
[INF] CORE: Notified of maker's redemption (btc: d4f742143aaef48cf55bc0f437584a5b6918953add06b931d533df1f2651df6b:0) and validated secret for order ac3fdd0a1eef8aaa989df8f754a8843c2fc0776e5379f4eebef5a4121d9b90b9...
[INF] CORE: Broadcasted redeem transaction spending 1 contracts for order ac3fdd0a1eef8aaa989df8f754a8843c2fc0776e5379f4eebef5a4121d9b90b9, paying to 36371fc8511e31f2a97bc86c28191e82bdb6ec3f9df8e35862a3f3cfb558fb28:0 (dcr)
[INF] CORE: Match f55e612d42b1a37efbcadd0925e830c5dad0f628088e7e5fcb3e6cbc5ed0960a complete: buy 2000000000 dcr
[INF] CORE: Retiring inactive order ac3fdd0a1eef8aaa989df8f754a8843c2fc0776e5379f4eebef5a4121d9b90b9

chappjc and others added 5 commits August 31, 2020 16:54
The connect response includes a list of active matches, and it triggers
sending pending requests and messages. This excludes match-related
requests from being sent if the match in question is not included in the
list of active matches. Only pending revoke_match requests are allowed
in this case, purely as a courtesy to the client. Similarly, the connect
timeout for revoke_match requests is nown 4 hours.

Test RequestWhenConnected/SendWhenConnected.
@chappjc chappjc changed the title Pending queue clean server/auth,multi: connect resp excludes revoked match requests Aug 31, 2020
Comment thread client/core/trade.go
if len(extras) > 0 {
details := fmt.Sprintf("%d matches reported by %s were not found for %s.", len(extras), host, t.token())
t.notify(newOrderNote("Match resolution error", details, db.ErrorLevel, corder))
// t.negotiate(extras) // missed match message request, but match not revoked (?!)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a thought. If the match isn't revoked yet, the pending match request should be coming, so this should not be necessary though.

Comment thread client/core/trade.go
Comment on lines +367 to +370
if t.matches[mid] != nil {
log.Warnf("Skipping match %v that is already negotiating.", mid)
continue
}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a way this could happen at present, but starting up a new matchTracker and overwriting an existing one would be bad news.

}()

s.log.Info("HandleConnect done.")
s.log.Trace("HandleConnect done.")
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, never should have been Info. However, with ws.UseLogger added to client/core/log.go, we can get a lot more information about what the frontend is doing when it spins on market load.

Comment thread server/auth/auth.go
return handler(client.acct.ID, msg)
msgErr := handler(client.acct.ID, msg)
if msgErr != nil {
log.Debugf("Handling of '%s' request for user %v failed: %v", route, client.acct.ID, msgErr)
Copy link
Copy Markdown
Member Author

@chappjc chappjc Aug 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auth route handlers like handleInit can respond to the client with an error, but we never knew what the errors were, making debugging from the server side difficult.

Comment thread server/auth/auth.go
Comment on lines +938 to +942
mid, err := pr.req.ExtractMatchID()
if err != nil {
log.Errorf("Failed to read matchid field from '%s' payload: %v", pr.req.Route, err)
// just send it
} else if mid != nil && pr.req.Route != msgjson.RevokeMatchRoute {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-nil error is an unmarshal error, but matchid is expected to be nil for payloads with no "matchid" (not an error).

Comment thread server/auth/auth.go
// back into the pending requests.
connectTimeout := DefaultConnectTimeout
if pr.req.Route == msgjson.RevokeMatchRoute {
connectTimeout = 4 * time.Hour // a little extra time for revoke_match to be courteous
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's little cost to queue thousands of these for hours, and it helps the client when they do reconnect, especially if their dexc never restarted, only reconnected after prolonged outage (laptop suspend or network off).

Copy link
Copy Markdown
Member

@buck54321 buck54321 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know about moving the one function, but otherwise g2g.

Comment thread dex/msgjson/types.go
Comment on lines +384 to +389
// ExtractMatchID attempts to extract a "matchid" value from a request-typed
// Message's Payload. The request must have one of the following routes:
// AuditRoute, RedemptionRoute, RedeemRoute, MatchRoute, or RevokeMatchRoute.
// A non-nil error is only returned if unmarshalling the payload fails.
func (msg *Message) ExtractMatchID() (Bytes, error) {
// Name the targeted request routes.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to only be used in one place in auth.go. Can this be an auth package function rather than a *Message method?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I actually started with this simply inline in auth.handleConnect and for some reason thought it better in msgjson. Not seeing that anymore though.. switching back.

Copy link
Copy Markdown
Member Author

@chappjc chappjc Sep 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking again, the reasoning for a msgjson package method:

  • the field "matchid" must match what's used in msgjson
  • the named routes must be current (and the exclusion of RevokeMatchRoute is the one auth-specific detail)

Changes to msgjson are likely to overlook an auth function that expects certain things such as the specific json tag and a list of relevant request routes with a relevant payload.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. I'm just policing the interface a little. In general, seeing Message methods that apply to only certain routes raises flags for me. But as long as there's a good justification, go for it.

Comment thread server/auth/auth.go
Comment on lines +398 to +400
sort.Slice(msgs, func(i, j int) bool {
return msgs[i].msg.ID < msgs[j].msg.ID
})
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're using a uint64 for these msg IDs, so at a billion messages a day, once the DEX has been running without interruption for 50,539,024 years, the IDs are going to start over at zero and this is going to break.

@chappjc chappjc merged commit 38443f3 into decred:master Sep 1, 2020
@chappjc chappjc deleted the pending-queue-clean branch September 1, 2020 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants