Skip to content

Subtype Parsing#2511

Merged
rustyrussell merged 16 commits into
ElementsProject:masterfrom
niftynei:dual-funded-subtype
Apr 10, 2019
Merged

Subtype Parsing#2511
rustyrussell merged 16 commits into
ElementsProject:masterfrom
niftynei:dual-funded-subtype

Conversation

@niftynei
Copy link
Copy Markdown
Collaborator

Adds parsing capabilities for 'subtypes' in csv structs.

@niftynei niftynei added the v1.1 label Mar 29, 2019
@niftynei niftynei requested a review from rustyrussell March 29, 2019 00:31
@niftynei niftynei requested a review from cdecker as a code owner March 29, 2019 00:31
Copy link
Copy Markdown
Contributor

@rustyrussell rustyrussell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally like it, not sure it technically needs to be based on the TLV branch though I can see it'd get messy to disentangle.

We should see if we can use it on various internal structures; all those places where we open-code fromwire_ and towire_. Eg. fromwire_route_info, fromwire_channel_config, fromwire_crypto_state, fromwire_basepoints, fromwire_added_htlc, etc. Some of them should be amenable I think?

Comment thread tools/generate-wire.py
messages.append(Message(parts[0], Enumtype("WIRE_" + parts[0].upper(), parts[1]), comments))
if parts[1] == '$': # this is a subtype
subtypes.append(Subtype(parts[0], comments))
else:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$? Can we just have a single entry be a subtype?

Note that there's a companion to these changes, which is that the spec itself needs to decide on a format, and lightning-rfc/tools/extract-formats.py needs to be taught to extract it into the form you want here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the RFC parser to output desired format here: lightning/bolts#597

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I moved my complaint to that PR :)

Otherwise, excellent. Be nice to try this out on some internal types, too: autogen FTW.

Comment thread wire/gen_peer_wire_csv Outdated
funding_compose,32,num_inputs,2
funding_compose,34,input_info,$num_inputs
funding_compose,34+$,num_outputs,2
funding_compose,36+$,output_info,$num_outputs
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe insist that subtypes are defined before use, and then you can just recognize the typename here, no need for $?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes the RFC's a bit harder to read, imo

Copy link
Copy Markdown
Collaborator Author

@niftynei niftynei Apr 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking about this more, we could swap out the $ with the type name in place, but to truly accurately represent the 'size' we need to know both the count field (if it exists) + the size of the subtype. using the $ was meant as a 'space holder for some amount of a subtype size calculation', but we could definitely replace it with num_outputs*input_info instead.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has been updated!

@niftynei
Copy link
Copy Markdown
Collaborator Author

niftynei commented Apr 2, 2019

We should see if we can use it on various internal structures; all those places where we open-code fromwire_ and towire_. Eg. fromwire_route_info, fromwire_channel_config, fromwire_crypto_state, fromwire_basepoints, fromwire_added_htlc, etc. Some of them should be amenable I think?

definitely! most of this piggy-backs on that infrastructure for building those already. i seem to remember one of them have some custom parsing logic, but otherwise that'd be a quick fix up.

@niftynei niftynei force-pushed the dual-funded-subtype branch from 20a57f0 to f40f398 Compare April 5, 2019 21:28
@niftynei niftynei force-pushed the dual-funded-subtype branch 4 times, most recently from cbd1aab to 718952c Compare April 9, 2019 22:32
niftynei added 16 commits April 9, 2019 15:49
make TLV messages their own subclass of Message. this makes
other clean ups easier
first pass at adding subtype structs
this probably could be consolidated, as it splits
out all the print_to/fromwire method stuff for the Subtype class
`m` might not be set on the optional set as well, so move this check
down so that we now encompass both codepaths
This is needed so that some csv's can expose their subtype parsing
functions in their header. This gets used in a later PR where
we start replacing manually created 'subtype' definitions with
generated ones.
the original version of the subtype generator emitted '$'
to designate that a field was a subtype; now it's got a different
format:

	funding_type,8,num_inputs,2
	funding_type,10,input_info,num_inputs*input_info

this patch updates our generator to understand this new format
subtype children should be allocated off of themselves. this was
failing to compile for embedded subtypes (subtype within a subtype).
subtypes don't use the fance type registration that other
'set structs' do, see devtools/printwire.c
including tlv's in the wire docs breaks the printwire because
there's a bad method name. this fixes that
now we print the subtypes out when you call printwire

note that we have to reverse the order the subtypes appear in
because
  1) they're static and,
  2) a few of them are nested
@niftynei niftynei force-pushed the dual-funded-subtype branch from 718952c to 4c27a05 Compare April 10, 2019 03:41
@rustyrussell
Copy link
Copy Markdown
Contributor

Ack 4c27a05

@rustyrussell rustyrussell merged commit f2ecf8e into ElementsProject:master Apr 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants