Metadata V15: Enrich extrinsic type info for decoding

[lexnv]

Extrinsic Metadata fields

Metadata is enriched to provide users with additional types for decoding extrinsics:

• Address type
• Call type
• Signature type
• Extra type

Although already available on the extrinsic's type generic parameters, having the fields
readily available provides enough information for users to decode extrinsics without
making assumptions about the metadata shape (e.g., this eliminates the need of subxt
to parse those fields manually here, feature also requested by the linked issue).

Note: the Extra type similar to others can be used to decode the extrinsics, as well as to
skip decoding over those fields (see subxt's decoding).

Incomplete UncheckedExtrisnic type

It may be possible for certain substrate-based chains, or even substrate tests, to construct a
runtime with different types, or missing types. For those edge cases, the type reported back in
the metadata is the type of the tuple meta_type::<()>().

Depends on frame-metadata release.
Closes #12929.

Subxt Testing

• Fetched the unstable metadata
• Type IDs match the extrinsic type params

# This is the extrinsic type with enriched info
"extrinsic": {
    "ty": 786,
    "version": 4,
    "address_ty": 161,
    "call_ty": 135,
    "signature_ty": 348,
    "extra_ty": 787,

# This is the extrinsic.ty which has the type params exactly the
# same as what is reported in the extrinsic for address, call, signature, extra
    "id": 786,
    "type": {
      "path": [
        "sp_runtime",
        "generic",
        "unchecked_extrinsic",
        "UncheckedExtrinsic"
      ],
      "params": [
        {
          "name": "Address",
          "type": 161
        },
        {
          "name": "Call",
          "type": 135
        },
        {
          "name": "Signature",
          "type": 348
        },
        {
          "name": "Extra",
          "type": 787
        }
      ],

// @paritytech/subxt-team @jsdw @bkchr

[kianenigma]

Looks overall good, just one suggestion about using a type alias.

[kianenigma]

I don't get why we have to break down SignaturePayload? can't this have TypeInfo itself?

And if it is being broken down and not used, I rather actually delete SignaturePayload if it is not being used anymore.

[kianenigma]

ahh, it is being done already 🙈

[kianenigma]

In this case, a type SignaturePayloadOf<E> = (<E as Extrinsic>::X, <E as Extrinsic>::Y, <E as Extrinsic>::Z) will help prevent potential error where the order of these 3 might be mistaken.

[jsdw]

Does extrinsic.ty have any use I wonder with this change? In Subxt at least, the only reason to ever inspect the type was to find out the type IDs of the Signature/Extra/etc bits, which are now handily provided by this change, so perhaps it can be removed from the metadata.

@ascjones might be able to provide some insight into the motivation for including that type in the first place (or maybe it's been around much longer than V14?)

[bkchr]

Instead of breaking most of the code we could just do

type SignaturePayload: SignaturePayload;

trait SignaturePayload {
      type Address: TypeInfo;
      type Signature: TypeInfo;
      type Extra: TypeInfo;
}

[lexnv]

Yep, I like this approach more! Thanks for the feedback!

[lexnv]

@bkchr What's your opinion on having the extrinsic.ty in the metadata after this change (the message James mentioned above)?

We should have enough type info by relying on:

• address_ty
• call_ty
• signature_ty
• extra_ty

The extrinsic.ty has a manual implementation of TypeInfo that adds the above ty as generic types, but since we explicitly expose that info with this PR, I believe we should remove the extrinsic.ty.

That is unless we'll break other things

[bkchr]

I have no idea if someone is using this. However, frankly speaking by only having the types you listed you don't know the encoding of the Extrinsic type. While currently it may also already not being correct as it probably doesn't include the Compact<u32> length in from of the extrinsinc encoding?

[lexnv]

Good idea! I'll add the TypeDefCompact<u32> type to the extrinsic metadata to provide even better decoding abilities to end users. Considering this, I'll remove the extrinsic_ty from the metadata, unless encountering some edge-cases from UX teams.

The type only contained the address, call, signature and extra on its type info definition, and users should be able to obtain those details (and more) directly.

[jsdw]

iirc the extrinsic type was quite useless anyway before; it mostly just said "hey we want to encode the extrinsic to some vec of bytes". It doesn't provide any useful information on how those bytes need to be formed etc.

There are already a bunch of things we must just assume already about encoding extrinsics, ie the order of the signature stuff, how the not-really-scale Option for the signature is encoded (ie it has a version number &&'d to it) etc. I've always assumed this format information to be encoded in the tx version (4 atm); if a transaction has this version, it must be encoded in this way.

The metadata, in my mind, only needs to provide the information that might vary from chain to chain (eg the actual additional params etc). This tx version can encode the actual format that this stuff is turned into an extrinsic.

All of this to say, what would adding a TypeDefCompact<u32> to the metadata help with? I'd assume it to be part of the V4 extrinsic format I guess, but perhaps I'm wrong on this :)

[bkchr]

There are already a bunch of things we must just assume already about encoding extrinsics, ie the order of the signature stuff, how the not-really-scale Option for the signature is encoded (ie it has a version number &&'d to it) etc. I've always assumed this format information to be encoded in the tx version (4 atm); if a transaction has this version, it must be encoded in this way.

Good point.

The metadata, in my mind, only needs to provide the information that might vary from chain to chain (eg the actual additional params etc). This tx version can encode the actual format that this stuff is turned into an extrinsic.

Just because every body is currently using our UncheckedExtrinsic doesn't mean that this will always be the case. However, it makes sense that this is a type that we can not depict in as type metadata.

[lexnv]

bot rebase

[paritytech-processbot]

Rebased

[jsdw]

Looks great; well done!

Remember to release frame-metadata etc before merging :)

[lexnv]

Polkadot companion is expected to fail since the latest companion conflicts, but will be solved by paritytech/polkadot#7401 (comment)

[lexnv]

bot merge