GenesisBuilder runtime API#14131
Conversation
GenesisConfigBuilder Runtime API: preliminary proposalGenesisBuilder Runtime API: preliminary proposal
|
This pull request has been mentioned on Polkadot Forum. There might be relevant details there: |
bkchr
left a comment
There was a problem hiding this comment.
For the naming I don't see any reason to always include genesis_config as this is already implied by GenesisBuilder.
Looking good so far I would say.
| fn default_genesis_config_as_json() -> sp_std::vec::Vec<u8>; | ||
|
|
||
| /// Deserialize the `GenesisConfig` from given json blob and put it into the storage. | ||
| fn build_genesis_config_from_json(json: sp_std::vec::Vec<u8>); |
There was a problem hiding this comment.
| fn build_genesis_config_from_json(json: sp_std::vec::Vec<u8>); | |
| fn build_from_json(json: sp_std::vec::Vec<u8>); |
| /// API to interact with GenesisConfig for the runtime | ||
| pub trait GenesisBuilder { | ||
| /// Instantiate default `GenesisConfig` and serializes it to json blob. | ||
| fn default_genesis_config_as_json() -> sp_std::vec::Vec<u8>; |
There was a problem hiding this comment.
| fn default_genesis_config_as_json() -> sp_std::vec::Vec<u8>; | |
| fn generate_as_json(name: sp_runtime::RuntimeString) -> sp_std::vec::Vec<u8>; |
Naming is still not really good, but yeah... We also need to supply some kind of name or whatever we want to call it to distinguish which genesis config we want to generate. Here we also have different kind of genesis configs: https://github.com/paritytech/substrate/tree/2c3b923423fac829b02842fbb9a0016b55c417df/bin/node/cli/src/chain_spec.rs
There was a problem hiding this comment.
Maybe I am missing something, but I don't see the need for extra parameter to construct default GenesisConfig. GenesisConfig of the runtime is just a struct containing all GenesisConfigs pallets, so all required parametrization is already self-contained in config (AFAIU it).
e.g. the generated GenesisConfig for kitchensink-runtime:
pub struct GenesisConfig {
pub system: SystemConfig,
pub babe: BabeConfig,
pub indices: IndicesConfig,
pub balances: BalancesConfig,
pub transaction_payment: TransactionPaymentConfig,
pub staking: StakingConfig,
pub session: SessionConfig,
pub democracy: DemocracyConfig,
pub council: CouncilConfig,
pub technical_committee: TechnicalCommitteeConfig,
pub elections: ElectionsConfig,
pub technical_membership: TechnicalMembershipConfig,
pub grandpa: GrandpaConfig,
pub treasury: TreasuryConfig,
pub sudo: SudoConfig,
pub im_online: ImOnlineConfig,
pub authority_discovery: AuthorityDiscoveryConfig,
pub society: SocietyConfig,
pub vesting: VestingConfig,
pub assets: AssetsConfig,
pub transaction_storage: TransactionStorageConfig,
pub alliance_motion: AllianceMotionConfig,
pub alliance: AllianceConfig,
pub nomination_pools: NominationPoolsConfig,
}
What I see in chain_spec.rs are different ChainSpecs: development_config, local_testnet_config.
Each of them has specific genesis: local_testnet_genesis, development_config_genesis. Those specific gensis are just differently parametrized GenesisConfig instances of kitchensink_runtime created in shared function testnet_genesis.
With new GenesisBuilder API approach, that custom GenesisConfig will be embedded in the form of pre-configured json files customized on top of the json for default GensisConfig.
If I missed something, would you please be more precise on where the extra parameter is actually required?
There was a problem hiding this comment.
With new
GenesisBuilderAPI approach, that customGenesisConfigwill be embedded in the form of pre-configured json files customized on top of thejsonfor defaultGensisConfig.
Could you give an example of this?
There was a problem hiding this comment.
Examples for test-runtime.
What I mean by default GenesisConfig:
substrate/test-utils/runtime/src/lib.rs
Line 1340 in e147d24
What I mean by GenesisConfig customised on top of default:
Since we won't have GenesisConfig on the native side of the node (at least this is my current understanding), we will not be able to hard-code the GenesisConfig in the form of rust variable. The alternative for this will be json string.
There was a problem hiding this comment.
substrate/bin/node/cli/src/chain_spec.rs
Lines 297 to 379 in 9d0200b
This is the stuff I'm "worried" about.
But I think we should be able to solve this with some tricks. However this would require to write there some json code, but I think this is doable.
I imagine something like this:
let config = json! {{
"session": {
"keys": initial_authorities.map().collect(),
},
"staking": {
"validator_count": validator_count,
},
}};
// It should be able to handle that we only pass parts of the config and the rest should be filled by the default values.
Runtime::build_from_json(config);
There was a problem hiding this comment.
BTW this json macro already exists: https://docs.rs/serde_json/latest/serde_json/macro.json.html
There was a problem hiding this comment.
Added in: 679a970
We would not need to put this functionality into the runtime, we can easily implement it outside the runtime, using functions already defined:
let json_str = Runtime::default_as_json();
let mut json: Value = serde_json::from_str(json_str).unwrap();
merge(&mut json, json!({
"session": {
"keys": initial_authorities.map().collect(),
},
"staking": {
"validator_count": validator_count,
},
});
Runtime::build_from_json(json.to_string());
However I do understand that build_with_patch is a nice convenient function.
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Davide Galassi <davxy@datawok.net>
|
I’ve just realized that we don’t need two functions: |
|
bot rebase |
|
Rebased |
bkchr
left a comment
There was a problem hiding this comment.
I think the current function naming is confusing.
If we think about what we need, then we need to get the default config as json. We need to be able to patch the default config to create configs for dev networks etc. Lastly, we need to be able to build the config. Maybe using here build is a too confusing term, not sure, but given the internals it makes sense.
With these three functions we should be able to implement all possibles functionality as it exists right now.
(docs then also require updating)
| /// | ||
| /// This function instantiates the default `GenesisConfig` struct for the runtime and serializes it into a JSON | ||
| /// blob. It returns a `Vec<u8>` containing the JSON representation of the default `GenesisConfig`. | ||
| fn get_default_as_json() -> sp_std::vec::Vec<u8>; |
There was a problem hiding this comment.
| fn get_default_as_json() -> sp_std::vec::Vec<u8>; | |
| fn create_default_config() -> sp_std::vec::Vec<u8>; |
| /// `GenesisConfig`. This is helpful for changing enum variant value. | ||
| /// | ||
| /// Please note that the patch may contain full `GenesisConfig`. | ||
| fn build_config(patch: sp_std::vec::Vec<u8>) -> Result; |
There was a problem hiding this comment.
| fn build_config(patch: sp_std::vec::Vec<u8>) -> Result; | |
| fn patch_default_config(patch: sp_std::vec::Vec<u8>) -> sp_std::result::Result<sp_std::vec::Vec<u8>, RuntimeString>; |
| /// It is recommended to log any errors encountered during the process. | ||
| /// | ||
| /// Please note that provided json blob must contain all `GenesisConfig` fields, no defaults will be used. | ||
| fn build_config_no_defaults(json: sp_std::vec::Vec<u8>) -> Result; |
There was a problem hiding this comment.
| fn build_config_no_defaults(json: sp_std::vec::Vec<u8>) -> Result; | |
| fn build_config(json: sp_std::vec::Vec<u8>) -> Result; |
Please note that we don’t need runtime call to merge jsons. Can be done on client side.
If you want stand alone patching then I propose to remove it from runtime and left only @bkchr wdyt? |
Very good point! :D
Sounds good. |
|
bot merge |
* GenesisConfigBuilder: preliminary API proposal * fmt * comment removed * build_default_config removed * Update client/genesis-builder/src/lib.rs * config -> gensis_config * GenesisConfigBuilder: helper added * moved to primitives * licesne changed to apache-2.0 * Cargo.toml: name/path to genesis-builder updated * helper removed * sp-sd version bumped * sp-std bump * naming + new function * fix * build_from_patch_json -> build_with_patch * fix * Cargo.lock updated * readme: license updated * Update primitives/genesis-builder/src/lib.rs Co-authored-by: Davide Galassi <davxy@datawok.net> * Update primitives/genesis-builder/src/lib.rs Co-authored-by: Davide Galassi <davxy@datawok.net> * Update primitives/genesis-builder/Cargo.toml Co-authored-by: Davide Galassi <davxy@datawok.net> * Cargo.lock updated * removed redundant function * GenesisConfigBuilder API: no_defaults function added * Cargo.lock updated * GenesisConfigBuilder API: patching fn removed * trigger CI job --------- Co-authored-by: parity-processbot <> Co-authored-by: Davide Galassi <davxy@datawok.net>
Proposal of new Runtime API:
GenesisBuilder.It allows to:
GenesisConfigand serialize it to a JSON blob,GenesisConfigfrom a given JSON blob, and put it into storage, (no defaults allowed)Step towards: paritytech/polkadot-sdk#25
Required by : #14310