Glutton pallet by Szegoo · Pull Request #12833 · paritytech/substrate

Szegoo · 2022-12-03T15:07:11Z

Glutton

Pallet

The purpose of the pallet is to test the limits of Kusama. The root origin can configure what percentage of the ref-time and proof size to consume from the leftover weight.

The on_idle hook of the pallet starts off by consuming the proof size and after that, if some ref-time is left over it runs the waste_ref_time function which hashes some arbitrary data using the Blake2 hashing algorithm.

The proof size consumption is achieved by reading from a storage map inside the pallet called TrashData. The data inside this storage map has to be initialized by calling the initialize_pallet extrinsic which is only callable by the root origin.

Glutton Parachains

The parachains that will be used for testing the limits of Kusama will be called Glutton Parachains.

It has the double meaning of one who over-consumes (the parachain), but also one who relishes in enduring something (i.e. "glutton for punishment"), and it is seeing how much validators can endure from parachains.

Closes: #12772

Polkadot address: 126X27SbhrV19mBFawys3ovkyBS87SGfYwtwa8J2FjHrtbmA

frame/pov-limit/src/lib.rs

frame/pov-limit/src/weights.rs

frame/pov-limit/src/benchmarking.rs

frame/pov-limit/src/lib.rs

ggwpez · 2023-02-20T11:48:52Z

bot bench $ pallet dev pallet_glutton

frame/glutton/src/lib.rs

koute · 2023-02-20T12:48:52Z

frame/glutton/src/lib.rs

+
+			let clobber = Self::waste_ref_time_iter(vec![0u8; 64], n);
+
+			// By casting it into a vec we can hopefully prevent the compiler from optimizing it


Usually you'd use something like std::ptr::read_volatile to prevent the compiler from optimizing a variable out. (These reads are guaranteed to not be elided or reordered by the compiler.)

Interesting. Didn't know about this feature in rust. IIUC this is an unsafe function but since we are not using std::ptr::write_volatile this is kind of safe. But I still think we are better off here using our current approach. @ggwpez Thoughts?

Yea I would rather not put unsafe into the runtime code - even if that is the more correct way.
I think it should be fine in this case, or?

Yes, the code should be also okay the way it is. unsafe isn't that bad and we also use unsafe in the runtime (but just hidden :P).

The way to go would probably be to just make an utility function somewhere that would wrap the unsafe call and make 100% sure it's always safe. There's at least one more place that I know of where this would be useful, if not more. :P

The thing with unsafe isn't that it should never be used, it's just that its use should be limited (it shouldn't be sprinkled all over the code), neatly encapsulated in a safe interface, and very carefully written and reviewed (ideally by experts).

But yeah, if it works then the way it is now is fine for now. It was more of a drive-by comment rather than a "hey, change this".

frame/glutton/src/lib.rs

frame/glutton/README.md

Co-authored-by: Koute <koute@users.noreply.github.com>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

ggwpez · 2023-02-20T19:26:15Z

bot rebase

ggwpez · 2023-02-20T19:43:28Z

bot bench $ pallet dev pallet_glutton

ggwpez · 2023-02-20T19:43:45Z

bot clean

command-bot · 2023-02-20T20:15:46Z

@ggwpez Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" pallet dev pallet_glutton has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2418567 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2418567/artifacts/download.

ggwpez · 2023-02-21T17:04:05Z

Any last words? Otherwise going to merge in a few hours.

ggwpez · 2023-02-21T22:39:18Z

bot merge

* Pov-Limit pallet * use Perbill & fixes * fixes * reads & writes * update docs * tests * calculate weight * fmt * benchmark * logic fix * naming fix * caclulate computation weight limit * make the Hasher generic * make the code compile * generate weight * fix on_idle * fix * fix * make reading generic * fix? * fixes * remove warning * fix * hasher fix :D * change value * test * actual weight and expected weight are the same * update * fix * add events * remove useless line * using actual hashing algorithm * better readability * fix nits * Update Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * add migration * hardcode proof_size in weights.rs * format * fixes * Fix weight * docs * fix * Update frame/pov-limit/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/pov-limit/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * empty on_ilde & update weight.rs * remove migration & fix benchmark * remove migration from migrations * initialize_pallet * use blake2 * Update frame/pov-limit/Cargo.toml Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/pov-limit/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/pov-limit/Cargo.toml Co-authored-by: Bastian Köcher <git@kchr.de> * rename pallet * make the wasters non-generic * rename to glutton * fix * small fixes & run benchmark * increase left over ref-time in tests * Update frame/glutton/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * or values * fix * generate weight * Update frame/glutton/Cargo.toml Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix nits * Adjustments Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix * Update weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Tweak constants Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * remove genesis config * passing tests * More precise results Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * update node-executor test * Calculate number of iterations in advance Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * increase criteria * Final fixes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/glutton/src/tests.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/glutton/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/glutton/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * fix typos * Update frame/glutton/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/glutton/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * expand/shrink * fmt * Revert "Update frame/glutton/src/lib.rs" This reverts commit 98a237a. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use CountedStorageMap Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add benchmark Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use manual map counter Something with the R/W count in the benchmarking result did not add up. Need to investigate but for now just using a manual counter. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use new template Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Doc+typos Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton * Apply suggestions from code review Co-authored-by: Koute <koute@users.noreply.github.com> * Add minimal README Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: command-bot <> Co-authored-by: Koute <koute@users.noreply.github.com>

Polkadot-Forum · 2023-03-09T19:30:21Z

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-release-analysis-v0-9-39/2277/1

* Pov-Limit pallet * use Perbill & fixes * fixes * reads & writes * update docs * tests * calculate weight * fmt * benchmark * logic fix * naming fix * caclulate computation weight limit * make the Hasher generic * make the code compile * generate weight * fix on_idle * fix * fix * make reading generic * fix? * fixes * remove warning * fix * hasher fix :D * change value * test * actual weight and expected weight are the same * update * fix * add events * remove useless line * using actual hashing algorithm * better readability * fix nits * Update Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * add migration * hardcode proof_size in weights.rs * format * fixes * Fix weight * docs * fix * Update frame/pov-limit/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/pov-limit/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * empty on_ilde & update weight.rs * remove migration & fix benchmark * remove migration from migrations * initialize_pallet * use blake2 * Update frame/pov-limit/Cargo.toml Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/pov-limit/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/pov-limit/Cargo.toml Co-authored-by: Bastian Köcher <git@kchr.de> * rename pallet * make the wasters non-generic * rename to glutton * fix * small fixes & run benchmark * increase left over ref-time in tests * Update frame/glutton/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * or values * fix * generate weight * Update frame/glutton/Cargo.toml Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix nits * Adjustments Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix * Update weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Tweak constants Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * remove genesis config * passing tests * More precise results Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * update node-executor test * Calculate number of iterations in advance Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * increase criteria * Final fixes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/glutton/src/tests.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/glutton/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/glutton/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * fix typos * Update frame/glutton/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/glutton/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * expand/shrink * fmt * Revert "Update frame/glutton/src/lib.rs" This reverts commit 98a237a. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use CountedStorageMap Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add benchmark Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use manual map counter Something with the R/W count in the benchmarking result did not add up. Need to investigate but for now just using a manual counter. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use new template Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Doc+typos Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton * Apply suggestions from code review Co-authored-by: Koute <koute@users.noreply.github.com> * Add minimal README Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: command-bot <> Co-authored-by: Koute <koute@users.noreply.github.com>

Pov-Limit pallet

9de0eee

joepetrowski reviewed Dec 3, 2022

View reviewed changes

frame/pov-limit/src/lib.rs Outdated Show resolved Hide resolved

ggwpez suggested changes Dec 4, 2022

View reviewed changes

frame/pov-limit/src/lib.rs Outdated Show resolved Hide resolved

Szegoo added 5 commits December 6, 2022 17:21

use Perbill & fixes

7c0b552

fixes

1a50132

reads & writes

0e79cd3

update docs

24ac38a

tests

fa70f0f

Szegoo commented Dec 6, 2022

View reviewed changes

frame/pov-limit/src/lib.rs Outdated Show resolved Hide resolved

Szegoo marked this pull request as ready for review December 6, 2022 21:20

bkchr reviewed Dec 6, 2022

View reviewed changes

frame/pov-limit/src/lib.rs Outdated Show resolved Hide resolved

frame/pov-limit/src/lib.rs Outdated Show resolved Hide resolved

Szegoo and others added 2 commits December 7, 2022 17:48

calculate weight

c1ecfe4

Merge branch 'master' into pov-limit

b99945d

Szegoo requested a review from bkchr December 8, 2022 07:45

Szegoo added 4 commits December 8, 2022 08:49

fmt

25153a1

benchmark

27868bd

logic fix

54f3117

naming fix

91178fb

Szegoo commented Dec 8, 2022

View reviewed changes

frame/pov-limit/src/weights.rs Outdated Show resolved Hide resolved

ggwpez reviewed Dec 12, 2022

View reviewed changes

frame/pov-limit/src/benchmarking.rs Outdated Show resolved Hide resolved

ggwpez reviewed Dec 12, 2022

View reviewed changes

frame/pov-limit/src/lib.rs Outdated Show resolved Hide resolved

caclulate computation weight limit

3565ff1

Szegoo marked this pull request as draft December 13, 2022 08:59

Szegoo and others added 7 commits December 13, 2022 10:06

Merge branch 'paritytech:master' into pov-limit

24edc74

make the Hasher generic

9f48aba

make the code compile

f2abdfa

generate weight

36b049c

fix on_idle

79c7816

fix

b3f72d5

Merge branch 'paritytech:master' into pov-limit

810bcd6

".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton

af6b3e4

kianenigma approved these changes Feb 20, 2023

View reviewed changes

koute reviewed Feb 20, 2023

View reviewed changes

ggwpez and others added 2 commits February 20, 2023 15:49

Apply suggestions from code review

2c8b112

Co-authored-by: Koute <koute@users.noreply.github.com>

Add minimal README

0db5a70

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Merge remote-tracking branch 'origin/master' into pov-limit

62f2d8d

command-bot bot deleted a comment from substrate-tip-bot bot Feb 20, 2023

command-bot bot deleted a comment from paritytech-processbot bot Feb 20, 2023

".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton

f27c73b

paritytech-processbot bot merged commit ea18266 into paritytech:master Feb 21, 2023

Szegoo mentioned this pull request Mar 9, 2023

Glutton Parachain paritytech/cumulus#2294

Merged

kiltbot mentioned this pull request Apr 3, 2023

[AUTOMATIC] Update Polkadot dependencies from 0.9.38 to 0.9.39 KILTprotocol/kilt-node#495

Closed

librelois mentioned this pull request Apr 6, 2023

Update substrate/polkadot/cumulus from v0.9.38 to v0.9.40 moonbeam-foundation/moonbeam#2200

Closed

kacperzuk-neti mentioned this pull request Jun 23, 2023

Polkadot v0.9.43 liberland/liberland_substrate#295

Merged

15 tasks

joepetrowski mentioned this pull request Jul 18, 2023

Glutton Parachains paritytech/roadmap#37

Closed

5 tasks

MOZGIII mentioned this pull request Nov 8, 2023

Tracking issue to bump substrate related deps to polkadot-v0.9.39 humanode-network/humanode#828

Closed

11 tasks


		let clobber = Self::waste_ref_time_iter(vec![0u8; 64], n);

		// By casting it into a vec we can hopefully prevent the compiler from optimizing it

Conversation

Szegoo commented Dec 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Glutton

Pallet

Glutton Parachains

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ggwpez commented Feb 20, 2023

Uh oh!

Uh oh!

Uh oh!

koute Feb 20, 2023

Choose a reason for hiding this comment

Uh oh!

Szegoo Feb 20, 2023

Choose a reason for hiding this comment

Uh oh!

ggwpez Feb 20, 2023

Choose a reason for hiding this comment

Uh oh!

bkchr Feb 22, 2023

Choose a reason for hiding this comment

Uh oh!

koute Feb 22, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ggwpez commented Feb 20, 2023

Uh oh!

ggwpez commented Feb 20, 2023

Uh oh!

ggwpez commented Feb 20, 2023

Uh oh!

command-bot bot commented Feb 20, 2023

Uh oh!

ggwpez commented Feb 21, 2023

Uh oh!

ggwpez commented Feb 21, 2023

Uh oh!

Polkadot-Forum commented Mar 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Szegoo commented Dec 3, 2022 •

edited

Loading