Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

chainHead: Limit ongoing operations#14699

Merged
lexnv merged 28 commits intomasterfrom
lexnv/chainhead_limits
Aug 15, 2023
Merged

chainHead: Limit ongoing operations#14699
lexnv merged 28 commits intomasterfrom
lexnv/chainhead_limits

Conversation

@lexnv
Copy link
Contributor

@lexnv lexnv commented Aug 2, 2023 •
edited by niklasad1
Loading

This PR adds support for limiting the number of ongoing operations:

  • at most 16 operations can be executed by a chainHead_follow subscription (items in chainHead_storage count each as a operation)
  • methods return limitReached when capacity cannot be reserved (or subscription ID is invalid or stale)
  • chainHead_storage discards elements when there's not enough capacity
  • struct ChainHeadConfig is added to have granular control over the limits of the API
  • reserving capacity and permits is a slim wrapper over Arc<Mutex<usize>> with a configurable maximum and a RAII style release of capacity on drop
  • the capacity permit is propagated to the BlockGuard
  • unit-test for low-level OngoingOperations and PermitOperations
  • integration-test for limit propagation to methods via ensure_operation_limits_works()

This builds on top of: #14692
Closes: #14640

// @paritytech/subxt-team

lexnv added 19 commits July 31, 2023 18:54
@lexnv
chainHead/api: Make storage/body/call pure RPC methods
4787dba 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Add mpsc channel between RPC methods
8050811 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/subscriptions: Extract mpsc::Sender via BlockGuard
3b30eb3 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/subscriptions: Generate and provide the method operation ID
d5305a9 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Generate chainHead_body response
50c840d 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Generate chainHead_call response
6c5940c 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Generate chainHead_storage responses
589efa8 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Propagate responses of methods to chainHead_follow
8062837 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/tests: Adjust chainHead_body responses
4f7b445 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/tests: Adjust chainHead_call responses
53fcd99 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/tests: Adjust chainHead_call responses
167dd72 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/tests: Ensure unique operation IDs across methods
35aca6d 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/events: Remove old method events
008fa22 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/subscriptions: Add limit helper
6c30aef 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/subscription: Expose limits to BlockGuard
a29d677 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/tests: Adjust testing to ongoing operations
a33d80c 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Make limits configurable via ChainHeadConfig
40a73fb 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/tests: Adjust testing to ChainHeadConfig
eb15293 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/tests: Ensure operation limits discards items
a026fe0 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv lexnv added A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit B1-note_worthy Changes should be noted in the release notes T2-API This PR/Issue is related to APIs. labels Aug 2, 2023
@lexnv lexnv self-assigned this Aug 2, 2023
Base automatically changed from lexnv/chainhead_extend_resp to master August 8, 2023 18:13
lexnv added 4 commits August 9, 2023 17:14
@lexnv
Merge remote-tracking branch 'origin/master' into lexnv/chainhead_limits
4e4cca8 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Improve documentation
defc196 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Rename OngoingOperations -> LimitOperations
4cacbc1 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Rename reserve -> reserve_at_most
09c743c 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
Copy link
Contributor Author

lexnv commented Aug 10, 2023

bot rebase

@paritytech-processbot
Copy link

paritytech-processbot bot commented Aug 10, 2023

Rebased

@lexnv lexnv requested a review from skunert August 10, 2023 13:00
@lexnv lexnv mentioned this pull request Aug 11, 2023
Merged
niklasad1
niklasad1 reviewed Aug 14, 2023
View reviewed changes
niklasad1
niklasad1 reviewed Aug 14, 2023
View reviewed changes
niklasad1
niklasad1 reviewed Aug 14, 2023
View reviewed changes
lexnv added 3 commits August 14, 2023 15:24
@lexnv
chainHead: Use duration const instead of u64
ab481e8 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/subscription: Use tokio::sync::Semaphore for limits
09e7372 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
Merge remote-tracking branch 'origin/lexnv/chainhead_limits' into lex…
8954c33 
…nv/chainhead_limits
niklasad1
niklasad1 reviewed Aug 14, 2023
View reviewed changes
skunert
skunert reviewed Aug 15, 2023
View reviewed changes
Copy link
Contributor

@skunert skunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Only some small comments.

@lexnv @skunert
Update client/rpc-spec-v2/src/chain_head/subscription/inner.rs
a3df109 
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
@lexnv lexnv merged commit a79a2af into master Aug 15, 2023
@lexnv lexnv deleted the lexnv/chainhead_limits branch August 15, 2023 12:17
Ank4n pushed a commit that referenced this pull request Aug 20, 2023
@lexnv @skunert @Ank4n
chainHead: Limit ongoing operations (#14699)
981de06 
* chainHead/api: Make storage/body/call pure RPC methods

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Add mpsc channel between RPC methods

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/subscriptions: Extract mpsc::Sender via BlockGuard

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/subscriptions: Generate and provide the method operation ID

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Generate `chainHead_body` response

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Generate `chainHead_call` response

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Generate `chainHead_storage` responses

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Propagate responses of methods to chainHead_follow

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/tests: Adjust `chainHead_body` responses

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/tests: Adjust `chainHead_call` responses

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/tests: Adjust `chainHead_call` responses

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/tests: Ensure unique operation IDs across methods

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/events: Remove old method events

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/subscriptions: Add limit helper

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/subscription: Expose limits to `BlockGuard`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/tests: Adjust testing to ongoing operations

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Make limits configurable via `ChainHeadConfig`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/tests: Adjust testing to `ChainHeadConfig`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/tests: Ensure operation limits discards items

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Rename `OngoingOperations` -> `LimitOperations`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Rename reserve -> reserve_at_most

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead: Use duration const instead of u64

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chainHead/subscription: Use tokio::sync::Semaphore for limits

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update client/rpc-spec-v2/src/chain_head/subscription/inner.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
@crystalin crystalin mentioned this pull request Oct 20, 2023
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers

@skunert skunert skunert approved these changes

@niklasad1 niklasad1 niklasad1 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@lexnv lexnv

Labels

A0-please_review Pull request needs code review. B1-note_worthy Changes should be noted in the release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit T2-API This PR/Issue is related to APIs.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[RPC-Spec-V2] Limit ongoing operations

3 participants

@lexnv @skunert @niklasad1