Complete `small_net` by marc-casperlabs · Pull Request #14 · casper-network/casper-node

marc-casperlabs · 2020-06-08T07:56:50Z

This PR contains two major themes:

It cleans up the handling of updating endpoints inside the networking system. This was hard to read in the past, now it should at least be a little bit clearer. As a reminder, an endpoint is a self-signed (timestamp, socket address, certificate) pair that nodes store and publish to other connecting nodes to spread the network topology info.
More important, it introduces "announcements". These implement publish-subscribe-style effects. In this case it is used to publish incoming network messages in small net.
(Bonus) The handling of requests and announcements changed, removing the dispatch in the From implementation. Instead it is derived and extra reactor events are added. The core benefit is that now all dispatching is easy-to-find inside the reactors message handling.

… implicitly during `From<>` conversion

src/components/small_network.rs

src/components/small_network/endpoint.rs

…te_listener`

remove simple subdir from cargo-casper

14: Checked runtime stack newtype. r=Fraser999 a=Fraser999 Added a configurable limit to the execution engine's runtime call stack. The limit can be configured via the chainspec. Closes [casper-network#88](https://github.com/casper-network/Governance/issues/88). Co-authored-by: Luís Fernando Schultz Xavier da Silveira <luis@casperlabs.io>

…ate-fanout Faster retrieve state fanout

2515: Port 1.4.3 changes to dev r=mpapierski a=mpapierski PRs that we already backported. There's one manual commit where I've run through a dry run of 1.4.3 merge and applied the remaining changes that we backported but modified without changing dev again. I cherry-picked squashed PRs that weren't backports. All PRs were squashed into one commit before cherry-picking. - `#13` deploy acceptor fix - `#14` Call stack - `#22` stack height - `#25` version bumps - `#27` Validator slots Co-authored-by: Fraser Hutchison <fraser@casperlabs.io> Co-authored-by: Luís Fernando Schultz Xavier da Silveira <luis@casperlabs.io> Co-authored-by: Michał Papierski <michal@casperlabs.io> Co-authored-by: Michał Papierski <michal@papierski.net>

Rename `next_address` to `random_bytes`

binary_port: Refactor component tests

* Add script data offset function to tx Script data offset will need to be accessible to users of the transaction API in order to create data for scripts * Update transaction script offset to return Option The API must be consistent and when a function that is directed to a specifc variant of transaction is called over a script or vice-versa, it should be protected by either a result or an option.

* Initial commit * Update README.md * Fuel VM in Rust merging files from original repo refactor impl ethcall completed todos refactor program into interpreter mod carry create2 programs through callframes impl create2 impl codesize based on callframes impl fuel tx init and data handling refactor update init code width impl eth ops impl fuel ops define eth and fuel ops Adding init field to eth tx format define eth account and tx types impl rotate ops sign extend from u16 to u64 impl subi op Sign extend imm val for arithmetic op fix and style impl of srav and sra impl of sltiu and sltu remove unnecessary refs working tx handler define op for data ld and test for tx exec first step of tx execution impl impl addtl tx types minor refactor initial tx format moving from usize to u8 for size types defining serder of eth opcodes advancing notes updating notes cleaner style base impl for all opcodes implement malloc/free, local var, and stack opcodes working call and return architecture improving architecture and execution some code improvements improved stack handling thru mutable references intermediate commit with some types and methods and cleanup intermediate commit for stackframes initial stack impl using values in registers for computations Run programs in loop to completion Update notes Adding tests for u32 bit manipulation Change opcode instructions to 32 bits wide change immediate values to 16-bit wide minor edits tidy completing a few opcode implementations including beq bug fix for tuple return Starting to implement opcodes intermediate commit with opcode deser intermediate commit with opcode ser Update status Adding simple unit tests to memory after execution Working simple execution with memory dump Removing unused const updating notes updating notes refactor u64 funcs to file applying unit tests to serder initial working serder of ADD opcode in program AND serialization and refactoring minor cleanup Adding some indexed methods Initial bit manipulation for opcode ser Intermediate commit to opcode serder Starting trait impl for Opcodes Implement bit manipulation for u64 Enable basic architecture for tests intermediate commit with restructured types for vm components Working memory with hi register splitting out base types adding binary that matches Cargo.toml adding binary that matches Cargo.toml running instance, more cleanup pending next step towards initial commit next step towards initial commit Fuel VM in Rust * Adding new tx format based on FuelLabs/fuel-specs#6 * lifting utxo_id into parent struct * working base ftx test * working new tx format * Adding `stateWitnessIndex` to OutputContract * Working extended Fuel tx contract subtype * impl sha256 Set $of flag in div and mod by 0 Rename of uf flags Rename of uf flags j jr tests xor xori tests sub subi tests sll sllv srl srlv tests or ori tests mult multi test updates to match spec #0d39f49 impl kecakk op Add convenience functions * jump ops memory copy and compare ops impl state opcodes using hashmap refactor overflow into distinct register impl sha256 Set $of flag in div and mod by 0 Rename of uf flags Rename of uf flags j jr tests xor xori tests sub subi tests sll sllv srl srlv tests or ori tests mult multi test updates to match spec #0d39f49 impl kecakk op Add convenience functions * cleanup jnzi test jnz test jump ops memory copy and compare ops impl state opcodes using hashmap refactor overflow into distinct register impl sha256 Set $of flag in div and mod by 0 Rename of uf flags Rename of uf flags j jr tests xor xori tests sub subi tests sll sllv srl srlv tests or ori tests mult multi test updates to match spec #0d39f49 impl kecakk op Add convenience functions * slight adjustment to hashmap access * added very basic gas * ecrecover op * Cfe and Cfs callframe stack opcodes * Add test and format CI (#12) * Add test and format CI. * fmt * Remove test suite unneeded. * Update test CI (#13) * Add private keys for dependencies. * Use minimal toolchain profile. * Use nightly toolchain. * Update opcodes list (#9) * Update opcodes list The opcodes list was outdated and needed to be resync with the specs. https://github.com/FuelLabs/fuel-specs/blob/master/specs/vm/opcodes.md Also, the previous implementation didn't consider the different sizes of immediate values, treating everything as u16. The API still need to improve to prevent users from allocating invalid register values with overflow. Resolves #8 * Update interpreter to latest specs The interpreter is not fully sync with the specs. Also, the functionality scope can be split into different modules to facilitate maintenance of the routines. * Implement interpreter mem and crypto functions Some of the functions contains particularities such as the ecrecover that uses a special encoding regarding the recovery id. These are not native to the public libraries and must be manually handled. * Add test case for keccak256 This new test case is symmetric to the sha256 test case with the crypto primitive as difference. * Add Transaction structure The transaction will define the VM initialization as well as the underlying input/output/contract types. * Implement recoverable secp256k1 signature The EcRecover scheme may benefit from a compression that can save an additional byte of storage. This crypto primitive is implemented using rusk-secp256k1 as backend. * Add MemClearI operation * Rename 32-byte storage ops to "quad word" * Rename opcodes to fit in 4 chars * Add serialization for all tx types * Change VM init to copy tx to stack The initialization of the VM should copy the hash of the transaction, its size and the serialized bytes to the stack area so the execution of the program can access it when required. * Update IO according to specs The specs points that every number should be stored as u64 and the arrays of bytes should be padded to multiples of 8. * Change `crypto::hash` to encapsulate primitive The underlying default hash function of the crypto module should not impact the API and any change in the primitive should be transparent to the users of the interpreter. * Add Log opcode This opcode will perform a naive dump of the selected registers ignoring the zeroed indices. This is a primitive implementation to a more complex log/event/topic tracing system. * Add binary interpreter This binary executes transactions and output the log opcodes. * Add tracing to interpreter execution * Add debug to registers The mutated registers should be print if the log level is set to debug. * Update memcopy to non-overlapping THe `MCP` operation should fail if the memory segments are overlapping. This will allow the internals to benefit from `memcopy` instead of `memmove`. * Add `is_valid` function for all transaction types * Add Tx validation to VM initialization The VM initialization must validate the transaction before the registers and memory are prepared for execution. The TX validation is critical and most of the edge cases must have test coverage. * Optimize code and remove redundant checks * Add Call, CallFrame and MemoryRange The memory range tuples in the VM are expected to behave as start/size pairs. However, Rust uses start/end patterns. The `MemoryRange` type is an abstraction to interface with the values provided by the bytecode operations and Rust memory scheme. The `Call` and `CallFrame` structures will heavily depend on that because they have internal references to owned memory regions. * Add Call opcode implementation The call frames structure and the reflection of a call in the registers and memory is critical to the system and must be broadly tested. This commit introduces the update of the program counter. * Update dependencies to use fuel-* libs The transaction and opcode structs were split into their own libraries. This commit introduces the usage of these libraries as dependencies. * Update ECR to clear err In case of failure, ECRecover should set the recovered public-key to `0` and turn the `err` flag on. In case of success, `$rA` will be set with the recovered public-key and the error flag will be cleared. * Add predicate verification for tx create coin * Add predicate verification Predicate verification should initialize the VM in a special state and execute the opcodes under special constraints defined in the specs. * Remove Travis config. (#14) * Add helper methods for VM executors (#17) * Change LW and SW to multiple of a Word immediate (#16) To maintain consistency with the high-level language, the LW and SW immediate values will be used as word-multiple values instead of raw memory addresses. * Mint and burn (#18) * Add CALL and CCP implementation Both call and ccp relies on contract deployment over the blockchain. These should be included with a refactor over the call frames structure to correctly point to a contract code after a call frame is pushed to the stack. * Fix memory boundaries ownership check The memory ranges are exclusive in the upper bound. This way, the ownership check must reflect that standard for range checks. * Add Mint and Burn opcode execution * Update SW and LW to take imm as words * Update LW and SW to mul word imm with explicit op * Add function to encapsulate external const check * Rename crate to `fuel-core` (#19) `fuel-vm-rust` don't reflect the repo name and this may be confusing under certain contexts. Also, there is no particular reason on why these two names should diverge. * Add data backend traits The VM must be agnostic to the data backend, and this should comply only with a non-restrictive trait. This commit introduces the `Storage` trait. Some of the interpreter functionality will specify implementation requirements for a given Key/Value concrete pair. * Use wrapper type for key structs * Rename to fuel-vm Co-authored-by: John Adler <adlerjohn@users.noreply.github.com> Co-authored-by: Shahan Khatchadourian <shahan.k.code@gmail.com>

* Add serde-types feature * Add iterator to bytes capability for Opcode

* Reintroduce auto_impl * cargo sort

# This is the 1st commit message: Initial wip # The commit message #2 will be skipped: # Basic case working. Pre-fixture tests # The commit message #3 will be skipped: # Initial fixture tests passing # The commit message #4 will be skipped: # Test case passing for disabled versions # The commit message #5 will be skipped: # EE tests passing # The commit message #6 will be skipped: # Additional fixture tests and PR prep # The commit message #7 will be skipped: # Run make format # The commit message #8 will be skipped: # X # The commit message #9 will be skipped: # X # The commit message casper-network#10 will be skipped: # X # The commit message casper-network#11 will be skipped: # X # The commit message casper-network#12 will be skipped: # Add flag to engine config to return error # The commit message casper-network#13 will be skipped: # Add flag to chainspec.toml # The commit message casper-network#14 will be skipped: # Address CI issues # The commit message casper-network#15 will be skipped: # Add contract runtime tests

# This is the 1st commit message: Initial wip # The commit message #2 will be skipped: # Basic case working. Pre-fixture tests # The commit message #3 will be skipped: # Initial fixture tests passing # The commit message #4 will be skipped: # Test case passing for disabled versions # The commit message #5 will be skipped: # EE tests passing # The commit message #6 will be skipped: # Additional fixture tests and PR prep # The commit message #7 will be skipped: # Run make format # The commit message #8 will be skipped: # X # The commit message #9 will be skipped: # X # The commit message casper-network#10 will be skipped: # X # The commit message casper-network#11 will be skipped: # X # The commit message casper-network#12 will be skipped: # Add flag to engine config to return error # The commit message casper-network#13 will be skipped: # Add flag to chainspec.toml # The commit message casper-network#14 will be skipped: # Address CI issues # The commit message casper-network#15 will be skipped: # Add contract runtime tests # The commit message casper-network#16 will be skipped: # rebasing with darthsiroftardis/change-package-version-calls # The commit message casper-network#17 will be skipped: # fmt fix # The commit message casper-network#18 will be skipped: # fmt fix # The commit message casper-network#19 will be skipped: # fmt fix

# This is the 1st commit message: Initial wip # The commit message #2 will be skipped: # Basic case working. Pre-fixture tests # The commit message #3 will be skipped: # Initial fixture tests passing # The commit message #4 will be skipped: # Test case passing for disabled versions # The commit message #5 will be skipped: # EE tests passing # The commit message #6 will be skipped: # Additional fixture tests and PR prep # The commit message #7 will be skipped: # Run make format # The commit message #8 will be skipped: # X # The commit message #9 will be skipped: # X # The commit message casper-network#10 will be skipped: # X # The commit message casper-network#11 will be skipped: # X # The commit message casper-network#12 will be skipped: # Add flag to engine config to return error # The commit message casper-network#13 will be skipped: # Add flag to chainspec.toml # The commit message casper-network#14 will be skipped: # Address CI issues # The commit message casper-network#15 will be skipped: # Add contract runtime tests # The commit message casper-network#16 will be skipped: # rebasing with darthsiroftardis/change-package-version-calls # The commit message casper-network#17 will be skipped: # fmt fix # The commit message casper-network#18 will be skipped: # fmt fix # The commit message casper-network#19 will be skipped: # fmt fix # The commit message casper-network#20 will be skipped: # refactoring put_key contract # The commit message casper-network#21 will be skipped: # refactoring put_key contract # The commit message casper-network#22 will be skipped: # X

# This is the 1st commit message: rebasing # The commit message #2 will be skipped: # WIP # The commit message #3 will be skipped: # WIP # The commit message #4 will be skipped: # wip # The commit message #5 will be skipped: # wip # The commit message #6 will be skipped: # wip # The commit message #7 will be skipped: # wip # The commit message #8 will be skipped: # wip # The commit message #9 will be skipped: # wip # The commit message casper-network#10 will be skipped: # wip # The commit message casper-network#11 will be skipped: # wip # The commit message casper-network#12 will be skipped: # wip # The commit message casper-network#13 will be skipped: # rebasing # The commit message casper-network#14 will be skipped: # wip # The commit message casper-network#15 will be skipped: # wip # The commit message casper-network#16 will be skipped: # wip # The commit message casper-network#17 will be skipped: # wip # The commit message casper-network#18 will be skipped: # wip # The commit message casper-network#19 will be skipped: # wip

# This is the 1st commit message: rebasing # The commit message #2 will be skipped: # WIP # The commit message #3 will be skipped: # WIP # The commit message #4 will be skipped: # wip # The commit message #5 will be skipped: # wip # The commit message #6 will be skipped: # wip # The commit message #7 will be skipped: # wip # The commit message #8 will be skipped: # wip # The commit message #9 will be skipped: # wip # The commit message casper-network#10 will be skipped: # wip # The commit message casper-network#11 will be skipped: # wip # The commit message casper-network#12 will be skipped: # wip # The commit message casper-network#13 will be skipped: # rebasing # The commit message casper-network#14 will be skipped: # wip # The commit message casper-network#15 will be skipped: # wip # The commit message casper-network#16 will be skipped: # wip # The commit message casper-network#17 will be skipped: # wip # The commit message casper-network#18 will be skipped: # wip # The commit message casper-network#19 will be skipped: # wip # The commit message casper-network#20 will be skipped: # rebasing # The commit message casper-network#21 will be skipped: # wip # The commit message casper-network#22 will be skipped: # wip # The commit message casper-network#23 will be skipped: # wip

# This is the 1st commit message: rebasing # The commit message #2 will be skipped: # WIP # The commit message #3 will be skipped: # WIP # The commit message #4 will be skipped: # wip # The commit message #5 will be skipped: # wip # The commit message #6 will be skipped: # wip # The commit message #7 will be skipped: # wip # The commit message #8 will be skipped: # wip # The commit message #9 will be skipped: # wip # The commit message casper-network#10 will be skipped: # wip # The commit message casper-network#11 will be skipped: # wip # The commit message casper-network#12 will be skipped: # wip # The commit message casper-network#13 will be skipped: # rebasing # The commit message casper-network#14 will be skipped: # wip # The commit message casper-network#15 will be skipped: # wip # The commit message casper-network#16 will be skipped: # wip # The commit message casper-network#17 will be skipped: # wip # The commit message casper-network#18 will be skipped: # wip # The commit message casper-network#19 will be skipped: # wip # The commit message casper-network#20 will be skipped: # rebasing # The commit message casper-network#21 will be skipped: # wip # The commit message casper-network#22 will be skipped: # wip # The commit message casper-network#23 will be skipped: # wip # The commit message casper-network#24 will be skipped: # wip # The commit message casper-network#25 will be skipped: # wip # The commit message casper-network#26 will be skipped: # wip # The commit message casper-network#27 will be skipped: # wip # The commit message casper-network#28 will be skipped: # wip # The commit message casper-network#29 will be skipped: # wip

# This is the 1st commit message: rebasing # The commit message #2 will be skipped: # WIP # The commit message #3 will be skipped: # WIP # The commit message #4 will be skipped: # wip # The commit message #5 will be skipped: # wip # The commit message #6 will be skipped: # wip # The commit message #7 will be skipped: # wip # The commit message #8 will be skipped: # wip # The commit message #9 will be skipped: # wip # The commit message casper-network#10 will be skipped: # wip # The commit message casper-network#11 will be skipped: # wip # The commit message casper-network#12 will be skipped: # wip # The commit message casper-network#13 will be skipped: # rebasing # The commit message casper-network#14 will be skipped: # wip # The commit message casper-network#15 will be skipped: # wip # The commit message casper-network#16 will be skipped: # wip # The commit message casper-network#17 will be skipped: # wip # The commit message casper-network#18 will be skipped: # wip # The commit message casper-network#19 will be skipped: # wip # The commit message casper-network#20 will be skipped: # rebasing # The commit message casper-network#21 will be skipped: # wip # The commit message casper-network#22 will be skipped: # wip # The commit message casper-network#23 will be skipped: # wip # The commit message casper-network#24 will be skipped: # wip # The commit message casper-network#25 will be skipped: # wip # The commit message casper-network#26 will be skipped: # wip # The commit message casper-network#27 will be skipped: # wip # The commit message casper-network#28 will be skipped: # wip # The commit message casper-network#29 will be skipped: # wip # The commit message casper-network#30 will be skipped: # wip # The commit message casper-network#31 will be skipped: # wip # The commit message casper-network#32 will be skipped: # wip

# This is the 1st commit message: rebasing # The commit message #2 will be skipped: # WIP # The commit message #3 will be skipped: # WIP # The commit message #4 will be skipped: # wip # The commit message #5 will be skipped: # wip # The commit message #6 will be skipped: # wip # The commit message #7 will be skipped: # wip # The commit message #8 will be skipped: # wip # The commit message #9 will be skipped: # wip # The commit message casper-network#10 will be skipped: # wip # The commit message casper-network#11 will be skipped: # wip # The commit message casper-network#12 will be skipped: # wip # The commit message casper-network#13 will be skipped: # rebasing # The commit message casper-network#14 will be skipped: # wip # The commit message casper-network#15 will be skipped: # wip # The commit message casper-network#16 will be skipped: # wip # The commit message casper-network#17 will be skipped: # wip # The commit message casper-network#18 will be skipped: # wip # The commit message casper-network#19 will be skipped: # wip # The commit message casper-network#20 will be skipped: # rebasing # The commit message casper-network#21 will be skipped: # wip # The commit message casper-network#22 will be skipped: # wip # The commit message casper-network#23 will be skipped: # wip # The commit message casper-network#24 will be skipped: # wip # The commit message casper-network#25 will be skipped: # wip # The commit message casper-network#26 will be skipped: # wip # The commit message casper-network#27 will be skipped: # wip # The commit message casper-network#28 will be skipped: # wip # The commit message casper-network#29 will be skipped: # wip # The commit message casper-network#30 will be skipped: # wip # The commit message casper-network#31 will be skipped: # wip # The commit message casper-network#32 will be skipped: # wip # The commit message casper-network#33 will be skipped: # wip # The commit message casper-network#34 will be skipped: # wip # The commit message casper-network#35 will be skipped: # wip # The commit message casper-network#36 will be skipped: # wip # The commit message casper-network#37 will be skipped: # wip # The commit message casper-network#38 will be skipped: # wip # The commit message casper-network#39 will be skipped: # wip # The commit message casper-network#40 will be skipped: # rebasing

marc-casperlabs added 6 commits June 7, 2020 23:06

Only try to connect to root node if we are not the root node

a3b98fe

Handle requests explicitly in event dispatcher in reactor, instead of…

a79559b

… implicitly during `From<>` conversion

Improve logging throughout networking module

85d4cd8

Cleanup code by replacing if with match in small_network

55e47c8

Cleanup endpoint replacement functions in small_network

30d797b

Introduce announcements, finally making the pinger work

66597bd

marc-casperlabs requested review from EdHastingsCasperAssociation and Fraser999 June 8, 2020 07:56

marc-casperlabs self-assigned this Jun 8, 2020

marc-casperlabs added 2 commits June 8, 2020 11:15

Add somewhat sensible defaults for reconnection attempts

c9f7aba

Fix/suppress clippy warnings

5303ef8

Fraser999 approved these changes Jun 8, 2020

View reviewed changes

src/components/small_network.rs Outdated Show resolved Hide resolved

src/components/small_network.rs Show resolved Hide resolved

src/components/small_network/endpoint.rs Outdated Show resolved Hide resolved

marc-casperlabs added 5 commits June 8, 2020 16:19

Fix verbosity in format macro in endpoint.rs as per suggestion

6ecf029

Add documentation for otherwise cryptic boolean return value of `crea…

8b86333

…te_listener`

Avoid confusion by renaming pk to private_key in small_network

5d46f22

Merge remote-tracking branch 'upstream/master' into mbr/fix-small-net

958a61b

Map payloads of network requests cleanly

bbc7bfc

marc-casperlabs merged commit fb1440a into casper-network:master Jun 8, 2020

marc-casperlabs deleted the mbr/fix-small-net branch June 8, 2020 15:28

casperlabs-bors-ng bot pushed a commit that referenced this pull request Oct 19, 2021

Merge pull request #14 from Fraser999/Revert-Erc20-Cargo-Casper

be07d5d

remove simple subdir from cargo-casper

marc-casperlabs pushed a commit to marc-casperlabs/casper-node that referenced this pull request Dec 14, 2021

Merge pull request casper-network#14 from xcthulhu/faster-retrieve-st…

45bb894

…ate-fanout Faster retrieve state fanout

mpapierski mentioned this pull request Dec 17, 2021

Port 1.4.3 changes to dev #2515

Merged

casperlabs-bors-ng bot pushed a commit that referenced this pull request May 16, 2022

Merge pull request #14 from rafal-ch/2965_rename_to_random_bytes

b1073b3

Rename `next_address` to `random_bytes`

casperlabs-bors-ng bot pushed a commit that referenced this pull request Feb 29, 2024

Merge pull request #14 from rafal-ch/test_refactor

f0dc549

binary_port: Refactor component tests

rafal-ch pushed a commit that referenced this pull request Sep 11, 2024

Add serde-types feature (#14)

d09d179

* Add serde-types feature * Add iterator to bytes capability for Opcode

rafal-ch pushed a commit that referenced this pull request Sep 11, 2024

Update author. (#14)

3f2c51b

rafal-ch pushed a commit that referenced this pull request Sep 11, 2024

Bump to v0.4.0 (#14)

0a6ccbf

rafal-ch pushed a commit that referenced this pull request Sep 11, 2024

Reintroduce auto_impl (#14)

fe104aa

* Reintroduce auto_impl * cargo sort

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Complete `small_net`#14

Complete `small_net`#14
marc-casperlabs merged 13 commits intocasper-network:masterfrom
marc-casperlabs:mbr/fix-small-net

marc-casperlabs commented Jun 8, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

marc-casperlabs commented Jun 8, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants