Skip to content

Factor substrate node runner into separate crate#913

Merged
jsdw merged 9 commits intomasterfrom
jsdw-substrate-runner
Apr 17, 2023
Merged

Factor substrate node runner into separate crate#913
jsdw merged 9 commits intomasterfrom
jsdw-substrate-runner

Conversation

@jsdw
Copy link
Collaborator

@jsdw jsdw commented Apr 14, 2023

So that we can re-use the logic for starting a test substrate node up in both the test-runtime crate (to hopefully avoid the retry errors we've been seeing a bit lately) and the integration-tests crate which also spawns a node.

I added support for @niklasad1's upcoming PR in terms of capturing stdout but not in terms of handling the invalid flag; perhaps I should! (but until then, this should still be a bit of an improvement I hope)

I also removed the Subxt dep on the build.rs of test-runtime in the hopes that it would mean it is triggered much less often (ie not every time Subxt is changed), but let's see aye!

Closes #909

@jsdw jsdw requested a review from a team as a code owner April 14, 2023 12:38
@jsdw jsdw marked this pull request as draft April 14, 2023 12:47
@jsdw jsdw marked this pull request as ready for review April 14, 2023 13:34
@jsdw jsdw marked this pull request as draft April 14, 2023 13:34
@jsdw jsdw marked this pull request as ready for review April 14, 2023 16:26
@jsdw
Copy link
Collaborator Author

jsdw commented Apr 14, 2023

When #907 merges I'll remove the parity-scale-codec dependency from test-runtime too

.arg("--dev")
.stdout(process::Stdio::piped())
.stderr(process::Stdio::piped())
.arg("--port=0")
Copy link
Contributor

@niklasad1 niklasad1 Apr 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we care about the libp2p port? :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the reasoning is to prevent any overlap with the port used for other substrate instances we spawn; I guess Substrate will pick a random port for us already if the one we give isn't any good, but probably just getting the kernel to pick an OK port straight off is a touch better anyway?

.stdout(process::Stdio::piped())
.stderr(process::Stdio::piped())
.arg("--port=0")
.arg("--rpc-port=0")
Copy link
Contributor

@niklasad1 niklasad1 Apr 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could omit the port for both --rpc--port, --ws-port here because substrate would fallback to 0 anyway, this way it would be backward-compatible :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm but ah true; backward compat is a good reason not to have them; I'll remove :)

// Consume a stderr reader from a spawned substrate command and
// locate the port number that is logged out to it.
fn try_find_substrate_port_from_output(r: impl Read + Send + 'static) -> Option<u16> {
BufReader::new(r).lines().take(50).find_map(|line| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe increase the number of a lines a bit to future proof?

Suggested change
BufReader::new(r).lines().take(50).find_map(|line| {
BufReader::new(r).lines().take(1024).find_map(|line| {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I wanted it to fail reasonably quickly if for some reason it wasn't picking up the port; I started at 100 but I think I counted right now that it's something like 25 lines to get the port, so 50 was my "safe" number; maybe 100 to compromise a bit? It should never take that long for the port to be logged I'd hope :)

jsdw and others added 4 commits April 17, 2023 11:51
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
@jsdw jsdw merged commit 2f1b67b into master Apr 17, 2023
@jsdw jsdw deleted the jsdw-substrate-runner branch April 17, 2023 10:42
@jsdw jsdw mentioned this pull request Jun 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test-runtime: Run Substrate node on port 0 for more reliable execution

4 participants