Skip to content

show elapsed time, seed#61

Closed
pushcx wants to merge 1 commit intoyippee-fun:mainfrom
pushcx:show_seed
Closed

show elapsed time, seed#61
pushcx wants to merge 1 commit intoyippee-fun:mainfrom
pushcx:show_seed

Conversation

@pushcx
Copy link
Contributor

@pushcx pushcx commented Feb 28, 2025

I have an order-dependent test failure but I can't reproduce it without knowing the seed.

There are a couple small things; I can remove or extract to standalone PRs if you prefer:

  • I included elapsed time because it's cool.
  • I turned off rubocop suggesting extensions because it's an irritating advertisement in tool output.
  • rubocop cleaned up single quotes from stop error backtrace at framework #59
  • rubocop edited whitespace, only some of which is from stop error backtrace at framework #59, but never again in my life will I investigate why rubocop edits whitespace

Maybe elapsed/seed/passed/failed/errors output should be one line?

@pushcx
Copy link
Contributor Author

pushcx commented Feb 28, 2025

Sample output:

████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 100%

Elapsed: 52.2ms | Seed: 199273721947018783666797001917529743377
Passed: 70 | Failed: 0 | Errors: 0

@pushcx
Copy link
Contributor Author

pushcx commented Mar 1, 2025

In discussion, @joeldrapper explained that displaying the seed is not yet useful. The process of fanning work out to processes and threads is nondeterministic. Tiny variations in how long each test takes to run or other activity on the machine means the tests will be allocated differently. So thread isolation bugs and order-dependent tests will fire in different orders even when the same --seed is provided.

Joel, would you like any or all of the other changes? I can edit this PR down. Otherwise, let's close it.

@joeldrapper
Copy link
Collaborator

The other stuff looks good. Let's not include the seed until we have a deterministic option. Perhaps a simple implementation of a deterministic option would just be to set the processes and threads to 1.

In the future, I would like to use a simpler algorithm that splits the work into buckets before forming.

@pushcx pushcx closed this by deleting the head repository Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants