Overhaul benchmarking infrastructure #63
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bare with me here... this is needed for a future commit in this PR
- BenchmarkBase is replaced by static calls to BenchmarkUtils - As much setup as possible is done statically and stored in static finals - Update SecureModules and avoid the deprecated constructors Still to do: - Reduce the amount of repetitive code - CacheConcurrent, CacheCopy and CacheClassLoader benchmarks - Benchmark class generation for arbitrary post sizes, instead of hardcoding 3 presets (single, dozen, hundred)
Allows for arbitrary multipliers of listeners with less code duplication
These are adding significant overhead on NoLoader results
This field is added at runtime
LexManos
reviewed
Jan 14, 2025
LexManos
approved these changes
Jan 17, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is the first step towards fixing EventBus' benchmarking infrastructure to reduce overhead and improve accuracy, based on work I've done over the past year or so working on NovaBus prototypes locally.
Comparisons show roughly a 5-25% reduction in like-for-like post times depending on the benchmark, without any changes to EventBus itself.
The two main changes are:
MinecraftForge.EVENT_BUS.post(...)Due to the nature of JMH, this sadly increases the amount of repetitive code, although this isn't as bad as it could've been thanks to the benchmark generation logic where applicable. As an added bonus, it's fairly trivial to benchmark arbitrary numbers of listeners now without being restricted to hardcoded presets.
The classloader variant benchmarks have been removed, as this mode is only functional when using ModLauncher anyway and the results are pretty much the same.The registration benchmarks are kept for now and avoid unnecessary synchronisation, but still have pretty high variance which limits their usefulness.Future work includes revisiting the Groovy code and adding a debug mode that re-enables JFR on request (a kind of non-benchmarking mode). I left those for now to get something functional out sooner rather than stalling for too long. I've been playing around with multiple optimisation ideas for current EventBus and comparing the before & after with the old system and the new system and the differences are much more visible on the new system.