Strip version metadata from Results dict when --no-metadata is used#874
Open
ilonatommy wants to merge 1 commit intomainfrom
Open
Strip version metadata from Results dict when --no-metadata is used#874ilonatommy wants to merge 1 commit intomainfrom
--no-metadata is used#874ilonatommy wants to merge 1 commit intomainfrom
Conversation
Member
I am confused, because they do exactly what they are meant for ;) They are placed as-is by binding the specific fields to Text outputs so we can see what version the KPIs were run with. |
Member
|
Also these are not "metadata" the way I see it. It's a real value of a run, hence why it's put in results directly. There are ways to alter the returned document if you don't want specific results, or want to add new ones. See how the 90th percentile of CPU is computed dynamically. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The ASP.NET Core benchmarks Power BI dashboard (KPIs page) shows two broken rows: "NETCore App" and "ASP.NET Core" that display raw version strings (e.g. 10.0.5+a612c2a1056f) instead of KPI charts
I was convinced some benchmarks are broken and was looking for failed jobs to fix them, it does not look intentional:
These are not real benchmark scenarios. No scenario YAML in aspnet/Benchmarks (https://github.com/aspnet/Benchmarks) defines them. They are phantom rows created when Power BI reads text-valued version entries from the results dictionary in stored JSON.
Root cause
The crank Agent registers AspNetCoreVersion and NetCoreAppVersion as measurements (Startup.cs (https://github.com/dotnet/crank/blob/main/src/Microsoft.Crank.Agent/Startup.cs#L3238-L3301)). The Controller's AggregateAndReduceResults() aggregates all measurements — including these text strings — into jobResult.Results.
The baseline pipelines pass
--no-metadata --no-measurements, but:--no-metadataonly clears Metadata[] (the description array)--no-measurementsonly clears Measurements[] (the raw samples)This has been the case since the initial commit (0347e98, Jul 2020).
Fix
When
--no-metadatais active, also remove the well-known version keys fromjobResult.Results: