Is your feature request related to a problem? Please describe.
With #14156 merged, we are able to collect statstictic data of performance for every command.
But lack of storing them and visualizing it, and leverage the data to monitor and alert any regression.
Describe the solution you'd like
Since measuring a command's performance could cost much time, we could setup a daily-scheduled pipeline to measuring all of them in ADO. Then the data could sent to datafactory to store, and extract statistic data into PowerBI to visualize.
As for monitor and alert, we could setup a job in CI to do incremental performance check on any modified commands of a PR, and compare with the histoical statistic data. If the differencies reach out a limit bar, the CI job would fail.
Describe alternatives you've considered
As for alert, we could use Gitub PR comment to warn user instead of fail the job.
Additional context
Is your feature request related to a problem? Please describe.
With #14156 merged, we are able to collect statstictic data of performance for every command.
But lack of storing them and visualizing it, and leverage the data to monitor and alert any regression.
Describe the solution you'd like
Since measuring a command's performance could cost much time, we could setup a daily-scheduled pipeline to measuring all of them in ADO. Then the data could sent to datafactory to store, and extract statistic data into PowerBI to visualize.
As for monitor and alert, we could setup a job in CI to do incremental performance check on any modified commands of a PR, and compare with the histoical statistic data. If the differencies reach out a limit bar, the CI job would fail.
Describe alternatives you've considered
As for alert, we could use Gitub PR comment to warn user instead of fail the job.
Additional context