Skip to content

[MAUI][PERF] Maui Perf add iOS Podcast SOD Tests #67165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
LoopedBard3 merged 9 commits into from
Mar 28, 2022
Merged

[MAUI][PERF] Maui Perf add iOS Podcast SOD Tests #67165

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
453f9ed
Setup pipeline for testing.
LoopedBard3 Mar 15, 2022
0eb6304
Only test the Maui install and build to figure out the path for the i…
LoopedBard3 Mar 15, 2022
0c2225c
Add build for Maui Podcast iOS with printout of all folders.
LoopedBard3 Mar 15, 2022
fae71d1
Add specifier for the csproj to build.
LoopedBard3 Mar 16, 2022
eb3117f
Build the components before the main build.
LoopedBard3 Mar 16, 2022
cf2a6bd
Try building instead of publishing.
LoopedBard3 Mar 17, 2022
b565c17
Try publishing instead of building.
LoopedBard3 Mar 24, 2022
e4a45a2
Pipe the built Podcast IPA file through and test it.
LoopedBard3 Mar 24, 2022
137e39f
Undo testing changes for final test.
LoopedBard3 Mar 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 33 additions & 4 deletions eng/pipelines/coreclr/templates/build-perf-maui-apps.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,12 +157,11 @@ steps:
pwd
chmod -R a+r .
../../../dotnet restore Microsoft.NetConf2021.Maui.csproj
../../../dotnet publish Microsoft.NetConf2021.Maui.csproj -bl:MauiAndroid.binlog -r android-arm64 --self-contained --no-restore -f net6.0-android -c Release
../../../dotnet publish Microsoft.NetConf2021.Maui.csproj -bl:MauiPodcastAndroid.binlog -r android-arm64 --self-contained --no-restore -f net6.0-android -c Release
mv ./bin/Release/net6.0-android/android-arm64/com.Microsoft.NetConf2021.Maui-Signed.apk $(Build.SourcesDirectory)/MauiTesting/MauiAndroidPodcast.apk
displayName: Build podcast app for MAUI
displayName: Build MAUI Podcast Android
workingDirectory: $(Build.SourcesDirectory)/dotnet-podcasts/src/Mobile


# This step pulls the product version from the used Microsoft.Maui.dll file properties and saves it for upload with the maui test counter.
# We pull from this file as we did not find another place to reliably get the version information pre or post build.
- powershell: |
Expand All @@ -186,6 +185,20 @@ steps:
displayName: Build MAUI iOS
workingDirectory: $(Build.SourcesDirectory)/MauiTesting

- script: |
chmod -R a+r .
# remove net6.0-maccatalyst to work around https://github.com/dotnet/sdk/issues/21877
cp Microsoft.NetConf2021.Maui.csproj Microsoft.NetConf2021.Maui.csproj.bak
sed -i'' -e 's/net6.0-ios;net6.0-maccatalyst/net6.0-ios/g' Microsoft.NetConf2021.Maui.csproj
Copy link
Contributor

@sblom sblom Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I bet we can come up with a sed line that will survive more classes of csproj file edits. I'll pester you in person.

Copy link
Member Author

@LoopedBard3 LoopedBard3 Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working on this: #67263

Copy link
Member Author

@LoopedBard3 LoopedBard3 Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will merge this, then copy the sed change to this section.


../../../dotnet build ../Web/Components/Podcast.Components.Maui.csproj
../../../dotnet publish Microsoft.NetConf2021.Maui.csproj -bl:MauiiOSPodcast.binlog -f net6.0-ios --self-contained -r ios-arm64 -c Release /p:_RequireCodeSigning=false
mv ./bin/Release/net6.0-ios/ios-arm64/publish/Microsoft.NetConf2021.Maui.ipa ./MauiiOSPodcast.ipa

cp Microsoft.NetConf2021.Maui.csproj.bak Microsoft.NetConf2021.Maui.csproj
displayName: Build MAUI Podcast iOS
workingDirectory: $(Build.SourcesDirectory)/dotnet-podcasts/src/Mobile

- script: |
chmod -R a+r .
../dotnet publish -bl:MauiMacCatalyst.binlog -f net6.0-maccatalyst -c Release
Expand Down Expand Up @@ -214,6 +227,13 @@ steps:
pathtoPublish: $(Build.SourcesDirectory)/MauiTesting/MauiMacCatalyst.binlog
artifactName: ${{ parameters.artifactName }}

- task: PublishBuildArtifacts@1
displayName: 'Publish MauiiOSPodcast binlog'
condition: always()
inputs:
pathtoPublish: $(Build.SourcesDirectory)/dotnet-podcasts/src/Mobile/MauiiOSPodcast.binlog
artifactName: ${{ parameters.artifactName }}

- template: /eng/pipelines/common/upload-artifact-step.yml
parameters:
rootFolder: $(Build.SourcesDirectory)/MauiTesting/MauiAndroidDefault.apk
Expand All @@ -234,7 +254,6 @@ steps:
archiveType: tar
tarCompression: gz


- template: /eng/pipelines/common/upload-artifact-step.yml
parameters:
rootFolder: $(Build.SourcesDirectory)/MauiTesting/MauiiOSDefault.ipa
Expand All @@ -245,6 +264,16 @@ steps:
archiveType: tar
tarCompression: gz

- template: /eng/pipelines/common/upload-artifact-step.yml
parameters:
rootFolder: $(Build.SourcesDirectory)/dotnet-podcasts/src/Mobile/MauiiOSPodcast.ipa
includeRootFolder: true
displayName: Maui iOS Podcast IPA
artifactName: MauiiOSPodcastIPA
archiveExtension: '.tar.gz'
archiveType: tar
tarCompression: gz

- template: /eng/pipelines/common/upload-artifact-step.yml
parameters:
rootFolder: $(Build.SourcesDirectory)/MauiTesting/MauiMacCatalystDefault.app
Expand Down
7 changes: 7 additions & 0 deletions eng/pipelines/coreclr/templates/perf-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,13 @@ jobs:
artifactFileName: 'MauiMacCatalystDefault.tar.gz'
artifactName: 'MauiMacCatalystDefault'
displayName: 'Maui MacCatalyst App'
- template: /eng/pipelines/common/download-artifact-step.yml
parameters:
unpackFolder: $(Build.SourcesDirectory)/MauiiOSPodcastIPA
cleanUnpackFolder: false
artifactFileName: 'MauiiOSPodcastIPA.tar.gz'
artifactName: 'MauiiOSPodcastIPA'
displayName: 'Maui iOS Podcast IPA'

# Create Core_Root
- script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(buildConfig) $(archType) generatelayoutonly $(librariesOverrideArg)
Expand Down
12 changes: 12 additions & 0 deletions eng/testing/performance/ios_scenarios.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,17 @@
<Command>$(Python) test.py sod --scenario-name &quot;%(Identity)&quot;</Command>
<PostCommands>$(Python) post.py</PostCommands>
</HelixWorkItem>
<HelixWorkItem Include="SOD - Maui iOS Podcast IPA Size" Condition="'$(iOSLlvmBuild)' == 'False'">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands>cd $(ScenarioDirectory)mauiios;copy %HELIX_CORRELATION_PAYLOAD%\MauiiOSPodcastIPA\MauiiOSPodcast.ipa .;$(Python) pre.py --name MauiiOSPodcast.ipa</PreCommands>
<Command>$(Python) test.py sod --scenario-name &quot;%(Identity)&quot;</Command>
<PostCommands>$(Python) post.py</PostCommands>
</HelixWorkItem>
<HelixWorkItem Include="SOD - Maui iOS Podcast IPA Size Unzipped" Condition="'$(iOSLlvmBuild)' == 'False'">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands>cd $(ScenarioDirectory)mauiios;copy %HELIX_CORRELATION_PAYLOAD%\MauiiOSPodcastIPA\MauiiOSPodcast.ipa .;$(Python) pre.py --unzip --name MauiiOSPodcast.ipa</PreCommands>
<Command>$(Python) test.py sod --scenario-name &quot;%(Identity)&quot;</Command>
<PostCommands>$(Python) post.py</PostCommands>
</HelixWorkItem>
</ItemGroup>
</Project>
1 change: 1 addition & 0 deletions eng/testing/performance/performance-setup.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,7 @@ if ($iOSMono) {
Copy-Item -path "$SourceDirectory\iosHelloWorld\nollvm" $PayloadDirectory\iosHelloWorld\nollvm -Recurse
Copy-Item -path "$SourceDirectory\MauiiOSDefaultIPA" $PayloadDirectory\MauiiOSDefaultIPA -Recurse
Copy-Item -path "$SourceDirectory\MauiMacCatalystDefault\MauiMacCatalystDefault.app" $PayloadDirectory\MauiMacCatalystDefault -Recurse
Copy-Item -path "$SourceDirectory\MauiiOSPodcastIPA" $PayloadDirectory\MauiiOSPodcastIPA -Recurse
Copy link
Contributor

@sblom sblom Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section of performance-setup seems awfully test case specific for a script with such a general name. Probably naively optimistic of me, but are there any ways we could move move test case specific knowledge closer to the test cases?

Copy link
Contributor

@naricc naricc Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's probably possible to take care of a lot of this with the yml templates, so we have a template that just takes a list of apps, and expands to the various stuff for them.

I was reluctant to try to do that given deadlines and the difficulty in debugging yml templates though. It would be a good thing to tackle in the future.

Copy link
Member Author

@LoopedBard3 LoopedBard3 Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this as an area of improvement: dotnet/performance#2113. This is 100% something we should at least look into.

Copy link
Member

@DrewScoggins DrewScoggins Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, in particular we are impacted by the fact that we only want to build the apps once, but use them in various test cases. That goes against some of the assumptions around how our scenario support was built. We don't need to design the feature in this comment thread, but I think that trying to bring the concept of pre-building dependencies of scenarios into our current python scenario execution code is where we should focus our efforts.

}

$SetupArguments = $SetupArguments -replace $Architecture, 'arm64'
Expand Down