Update SDK to 7.0.100-alpha.1.21480.20

[Tratcher]

Weekly SDK update.

[wtgodbe]

@Tratcher you'll probably need all the changes from #36783

[Tratcher]

Fixes #37151

[Tratcher]

Running tests: dotnet run --no-restore --project RunTests/RunTests.csproj -- --target Microsoft.AspNetCore.WebSockets.Tests.dll --runtime 7.0.0-ci --queue Ubuntu.1804.Amd64.Open --arch x64 --quarantined false --ef 7.0.0-alpha.1.21501.1 --helixTimeout 00:30:00
It was not possible to find any compatible framework version
The framework 'Microsoft.NETCore.App', version '6.0.0-rc.2.21452.2' (x64) was not found.
  - The following frameworks were found:
      7.0.0-alpha.1.21480.1 at [/datadisks/disk1/work/9AA508EB/p/dotnet-cli/shared/Microsoft.NETCore.App]
      7.0.0-alpha.1.21501.1 at [/datadisks/disk1/work/9AA508EB/p/dotnet-cli/shared/Microsoft.NETCore.App]

Helix seems to be stuck on RC2? Any idea where that's specified?

Nevermind, I think it's here:

aspnetcore/eng/helix/content/RunTests/RunTests.csproj

Line 5 in 3b7f281

<TargetFramework>net7.0</TargetFramework>

[Tratcher]

Linux build failure:

Cannot get required symbol SSL_set_alpn_protos from libssl
/mnt/vss/_work/1/s/eng/common/tools.sh: line 475:   139 Aborted                 (core dumped) "$_InitializeBuildTool" "$@"

dotnet/runtime#59431 (comment)

Are our linux agents old?

[dougbu]

New image seems to require a change to our commands:

  /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/xpath_parser.rb:12: warning: Refinements are experimental, and the behavior may change in future versions of Ruby!
  /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/document.rb:12:in `require_relative': /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/parsers/baseparser.rb:515: syntax error, unexpected ',' (SyntaxError)
  /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/parsers/baseparser.rb:542: syntax error, unexpected ','
  /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/parsers/baseparser.rb:673: syntax error, unexpected keyword_end, expecting end-of-input
  	from /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/document.rb:12:in `<top (required)>'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/local/share/gems/gems/fpm-1.13.1/lib/fpm/package/osxpkg.rb:7:in `<top (required)>'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/local/share/gems/gems/fpm-1.13.1/lib/fpm.rb:15:in `<top (required)>'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/local/share/gems/gems/fpm-1.13.1/bin/fpm:4:in `<top (required)>'
  	from /usr/local/bin/fpm:23:in `load'
  	from /usr/local/bin/fpm:23:in `<main>'
/mnt/vss/_work/2/s/src/Installers/Rpm/Directory.Build.targets(80,5): error MSB3073: The command "fpm --verbose --input-type=dir --output-type=rpm --name=aspnetcore-runtime-7.0 --package=/mnt/vss/_work/2/s/artifacts/installers/Release/aspnetcore-runtime-7.0.0-ci-x64.rpm --version=7.0.0 --iteration=0.1.ci --architecture=amd64 --depends="dotnet-runtime-7.0 >= 7.0.0" --rpm-changelog="/mnt/vss/_work/2/s/artifacts/obj/Rpm.Runtime/Release/linux-x64/changelog" --rpm-summary="Microsoft ASP.NET Core 7.0.0-ci Shared Framework" --description="Shared Framework for hosting of Microsoft ASP.NET Core applications. It is open source, cross-platform and is supported by Microsoft. We hope you enjoy using it! If you do, please consider joining the active community of developers that are contributing to the project on GitHub (https://github.com/dotnet/aspnetcore). We happily accept issues and PRs." --maintainer="Microsoft <nugetaspnet@microsoft.com>" --vendor="Microsoft Corporation" --license=MIT --url=https://asp.net --directories="/usr/share/dotnet/shared/Microsoft.AspNetCore.App" "/mnt/vss/_work/2/s/artifacts/obj/SharedFx.Layout/Release/linux-x64/=/usr/share/dotnet/"" exited with code 1. [/mnt/vss/_work/2/s/src/Installers/Rpm/Runtime/Rpm.Runtime.rpmproj]

[Tratcher]

Hmm, the only commas are in the description.
--description="Shared Framework for hosting of Microsoft ASP.NET Core applications. It is open source, cross-platform and is supported by Microsoft. We hope you enjoy using it! If you do, please consider joining the active community of developers that are contributing to the project on GitHub (https://github.com/dotnet/aspnetcore). We happily accept issues and PRs."

We could always try a run-on sentence 😁.

[dougbu]

We could always try a run-on sentence 😁.

Maybe the command supports escaping the comma in some way❔ I'm not sure how to find the Ruby sources but I'm hoping it's more clear when looking at baseparser.rb:673, baseparser.rb:542, or baseparser.rb:515 what it's attempting to parse.

[Tratcher]

Hmm
https://github.com/ruby/rexml/blob/v3.2.5/lib/rexml/parsers/baseparser.rb#L515

I'm not sure it's the command, it looks more like an issue loading the ruby code. Does this image have the right version?

[Tratcher]

ruby/rexml#69
https://www.server-world.info/en/note?os=CentOS_7&p=ruby25

[janvorli]

It is possible to install ruby 2.5 on centos 7 though, from the centos-sclo-rh repo. You's need to execute anything that uses ruby using scl enable rh-ruby25 or run the bash shell that you run your stuff in using that command (scl enable rh-ruby25 bash) or add /etc/profile.d/rh-ruby25.sh script like described here: https://www.server-world.info/en/note?os=CentOS_7&p=ruby25
Would that be feasible?

[TanayParikh]

It is possible to install ruby 2.5 on centos 7 though, from the centos-sclo-rh repo. You's need to execute anything that uses ruby using scl enable rh-ruby25 or run the bash shell that you run your stuff in using that command (scl enable rh-ruby25 bash) or add /etc/profile.d/rh-ruby25.sh script like described here: https://www.server-world.info/en/note?os=CentOS_7&p=ruby25 Would that be feasible?

I believe that'd require updating the dockerfile here with the ruby25 script / instantiation you suggest. This would likely involve creating and publishing a new image with those changes. Would that be possible? cc/ @dougbu ?

[dougbu]

@janvorli

As for using CentOS 8, is this image used to build some native components too?

We use the image to build only our two RPM projects: src/Installers/Rpm/Runtime/Rpm.Runtime.rpmproj and src/Installers/Rpm/TargetingPack/Rpm.TargetingPack.rpmproj. Most of the real work is done in src/Installers/Rpm/Directory.Build.targets. From what I can see, fpm gets run but not much else. See a recent log for example.

If we can move to CentOS 8 without breaking our RHEL 7 compatibility, I guess I'm 🆗 with it. I continue to have nagging doubts about whether the fpm version there produces the right RPMs.

Hmm, I've just loaded the rhel 7 image mcr.microsoft.com/dotnet-buildtools/prereqs:rhel-7-rpmpkg-e1b4a89-20175311035359 and the ruby -v shows ruby 2.0.0p648 (2015-12-16) [x86_64-linux].

That's not a big surprise because that image is for RHEL 7.5 and is 3 or 4 years old. We were using it until it stopped working with the OpenSSL expectations in the SDK. That's why we've been messing w/ the image in the first place.

An updated RHEL 7 or CentOS 7 image containing compatible rexml and ruby versions would be ideal.

scl enable rh-ruby25 bash

I guess we could add this to eng/docker/rhel.Dockerfile. But, wouldn't a working and modern rpmpkg image on mcr be the right way forward❔

[dougbu]

This would likely involve creating and publishing a new image with those changes.

No, that wouldn't be necessary @TanayParikh. the Dockerfile is completely additive; it creates layers on top of the image we download from mcr.

[pranavkm]

@dougbu could you summarize what the next steps are to get this merged?

[dougbu]

@dougbu could you summarize what the next steps are to get this merged?

Sure @pranavkm. We're basically waiting for more information from people like @janvorli @mthalman @NikolaMilosavljevic. The options are

1. Leave this PR as-is and merge it. That would move us from an old RHEL 7 image to a current CentOS 8 image when we execute fpm. A potential downside is changing how (or if) the RPM works on RHEL 7.
2. Change eng/docker/rhel.Dockerfile to include scl enable rh-ruby25 bash and use mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-rpmpkg-20210714125435-9b5bbc2. This avoids the above downside but hacks around an incompatibility w/in the image (which is intended for fpm execution).
3. Wait for a compatible CentOS or RHEL 7 image on mcr i.e. one w/ aligned rexml and ruby versions.

Looking back at this list, I lean toward (2) unless someone tells us my concerns about creating the RPM on a CentOS 8 image and running it on RHEL 7 are unfounded.

[dougbu]

Nice to have: Figure out why the new SDK appears to be incompatible w/ the OpenSSL version on RHEL 7.5. It may not matter because 7.5 is out of support (right @Tratcher❔) but we tend not to break things this way.

[Tratcher]

The original runtime change is discussed in dotnet/runtime#59431 (comment), which requires features from updated versions of OpenSSL. This is a harder requirement than has been enforced in the past.

[pranavkm]

How about we do option (b) and file an issue to mop this up based on a resolution? PR is out for a week and it'd be good to pick up SDK changes because waiting in the past has come to bite us.

[dougbu]

Perhaps a missing /etc/scl/conf folder @pranavkm❔

Step 14/14 : RUN echo -e "#!/bin/bash\nsource /opt/rh/rh-ruby25/enable\nexport X_SCLS=\"`scl enable rh-ruby25 'echo $X_SCLS'`\"\n" >> /etc/profile.d/rh-ruby25.sh
 ---> Running in b2b9d5da7dbc
Unable to open /etc/scl/conf/rh-ruby25!
/bin/sh: /etc/profile.d/rh-ruby25.sh: Permission denied
The command '/bin/sh -c echo -e "#!/bin/bash\nsource /opt/rh/rh-ruby25/enable\nexport X_SCLS=\"`scl enable rh-ruby25 'echo $X_SCLS'`\"\n" >> /etc/profile.d/rh-ruby25.sh' returned a non-zero code: 1
##[error]Bash exited with code '1'.

/fyi I tend to fire up docker in WSL, install an image I'm interested in, then run docker run -it --rm 8455aeb53704 /bin/bash to play around a bit before trying things on the CI. (WSL is a bit of a pain for this because it doesn't use systemd or similar. I always forget to run sudo dockerd, authenticating, then putting the process in the background before doing anything else.)

[janvorli]

You have accidentally removed the

RUN yum --enablerepo=centos-sclo-rh -y install rh-ruby25

in the last iteration

[dougbu]

This is a lot and all the more reason to move to a Docker image with matching fpm and ruby installations soon

[pranavkm]

Yup. @janvorli helped with it. The problem was that fpm uses the ruby in the /usr/bin dir and we had to reset enough state to be have it use the new version. Again, I'd much rather we use this as a stop-gap and merge it (assuming it works this time), than hold out for a compatible image.

[dougbu]

I completely agree on this being a stop gap. My comment amounted to "Ick, let's burn this w/ fire ASAP" 😃

[pranavkm]

#37406 tracks undoing these changes.

[dougbu]

Wahoo❕