Update shared framework to target netcoreapp1.1

[weshaggard]

@ericstj can you please have a look Microsoft.NETCore.App changes. I essentially copied all the dependencies from the what is in release/1.0.0 branch which we will need to keep updated with whatever versions we bump in that branch. I was considering using your harvesting but I didn't figure it would map easily in this repo setup for meta-packages.

@dagood this PR replaces #460.

@gkhanna79 @mellinoe could you guys please take a look.

[weshaggard]

@mellinoe what are our options to keep the new distro's working as well as picking up the latest nuget libraries as part of the CLI to enable correct restore due to the framework mappings.

[mellinoe]

• Set up the CLI to build for that channel; it is only set up to build in the rel/1.0.0-preview2.1 branch right now.

Or

• Manually modify the bootstrap CLIs to support the new functionality. And then upload them to the appropriate location in Azure.

[gkhanna79]

Excluding comments from @mellinoe about CLI download for the new repos, it seems reasonable to me.

[weshaggard]

Set up the CLI to build for that channel; it is only set up to build in the rel/1.0.0-preview2.1 branch right now.

I think that we should setup those builds in the preview3 channel. @livarcocc is anyone working on enabling builds for the new distro's? If not what do we need to do to make that happen as without it we will not be able to update the shared framework in master.

[ericstj]

Code changes look OK, but it looks like you'll need more changes to get the CI to pass.

[weshaggard]

https://ci.dot.net/job/dotnet_core-setup/job/master/job/release_ubuntu_x64_prtest/57/console

/opt/code/artifacts/ubuntu.14.04-x64/tools/independent/RuntimeGraphGenerator/RuntimeGraphGenerator "--project" "/opt/code/artifacts/ubuntu.14.04-x64/intermediate/sharedFramework/framework" "--deps" "/opt/code/artifacts/ubuntu.14.04-x64/intermediate/sharedFrameworkPublish/shared/Microsoft.NETCore.App/1.2.0-beta-001014-00/Microsoft.NETCore.App.deps.json" "linux"
09:00:10 System.IO.FileNotFoundException: Could not find file '/opt/code/.nuget/packages/Microsoft.NETCore.DotNetHost/1.2.0-beta-001014-00/runtime.json'.
09:00:10 File name: '/opt/code/.nuget/packages/Microsoft.NETCore.DotNetHost/1.2.0-beta-001014-00/runtime.json'
09:00:10    at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
09:00:10    at Interop.CheckIo[TSafeHandle](TSafeHandle handle, String path, Boolean isDirectory, Func`2 errorRewriter)
09:00:10    at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
09:00:10    at System.IO.UnixFileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, FileStream parent)
09:00:10    at System.IO.UnixFileSystem.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, FileStream parent)
09:00:10    at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
09:00:10    at NuGet.RuntimeModel.JsonRuntimeFormat.ReadRuntimeGraph(String filePath)
09:00:10    at Microsoft.DotNet.ProjectModel.RuntimeGraphManager.Collect(IEnumerable`1 exports)
09:00:10    at RuntimeGraphGenerator.Program.Main(String[] args)

@schellap any ideas why on the linux builds we hit that failure? Looking at https://github.com/dotnet/core-setup/blob/master/tools/independent/RuntimeGraphGenerator/RuntimeGraphManager.cs#L22 it should only ever try to find that runtime.json file if it was in the deps.json file which I would expect it to only be in there if it were in the package. I'm kind of at a lose as to what might cause this error.

[gkhanna79]

@weshaggard I am referring to https://github.com/dotnet/core-setup/issues/456

[gkhanna79]

@mellinoe has been helping prime the CLI builds for new distros for which official CLI builds are not available and should be able to help prime them for the build you want.

[weshaggard]

@weshaggard I am referring to #456

That is a similar fix but it is independent, I just filed PR #464 for that.

[mellinoe]

I think that we should setup those builds in the preview3 channel

I think that this is the rel/1.0.0 branch in CLI (I can never keep the branches straight). Since that one is up to date and using new packages, I'm hoping it isn't a lot of work to enable the build there. I can take a crack at it in a little bit. Hopefully it is just:

• Add the TFM to a few code files which do some rid-moniker mappings
• Add the TFM to all the standalone test projects
• Add the docker image from this repo
• Clone an official build definition
• [ Add to CI ]

[gkhanna79]

@schellap PTAL as well.

[mellinoe]

It looks like updating the rel/1.0.0 branch of CLI is going to be quite a bit more complicated than before, because we are now always downloading the "latest" version of CLI there, and that version seems to be controlled by a single text file, rather than one-per-distro. So even if we uploaded a bootstrap CLI for that version, it would become out-of-date once a new "latest.version" file was uploaded.

[gkhanna79]

@mellinoe Are you sure that master of CLI, which consumes master of Core-Setup, is expected to be in release/1.0.0, as opposed to Preview3?

@livarcocc @piotrpMSFT Can you please chime in as well? We need to get the CLI dependency sorted soon not only yo unblock this PR, but also the PGO work of PErf team and #465.

[mellinoe]

@mellinoe Are you sure that master of CLI, which consumes master of Core-Setup, is expected to be in release/1.0.0, as opposed to Preview3?

I'm not sure I understand what you're asking. I thought that the "preview3" version listed above (in the "Sdk" channel) comes from the rel/1.0.0 branch of CLI. But that may be incorrect; I have a very hard time trying to keep those versions lined up in my head.

[gkhanna79]

I think we need to do two things here:

1. Get changes from @weshaggard clean for the distros for which CLI is present in the Preview3 channel. This means fixing the following build failure:

22:41:22 �EC       >]��  � /mnt/resource/j/workspace/dotnet_core-setup/master/debug_centos7.1_x64_prtest/artifacts/centos.7-x64/tools/independent/RuntimeGraphGenerator/RuntimeGraphGenerator "--project" "/mnt/resource/j/workspace/dotnet_core-setup/master/debug_centos7.1_x64_prtest/artifacts/centos.7-x64/intermediate/sharedFramework/framework" "--deps" "/mnt/resource/j/workspace/dotnet_core-setup/master/debug_centos7.1_x64_prtest/artifacts/centos.7-x64/intermediate/sharedFrameworkPublish/shared/Microsoft.NETCore.App/1.2.0-beta-001014-00/Microsoft.NETCore.App.deps.json" "linux"
22:41:23 System.IO.FileNotFoundException: Could not find file '/mnt/resource/j/workspace/dotnet_core-setup/master/debug_centos7.1_x64_prtest/.nuget/packages/Microsoft.NETCore.DotNetHost/1.2.0-beta-001014-00/runtime.json'.
22:41:23 File name: '/mnt/resource/j/workspace/dotnet_core-setup/master/debug_centos7.1_x64_prtest/.nuget/packages/Microsoft.NETCore.DotNetHost/1.2.0-beta-001014-00/runtime.json'
22:41:23    at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
22:41:23    at Interop.CheckIo[TSafeHandle](TSafeHandle handle, String path, Boolean isDirectory, Func`2 errorRewriter)
22:41:23    at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
22:41:23    at System.IO.UnixFileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, FileStream parent)
22:41:23    at System.IO.UnixFileSystem.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, FileStream parent)
22:41:23    at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
22:41:23    at NuGet.RuntimeModel.JsonRuntimeFormat.ReadRuntimeGraph(String filePath)
22:41:23    at Microsoft.DotNet.ProjectModel.RuntimeGraphManager.Collect(IEnumerable`1 exports)
22:41:23    at RuntimeGraphGenerator.Program.Main(String[] args)

@schellap Can you please look into why this is happening for the host package?

2. Get Preview3 CLI in place so that it is always available for OpenSuse 42.1 and Ubuntu 16.10. @livarcocc @piotrpMSFT Can you please help look into bringing that up?

Since both these issues are blocking the adoption of master payloads (from CoreCLR and CoreFX), @schellap @livarcocc @piotrpMSFT please do prioritize looking into this.

[gkhanna79]

I am able to repro this locally on Linux Mint

[gkhanna79]

Here is the suspect problem.

The new CLI is bringing the new nuget which creates folders (say, for a package restore operation) in lowercase. Later, the RuntimeGraphGenerator code is getting the path in CamelCasing. As a result, on Linux OS where file system access is case-sensitive, the mismatch happens and file is not found.

I will keep this updated as I look into a possible solution.

CC @joelverhagen

[gkhanna79]

@weshaggard I have a local fix for the issue above. This affects crossgen of shared FX as well, so I am fixing few more places. Once I am done, I will push an update into your branch.

[gkhanna79]

I have fixed the lower-casing issues that broke the above builds. However, now I see something else:

gkhanna79@gkhanna-lm18 ~/GH/wesh/core-setup $  /home/gkhanna79/GH/wesh/core-setup/artifacts/ubuntu.16.04-x64/intermediate/sharedFrameworkPublish/dotnet "exec" "--runtimeconfig" "/home/gkhanna79/GH/wesh/core-setup/artifacts/ubuntu.16.04-x64/tests/artifacts/0/PortableTestApp/bin/PortableTestApp.runtimeconfig.json" "--depsfile" "/home/gkhanna79/GH/wesh/core-setup/artifacts/ubuntu.16.04-x64/tests/artifacts/0/PortableTestApp/bin/PortableTestApp.deps.json" "--additionalProbingPath" "/home/gkhanna79/GH/wesh/core-setup/test/HostActivationTests/../../.nuget/packages" "/home/gkhanna79/GH/wesh/core-setup/test/HostActivationTests/../../.nuget/packages/dotnet-test-xunit/1.0.0-rc2-192208-24/lib/netcoreapp1.0/dotnet-test-xunit.dll" "/home/gkhanna79/GH/wesh/core-setup/artifacts/ubuntu.16.04-x64/tests/artifacts/0/PortableTestApp/bin/PortableTestApp.dll"
The specified framework 'Microsoft.NETCore.App', version '1.2.0-beta-001015-00' was not found.
  - Check application dependencies and target a framework version installed at:
      /home/gkhanna79/GH/wesh/core-setup/artifacts/ubuntu.16.04-x64/intermediate/sharedFrameworkPublish/shared/Microsoft.NETCore.App
  - The following versions are installed:
      1.2.0-beta-001011-00
  - Alternatively, install the framework version '1.2.0-beta-001015-00'.

As can be seen above, the tests (under TestAssets folder) are being published with runtimeconfig.json that reference "1015" as the commit count, while M.N.App is built with version "1011". And I have confirmed that the commit count is, indeed, 1011 in the branch.

@weshaggard Why the tests are getting published with this version? I dont see anything in your change that could result in this? Could this be a CLI bug @livarcocc ?

[gkhanna79]

I was not able to push changes to your branch @weshaggard, so I have created a new PR with my fixes for the various issues at #524.

The last issue I am running into is the mismatch of SharedFW build number above.

[weshaggard]

closing in favor of #524

[gkhanna79]

Closing in favor of #524