Auto-update submodules to the commits used in latest ProdCon build

[dagood]

To try it out, use build.cmd /t:UpdateDependencies to update based on the ProdConCurrentRef source of truth (e.g. if you want to pin to a specific ProdCon build) and build.cmd /t:UpdateToRemoteDependencies to update to the latest manifest on dotnet/versions master.

Uses new updater and dependency info from dotnet/buildtools#1968.

There's some metadata checked into the project files that this updater won't change. I have a followup issue for that: #370.

#342

Also adds a BuildTools updater: #270

[dagood]

For context, this points to https://github.com/dotnet/versions/blob/7daa2e55de668c4c8ee832059aab04917153881e/build-info/dotnet/product/cli/release/2.1-preview1/build.xml#L2938-L2953.

[dagood]

I forgot to actually update BuildTools here, and that this branch pulls BuildTools release/2.1. Submitted dotnet/buildtools#1975 to port the updater to 2.1 BuildTools, then I'll update this PR.

[eerhardt]

I'm not sure I understand why you added the condition. You are adding the files, so they will always exists, right?

[dagood]

Kinda described in the commit message:

There's no need for the BfS tarball to include these files: it doesn't need to run repo-global auto-updates like submodule update.

The project files that get into the tarball are manually listed here:

source-build/build-source-tarball.sh

Lines 41 to 49 in e2ecb4c

	cp $SCRIPT_ROOT/build.proj $TARBALL_ROOT/
	cp $SCRIPT_ROOT/dir.props $TARBALL_ROOT/
	cp $SCRIPT_ROOT/DotnetCLIVersion.txt $TARBALL_ROOT/
	cp -r $SCRIPT_ROOT/keys $TARBALL_ROOT/
	cp -r $SCRIPT_ROOT/patches $TARBALL_ROOT/
	cp -r $SCRIPT_ROOT/scripts $TARBALL_ROOT/
	cp -r $SCRIPT_ROOT/src $TARBALL_ROOT/
	cp -r $SCRIPT_ROOT/repos $TARBALL_ROOT/
	cp -r $SCRIPT_ROOT/tools-local $TARBALL_ROOT/

I figure things that aren't used in the tarball build aren't included, and there's no need to add these two files to that list.

@crummel is that the intention? Does it need to be so explicit?

[crummel]

I think we want to keep the tarball as minimal as possible so if the workaround isn't causing too much extra work I think we'd prefer to keep it out. If it's causing issues we can include the new files.

[dagood]

The only concern I have is that someone could add something to dependencies.props/targets and expect it to be set during all source-build builds. Not sure how easy that would be to debug depending on what's added.

Keeping it out of the tarball is a bit of complexity that I'm not sure we need. But maintaining that list of files is also annoying... would *.proj, *.props, *.targets at the root be ok?

[crummel]

That sounds fine to me.

[dagood]

Look like the buildtools upgrade broke the build in general. Lots of:

error MSB4064: The "OverrideToolHost" parameter is not supported by the "Csc" task. Verify the parameter exists on the task, and it is a settable public instance property.
error MSB4063: The "Csc" task could not be initialized with its input parameters.

Makes sense: BuildTools merged master into release/2.1 for preview2. Similar merges have also happened in the submodule repos, but we have the old preview1 commits checked out.

I think the cleanest way to get this feature in is to update to preview2 commits first. Getting it working should involve updating BuildTools, so this PR wouldn't need to handle upgrading BuildTools over the preview1-preview2 border.

[dagood]

I talked with @eerhardt and @weshaggard about the CI errors in dotnet/standard. The plan is to update standard's release/2.0.0 branch to use a compatible BuildTools. @eerhardt pointed out that the change to use the CLI's copy of MSBuild rather than the one from BuildTools is probably what's breaking--I'm working on fixing and making a dotnet/standard PR.

What I was talking about above about 2.1-preview1 vs. 2.1-preview2 doesn't really apply in this case, since we're still building dotnet/standard 2.0.

[dagood]

@dseefeld @crummel PTAL