Automate integrations and publication?

[dsyme]

The current way FSharp.Compiler.Service releases are published is to

1. integrate Microsoft/visualfsharp --> this repo, deleting irrelevant files in vsintegration, setup, src\fsharp\FSharp.Core and elsewhere
2. bump the version number in fcs\RELEASE_NOTES.md and fcs\fcs.props,
3. submit the PR
4. tag the commit
5. fetch the packages from AppVeyor CI once built and push them to nuget.org

This is fairly painless but takes a bit of time

The question I'm asking here is whether we can automate this process a bit more.

@forki @Krzysztof-Cieslak I know you're inteterested in this and have worked a lot on the automation of CI for fcs in Microsoft/visualfsharp

As an aside, @cartermp @KevinRansom, @brettfo and myself had a discussion about whether to publish official nuget packages for FCS from this repo (fsharp/FSharp.Compiler.Service) or whether we would propose to build and publish the component from Microsoft/visualfsharp. The consensus is that the current situation where FSharp.Compiler.Service is a component published by the F# community from this repo is appropriate at this stage - among other things the Visual F# Tools don't even use this component at the moment (they have a private build of it). There are also a number of pros and cons - some about engineering efficiency and some about perception. From the perspective of Microsoft/visualfsharp FCS is like the Mono releases of F# - a crucial set of acceptance tests and an important scenario supported by packages published from other repos, but not a release made from that repo.

[cartermp]

Something I'll add (which you can see tracked here: dotnet/fsharp#3819) is that this business of VisualFSharp having its own, private copy of FCS and calling into it differently is something we're very motivated to change. There are a two primaryreasons for this:

1. It's accomplished via IVTs, which are a gloriously awful hack that continually bite us in the ass any time we need to change our internal build into Visual Studio. We're constantly nagged by it as a problem. In fact, on a day where I set up a meeting to discuss this (and other things) with our engineering lead, we literally had a bug that was the result of not correctly accounting for IVTs. How appropriate.
2. Other editors must conform to the FCS public API as it is today: Ionide, Rider, Vim, Emacs, etc. The odd one out here is Visual Studio. And yet, the typical flow of code is that it comes into the VisualFSharp repository first, and is then migrated back here by @dsyme. Visual Studio must consume FCS in the same manner that other editors do. Not only does this make things easier for everyone, but it helps push editor tooling for F# towards memetic independence.

As we formalize what we're going to be working on with our engineering director, we're going to treat the work I mention as something we're to be held accountable for. Once complete, it does bring up the questions of how we ultimately consume the language service, where we publish it from, and what changes we can collaborate with the community on to arrive at a solution which makes sense and results in more contributions from folks who use different editors.

[dsyme]

@cartermp That sounds great. It would be excellent to get the Visual F# Tools on the same engineering basis as other tools.

It's accomplished via IVTs, which are a gloriously awful hack

FWIW I feel IVTs to a private DLL (shipped alongside the components that have special access) aren't that bad, in contrast to IVTs to separately versioned/shipped/built DLLs. But I still agree that it would be much better to get on an equal basis.

[enricosada]

@dsyme can be this ok? i can write that.

Automatically dunno, but what i can do is:

• checkout master
• create a branch for integration
• fetch latest vf# repo master (or specific branch)
• merge from vf# to branch
• delete blacklisted directories (so new vf# files are not added)
• bump the version number in fcs\RELEASE_NOTES.md and fcs\fcs.props,
• stop for manual check
• send a PR

after PR is merged:

• push a tag:
• appveyor on tag create the package, and push it

[dsyme]

@enricosada Yes, that's it. Sometimes there are conflicts in README.md and fcs\RELEASE_NOTES.md

[KevinRansom]

We are perfectly happy to take over the responsibility of publishing the FCS. In a similar manner to how we publish FSharp.Core.nuget these days.

Having a set of APIs private in the VS tooling and public in the Ionide tooling is silly and historical. Since we are probably not going to be any happier about breaking Ionide than we are about breaking VS.

Publishing FCS builds from this repo is not, however, going to be an accidental by-product of Don's work-flow. If VF# tools is going to be how it's published and responsible for it then that's fine. Otherwise how we are currently set up is also fine.

Kevin

[dsyme]

Re IVTs - yes, I think we all agree these should be removed.

Re publishing - yes, the current setup is fine. It is better that the community continue to publish the component from fsharp/FSharp.Compiler.Service. Perhaps we could reconsider after the use of IVTs is removed in visualfsharp, and the visualfsharp repo is split. However other factors like docs, samples and the community-provided Fable bootstrap in #688 are significant too.

Publishing FCS builds from this repo is not going to be an accidental by-product ...

Agreed, though I think you mean "Publishing FCS builds from Microsoft/visualfsharp is not going to be accidental by-product..."