NuGet pack ignores nuspec versions [unwanted version normalization?]

[timotei]

Hello,

It seems latest NuGet pack ignores the version set in the nuspec (or via the -Version override flag) and the dependencies' versions, by normalising them, even if nobody asked for it :(

It seems this was "fixed" for this #2039, but to me this seems a super counter-intuitive change, that breaks (at least for us) the daily workflows.

For the meantime, we had to do a workaround to "fix" the versions in the generated nupkg. I'm hoping that this can be fixed either by removing this weird logic, or allowing the nuget pack command to generate nuget package which contains the same versions as in the nuspec.

I'm using NuGet Version: 3.4.4.1321.

With this nuspec:

<?xml version="1.0" encoding="UTF-8"?><package xmlns="http://schemas.microsoft.com/packaging/2011/08/nuspec.xsd">
  <metadata>
    <authors>author</authors>
    <version>1.9.0.0-SNAPSHOT</version>
    <id>mylib</id>
    <dependencies>
      <dependency id="dependency" version="3.16.0.0"/>
    </dependencies>
    <description>mylib</description>
  </metadata>
</package>

executing nuget pack mylib.nuspec -Version 1.9.0.0-SNAPSHOT generates this .nuspec:

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2013/05/nuspec.xsd">
  <metadata>
    <id>mylib</id>
    <version>1.9.0-SNAPSHOT</version>
    <authors>author</authors>
    <owners>author</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>mylib</description>
    <dependencies>
      <dependency id="dependency" version="3.16.0" />
    </dependencies>
  </metadata>
</package>

As we can see:

• Version is 1.9.0-SNAPSHOT instead of 1.9.0.0-SNAPSHOT (even if we specifically overrode it via the command line) - same happens with a non-SNAPSHOT version
• Even worse, the dependency version is 3.16.0 instead of 3.16.0.0
• Funnily enough, the filename has the full version: mylib.1.9.0.0-SNAPSHOT.nupkg

[emgarten]

@timotei would you share more about your scenarios here? Are you expecting the version string to be an exact match as part of other scripts or tools?

[timotei]

Well, we have a ton of internal libraries that are used throughout the end products (private NuGet repo based on Artifactory). Some scripts/tools generate them directly via the nuget pack, others are using some more legacy build systems and we managed to make some bridge between them (not possible/feasible to migrate them to the newer NuGet-based process).

The issue comes from the fact that newer NuGet automatically normalizes the version, making the interop between the two types of projects hard to almost impossible. Basically, we have interop between Java and .NET libraries (via IKVM), and thus we have a specific format for the versions, which we must respect.

That's why, it's very important (for us and I think in general) that tools that generate stuff, do it in a expected way, based on the .nuspec, because that impacts the entire dependency tree. So, I'd expect the nuget pack step not to change anything that's already set in the nuspec. Or, at least, provide a way to skip that behavior.

[emgarten]

I'd like to understand how this is working, are your scripts resolving nuget dependencies without nuget? Does IKVM consume nupkgs directly? Which parts of your build process read the nupkgs without nuget?

[timotei]

Yeah, up until NuGet 3.4, we had in place a simple mechanism which would translate 1-to-1 from Gradle dependencies, to NuGet packages (which are basically IKVM-compiled assemblies - Java to .NET). We were able to create the packages.config file with the mapped dependencies + versions, then do a nuget restore, use the restored deps in a custom processing step which does the final .NET compilation.

Once NuGet 3.4+ went in, we found the issue that the versions wouldn't be mapped properly, all depending on whether the restored NuGet packages were created with NuGet 3.4+ or not. The thing is, nuget places the restored packages in folder with the full version, not the normalized one, which means, if a package was created with NuGet prior to 3.4 it will have 1.0.0.0, while with NuGet 3.4+, it will be 1.0.0. This means there's suddenly no reliable/deterministic rule for the location of the packages. This issue comes from the fact that nuget pack will normalize the generated version.

[emgarten]

The thing is, nuget places the restored packages in folder with the full version, not the normalized one

This has been fixed in 3.5.0, you can find it here: https://dist.nuget.org/win-x86-commandline/v3.5.0-beta2/NuGet.exe

there's suddenly no reliable/deterministic rule for the location of the packages

With normalization there is only one possible string, making this deterministic now. Paths can be determined upfront without opening every nupkg to check if it is the expected version.

Here is how it works:

1. Versions have three parts to match SemVer
2. If the version contains a 4th non-zero digit, then it is also added since NuGet allowed 4 part versions at the start.
3. Leading zeros are always removed

Examples:
1.0.0.1 -> 1.0.0.1
1.0.0.01 -> 1.0.0.1
1.0 -> 1.0.0
1.0.0.0 -> 1.0.0
1.0.0 -> 1.0.0

[timotei]

Hmm, yeah, but AFAICT, the 3 or 4 digits folder name will depend whether the NuGet package was generated with NuGet <3.4 or NuGet 3.4+, no?

About the normalization part, isn't possible to make nuget pack not normalize the version if explicitly not wanting? Or is the whole 3.4+ Normalized Versions Saga about normalizing the behavior (pun intended) of all NuGet behavior so it uses normalized versions everywhere? (pack + install + restore)

How I see this is, that NuGet is kinda forcing people to use normalized versions even if they don't want it, right :) ?

[emgarten]

Normalizing solves many issues within NuGet. I think your scenario is valid, but less common than the problems normalizing fixes. You can work around this problem by finding the normalized version from the package after packing and using that for your mappings instead of the -Version input. You could also modify the nuspec in the package after pack to un-normalize it.

[timotei]

Does this mean that a PR that enables the NuGet Pack to generate the NuGet package "as is" from the NuSpec (without normalizing versions) would not be accepted into the client?

IIRC, there was once a warning when doing a pack for 4 digits (that the version doesn't satisfy SemVer 2?), but that warning does not show up now, and instead it directly normalizes the version (not sure why normalize it only when ending in '0', since SemVer2 is 3 digits only nevertheless the last 4th digit)

[rrelyea]

@timotei - yes, 3.4.* does this behavior during pack. Earlier versions didn't.
We'd prefer to not add more command line options to not normalize.
Is it possible for you to adapt your mapping file to understand ".0" as the 4th version number will be dropped? We'd prefer to keep everybody normalizing.

[timotei]

Well, in the end we managed to get a post-NuGet pack step that denormalizes the version, to keep the previous behavior. So far it seems to work.

But this is not a very satisfactory solution :( I'd rather have NuGet not change things I specifically set in the nuspec.

[slaneyrw]

This is now causing problems downstream as tools now start upgrading their nuget versions. Octopack now uses 3.4.x which is dropping the 4th part of the version number if it's zero, and breaking templated builds in our TeamCity. Please consider command line options

[emgarten]

@slaneyrw can you explain how TeamCity uses this? Which part of the build breaks due to this change?