Use openblas/clapack on windows static

[JAicewizard]

Fixes #46715

• Changes comply with the maintainer guide.
• SHA512s are updated for each updated download.
• The "supports" clause reflects platforms that may be fixed by this new version.
• Any fixed CI baseline entries are removed from that file.
• Any patches that are no longer applied are deleted from the port's directory.
• The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
• Only one version is added to each modified port's versions file.

[JAicewizard]

This simplifies the blas support table a lot as well, as now only on macos the apple framework is used. No changes on the ports individual port side was neccesary

[JAicewizard]

New table to update #24327 (review) if this gets merged:

	blas before	after	lapack before	after
windows	openblas	openblas	lapack-reference	lapack-reference
windows-static	lapack-reference[cblas]	openblas	lapack-reference	clapack
uwp	openblas	openblas	clapack	clapack
osx	accalerate framework	accalerate framework	accelerate framework	accelerate framework
linux	openblas	openblas	lapack-reference	lapack-reference

(Code for easy copy-paste for others:)

|                | blas before             | after                | lapack before        | after                |
|----------------|-------------------------|----------------------|----------------------|----------------------|
| windows        | openblas                | openblas             | lapack-reference     | lapack-reference     |
| windows-static | lapack-reference[cblas] | **openblas**         | lapack-reference     | **clapack**          |
| uwp            | openblas                | openblas             | clapack              | clapack              |
| osx            | accalerate framework    | accalerate framework | accelerate framework | accelerate framework |
| linux          | openblas                | openblas             | lapack-reference     | lapack-reference     |

[BillyONeal]

The blas/lapack monster strikes again. The last time which backends we choose was litigated was back in #24327 (see very similar table I wrote then: #24327 (review) ) where @Neumann-A wrote:

The problem in static builds is that lapack-reference cannot link openblas due to the current choice of the Fortran compiler being gfortran. This leads to the situation where both lapack-reference and openblas would provide blas symbols. Since lapack typically depends on blas and thus comes earlier on the link line only symbols from lapack-reference would be used any way. As such the switch just deals with the inconsistency and make lapack-reference the only symbol provider. The forcing of the blas vendor also helps with that.

Can you explain if something has changed about this situation and why we aren't going to immediately get another PR trying to put things back?

[BillyONeal]

(I believe the intent was to use lapack-reference as lapack whenever possible over clapack for perf reasons)

[JAicewizard]

Can you explain if something has changed about this situation and why we aren't going to immediately get another PR trying to put things back?

Nothing has changed, this is still the same situation. The real issue is that, because we use gfortran (or just, any mingw fortran compiler, as microsoft doesn't have one), we cannot link lapack-reference statically to anything on windows non-mingw. This is also printed by vcpkg (somewhere hidden in the long output), when trying to use lapack-reference on windows-static.

So nothing has really changed, but windows-static has just always been "broken", working but not statically.

Clapack might be a little slower, I don't know as I cannot find much online, however the ability to propperly support windows-static seems like a much better alternative. Since clapack is also just a port of lapack, the difference should not be as major as with other lapack implementations. On top of that lapack mentions that using the builtin blas implementation is very slow, so using lapack-reference[cblas] seems like a last-resort in general.

However I am not a performance geek wrt lapack/blas, I am merely trying to get a static binary on windows.

If you need some performance numbers on lapack vs clapack, I can try and get them, but it would likely be on linux.

[JAicewizard]

Ok there seems to be a bit of an issue.

arpack-ng:

-- VCPKG_CRT_LINKAGE linkage for arpack-ng using vcpkg's internal gfortran cannot be static due to linking against MinGW libraries. Forcing dynamic CRT linkage
-- VCPKG_LIBRARY_LINKAGE linkage for arpack-ng using vcpkg's internal gfortran cannot be static due to linking against MinGW libraries. Forcing dynamic library linkage

Options: disable this for windows-static, or allow add a force-reference-cblas option

coin-or-ipopt:

This seems to be a a general incompatibility with openblas, as it also doesn't build on x64-linux, this just seems to be a case of not supporting openblas and requirering the reference blas implementation. Again, we can disable this for windows-static as it is a bad package, or add a force-reference-blas flag

openimageio:

No idea, seems to work locally (on windows-static). Weird that it can't find f2c, as it should be present. Also doesn't depend on lapack, not sure why it was build in CI or gives an issue related to lapack.

openmvs/paraview:

Seems to be the same issue as openimageio, havn't verified that it builds locally yet

[dg0yt]

(I believe the intent was to use lapack-reference as lapack whenever possible over clapack for perf reasons)

I would like to note that

• clapack is "3.2.1", lapack-reference is "3.12.1".
  openblas offers to build an updated clapack "3.4.2" or lapack-reference aka lapack-netlib "3.12.0"
• building Fortran on Windows with gfortran in vcpkg means building C and C++ code with mingw gcc instead of MSVC ATM. (CMake could link gfortran libs with MSVC libs, but the packages don't seem to make use of it.)

I have difficulties with the fact the default LAPACK with openblas isn't the lapack from openblas...

[Neumann-A]

clapack is outdated.
If you want x64-windows-static to work use a toolchain/environment which has a Windows Fortran Compiler setup. This can be flang, flang-classic, Intel Fortran, ....., I have done every possible combination in different PRs.

[Neumann-A]

I created llvm/llvm-project#153618 so that LLVM might package flang with the next release on windows. So that vcpkg can use flang instead of gfortran in the future.

[JAicewizard]

I agree that this would be a much, much better solution, I am fine with keeping my fork alive for private use until vcpkg switches over to flang. If that is not an option, I would still suggest this PR gets merged (preferably disabling the broken ports on windows-static) as currently blas/lapack on windows-static is just broken

[BillyONeal]

@ras0219-msft @vicroms and I discussed today. I wrote this to frame the conversation:

BLAS/LAPACK change to make windows-static actually be static. The contributor has written a very high quality writeup which convinces me of the correctness of their change. It does break some things though. I think we should ci.baseline.txt the broken stuff. I don't want "supports" as the windows-static incompatibilities may be resolved by someone using an overlay-port of blas/lapack that makes different choices.

@ras0219-msft asked "how does arpack-ng:arm64-windows work now?" answer:

PS C:\Dev\vcpkg> .\vcpkg.exe install arpack-ng:arm64-windows --dry-run
Computing installation plan...
vcpkg-gfortran is only supported on 'windows & !arm & !uwp', which does not match arm64-windows. This usually means that there are known build failures, or runtime problems, when building other platforms. To ignore this and attempt to build vcpkg-gfortran anyway, rerun vcpkg with `--allow-unsupported`.

@JAicewizard Please go ahead and make the change for windows-static as originally proposed and add entries to ci.baseline.txt / ci.feature.baseline.txt for broken pieces.

[JAicewizard]

That is much appreciated! I think I fixed all the issues related to this PR. I have no idea why arm64_osx is failing, that seems to be an unrelated issue(?)

[BillyONeal]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[BillyONeal]

Disabled CPPMAC_ARM64_02 and restarted...

[BillyONeal]

This passed for me locally. Are we sure it needs to be disabled?

[JAicewizard]

I will rerun with these enabled again, but if I disabled them it must have been broken in one way or another, I think it was some kind of cascade issue

[BillyONeal]

Something in 1ES Hosted Pools for our non-Windows pool is broken :(. Will keep you posted.

[BillyONeal]

I rechecked the ones below and they all passed for me. (arpack-ng and coin-or-ipopt are indeed broken)

PS C:\Dev\vcpkg> .\vcpkg.exe install --overlay-ports .\scripts\test_ports arpack-ng:x64-windows-static-md arpack-ng:x64-windows-static coin-or-ipopt:x64-windows-static-md coin-or-ipopt:x64-windows-static openmvs:x64-windows-static-md openmvs:x64-windows-static paraview:x64-windows-static-md paraview:x64-windows-static --keep-going
[...]
RESULTS
  openmvs:x64-windows-static: SUCCEEDED: 0 ns
  openmvs:x64-windows-static-md: SUCCEEDED: 0 ns
  paraview:x64-windows-static: SUCCEEDED: 0 ns
  paraview:x64-windows-static-md: SUCCEEDED: 0 ns
  arpack-ng:x64-windows-static: BUILD_FAILED: 9 s
  arpack-ng:x64-windows-static-md: BUILD_FAILED: 6.2 s
  coin-or-ipopt:x64-windows-static: BUILD_FAILED: 43 s
  coin-or-ipopt:x64-windows-static-md: BUILD_FAILED: 42 s

SUMMARY FOR x64-windows-static
  SUCCEEDED: 2
  BUILD_FAILED: 2
SUMMARY FOR x64-windows-static-md
  SUCCEEDED: 2
  BUILD_FAILED: 2

[JAicewizard]

I removed the fails, lmk if its good. Some of these were still failing in CI, but it seems untrustworthy

[dg0yt]

FTR drive-by fix to clapack link lib export in #47060.

[BillyONeal]

Something in 1ES Hosted Pools for our non-Windows pool is broken :(. Will keep you posted.

Hopefully fixed now! But lab will be busy catching up....

[BillyONeal]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[BillyONeal]

Sorry for the delay! Thanks.

[JAicewizard]

No worries! I understand this is a complex PR to merge, and I was slow myself as well. Thank you for merging this!! it is much appreciated