`test_plane_stress` failures on certain OS

[cjwgriesel]

FAILED tests/unit/test_sctfcoil.py::test_plane_stress[planestressparam0] - assert array([-2.442...61051384e+04]) == approx([0.0 ±...48 ± 3.6e-02])
FAILED tests/unit/test_sctfcoil.py::test_plane_stress[planestressparam1] - assert array([-2.442...61051384e+04]) == approx([0.0 ±...48 ± 3.6e-02])
================================================================= 2 failed, 426 passed, 1 skipped, 18 warnings in 1232.48s (0:20:32) ===<!--- Please consider whether this issue will be better suited to the Discussion section of this GitHub repository --->

Additional info:-

  assert sigr == pytest.approx(planestressparam.expected_sigr)

E assert array([-2.442...61051384e+04]) == approx([0.0 ±...48 ± 3.6e-02])
E comparison failed. Mismatched elements: 1 / 300:
E Max absolute difference: 2.4424334654893232e-08
E Max relative difference: 1.0
E Index | Obtained | Expected
E 0 | -2.4424334654893232e-08 | 0.0 ± 1.0e-12

/home/ir-grie1/conda/Unruns/process_ADev1/PROCESS/tests/unit/test_sctfcoil.py:7683: AssertionError
========================================================================================== warnings summary ==========================================================================================
:241
:241: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 16 from C header, got 96 from PyObject

tests/integration/test_blanket_library_int.py::test_hydraulic_diameter
/home/ir-grie1/conda/Unruns/process_ADev1/PROCESS/tests/conftest.py:124: UserWarning:
You are running the PROCESS test suite on an outdated system.
This can cause floating point rounding errors in regression tests.

There is also a warning about running PROCESS on an outdated system so maybe that just needs updating????

From the output above when running the pytest on CSD3 all passes apart from two tests (test_plane_stress). Possibly might be fixed by updating the system? But not sure? Tried to update conda but stubbornly refused to?

[timothy-nunn]

Also occurs on CSD3 login without using conda

[timothy-nunn]

This issue also occurs on Mac:

[timothy-nunn]

Removing the @numba.njit(cache=True) from plane_stress stops the error from occurring.

Numba seems to use scipy linear algebra routines when calling numpy.linalg.* in compiled functions.

	Ubutnu X86_64	Mac M1
Numba compile + np.linalg.solve (current)	/	x
NOT Numba compile + np.linalg.solve	/	/
NOT Numba compile + scipy.linalg.solve	/	x

Since Numba uses Scipy implementations of linear algebra routines ("it obtains the bindings for the necessary functions from SciPy" from https://numba.pydata.org/numba-doc/dev/user/performance-tips.html) it is not surprising that we observe a failure when using Numba given we observe the same failure when using SciPy--clearly the implementation of scipy.linalg.solve produces slightly different results on ARM architecture.

Last weird point: using the fastmaths flag with numba.njit exactly doubles "max absolute difference".

[nickodell]

Hi, one of the SciPy developers here, following up on your bug ticket.

Removing the @numba.njit(cache=True) from plane_stress stops the error from occurring.

Numba seems to use scipy linear algebra routines when calling numpy.linalg.* in compiled functions.

It's worth noting that if you want to de-activate nopython mode, and fall back to object mode for a single line of code, you can do this by using the numba.objmode() context manager. This will prevent numba from re-writing your code to use SciPy's solve. Here is an example of how to do this. This patch fixes the test you describe on my Apple M3.

main...nickodell:PROCESS:njo-nb-objmode

Misc notes:

• Tested with SciPy 1.13.1, 1.14.0, and 1.15.2 on Python 3.10.
• The command pytest tests/ -k test_plane_stress was used for testing.
• Documentation on using numba.objmode(): https://numba.pydata.org/numba-doc/dev/user/withobjmode.html#callback-into-the-python-interpreter-from-within-jit-ed-code
• Performance test: The main branch version of plane_stress() takes approx 4.3 μs. The modified plane_stress() takes approx 11.9 μs.

[timothy-nunn]

Hi @nickodell

Thanks for your reply on this issue and the SciPy issue. That objmode() mode suggestion seems to be a good solution to this problem.

I have done a bit more investigating myself and have decided (as I believe you realised on the issue) that this is not an ARM issue. To maybe add a spanner in the works, this issue was originally created for a CentOS X86-64 system. As you can see from the original traceback the test failure happens with the exact same absolute difference as I get on my Mac M1. This would suggest it is not related to the Apple Accelerate BLAS.

Unfortunately this system is down today, but I would be happy to run some tests on it when it is back up and report my findings on the SciPy issue if you think this would be useful.

Cheers,
Tim

[nickodell]

Unfortunately this system is down today, but I would be happy to run some tests on it when it is back up and report my findings on the SciPy issue if you think this would be useful.

Please do - in particular, could you check if the issue occurs on systems where NumPy and SciPy use the same BLAS/LAPACK implementation? scipy.show_config() and np.show_config() can be used to check this.