tofu 1.4.8 fails for Windows only on conda-forge

[Didou09]

The attempt to export tofu 1.4.8 to conda forge reveals that all unit tests are passed except for Windows platforms and for python 3.6 and 3.7 (3.8 and 3.9 are fine)

There are 2 errors in the surface and volume sampling routines:

======================================================================
ERROR: tests.tests01_geom.tests01_GG.test07_Ves_Vmesh_Tor
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\bld\tofu_1607509981720\_test_env\lib\site-packages\nose\case.py", line 197, in runTest
    self.test(*self.arg)
  File "D:\bld\tofu_1607509981720\test_tmp\tofu\tests\tests01_geom\tests01_GG.py", line 309, in test07_Ves_Vmesh_Tor
    margin=1.e-9)
  File "tofu\geom\_GG.pyx", line 1102, in tofu.geom._GG._Ves_Vmesh_Tor_SubFromD_cython
ValueError: Buffer dtype mismatch, expected 'long' but got 'long long'

======================================================================
ERROR: tests.tests01_geom.tests03_core.Test01_Struct.test15_get_sampleV
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\bld\tofu_1607509981720\test_tmp\tofu\tests\tests01_geom\tests03_core.py", line 452, in test15_get_sampleV
    algo='new')
  File "D:\bld\tofu_1607509981720\_test_env\lib\site-packages\tofu\geom\_core.py", line 1432, in get_sampleV
    return _comp._Ves_get_sampleV(*args, **kwdargs)
  File "D:\bld\tofu_1607509981720\_test_env\lib\site-packages\tofu\geom\_comp.py", line 712, in _Ves_get_sampleV
    num_threads=num_threads,
  File "tofu\geom\_GG.pyx", line 1102, in tofu.geom._GG._Ves_Vmesh_Tor_SubFromD_cython
ValueError: Buffer dtype mismatch, expected 'long' but got 'long long'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\bld\tofu_1607509981720\_test_env\lib\site-packages\nose\case.py", line 197, in runTest
    self.test(*self.arg)
  File "D:\bld\tofu_1607509981720\test_tmp\tofu\tests\tests01_geom\tests03_core.py", line 462, in test15_get_sampleV
    raise Exception(msg)
Exception: Buffer dtype mismatch, expected 'long' but got 'long long'
Failed for Tor_Ves_VesIn
	- ii = 0
	- Lim = []
	- domain = None
	- algo = 'new'

----------------------------------------------------------------------
Ran 127 tests in 140.733s

@lasofivec could you have a look at these ?
I feel that we already encountered that kind of issue, but can't remember how we fixed it, any memories @flothesof ?

[flothesof]

Yes, I have memories of this :)
I think this is similar to the fixes on Windows where the dtypes are a bit different than on linux and mac.
A good starting point might be #199, where the tests needed to have explicit casts like Indin = indin.nonzero()[0].astype(np.long).

[flothesof]

Indeed the solution was as I thought: np functions (nonzero in the example above, argmax in this case) return default dtypes depending on platform.

Since this breaks windows and is difficult to remember, this will probably happen again.

At the moment, conda-forge will make us notice since it builds on windows.

A better solution would be to switch from travis to a platform that supports windos (could be windows-azure).

Overall, we would probably benefit since the aim of tofu is to be truly cross platform.

[Didou09]

Yes, good point.
However, since it only fails for python 3.6 and 3.7 it looks a bit like the problem was somehow identified by numpy or Python developers and fixed in Python 3.8 and 3.9.
Maybe we could try to know which version of numpy fixes this issue and enforce that minimum version in the tofu dependencies ?

[flothesof]

I'm not so sure about that. If I read the CI logs correctly, the tests just didn't run under these versions you mention. Another bug...? Le mer. 9 déc. 2020 à 13:10, Didier <notifications@github.com> a écrit :

…

Yes, good point. However, since it only fails for python 3.6 and 3.7 it looks a bit like the problem was somehow identified by numpy or Python developers and fixed in Python 3.8 and 3.9. Maybe we could try to know which version of numpy fixes this issue and enforce that minimum version in the tofu dependencies ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#431 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASAKXL3DJ3PLP4RJ23WSJTST5SLTANCNFSM4UTKBXWQ> .

[lasofivec]

which test didn't run on python 3.8 and 3.9 ?

[flothesof]

For reasons I don't understand, the 3.6 and 3.7 win builds started running the tests as intended by the build, while the 3.8 and 3.9 did call nosetests but nosetests did not run any tests.

See for instance this log:
https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=249918&view=logs&j=e5cdccbf-4751-5a24-7406-185c9d30d021&t=66e4a587-a723-53e9-873c-6c9a2c5971ea&l=8010

which shows:

(%PREFIX%) %SRC_DIR%>nosetests tofu 

----------------------------------------------------------------------
Ran 0 tests in 0.005s

OK

This is a false positive but I don't have any time to look into that...

[flothesof]

@Didou09 @lasofivec since the fix has been merged, is it possible to release a 1.4.8.1 version so that we can run the conda-forge builds again? Or do you want to retag the current version with the 1.4.8 tag and reupload?
Anyhow, I'll let you guys deal with it. When pypi gets updated, I'll rerun the conda-forge build.

[Didou09]

Hum, I don't know,

• We can't retag because you can't replace files on Pypi, we'll have to issue a new release
• So either we issue a 1.4.8.1 or a 1.4.9., or we skyp conda-forge for this release and the merge will just get inside 1.4.9 in a few weeks.

What do you guys think ? @lasofivec @flothesof ?
I would probably go for issuing a 1.4.9 now, even if it is minor it remains consistent with previous versioning.

[flothesof]

I would also keep the numbering. I vote for 1.4.9 to be released immediately! 🙂 Le mer. 9 déc. 2020 à 17:09, Didier <notifications@github.com> a écrit :

…

Hum, I don't know, - We can't retag because you can't replace files on Pypi, we'll have to issue a new release - So either we issue a 1.4.8.1 or a 1.4.9., or we skyp conda-forge for this release and the merge will just get inside 1.4.9 in a few weeks. What do you guys think ? @lasofivec <https://github.com/lasofivec> @flothesof <https://github.com/flothesof> ? I would probably go for issuing a 1.4.9 now, even if it is minor it remains consistent with previous versioning. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#431 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASAKXKVEXCBDDG4TAN3ZZ3ST6OLTANCNFSM4UTKBXWQ> .

[flothesof]

1.4.9 is on conda-forge! yay