tests fail on AArch64, Fedora 33

[LutzWeischerFujitsu]

[jw@cn06 build]$ make VERBOSE=1 -j$(nproc) test
Running tests...
/usr/bin/ctest --force-new-ctest-process
Test project /data/jw/itk/build
...
99% tests passed, 19 tests failed out of 2698
...
The following tests FAILED:
889 - itkHDF5ImageIOTest (Failed)
985 - itkMINCImageIOTest3 (Failed)
995 - itkMINCImageIOTest-COM-t1_z-_byte_cor (Failed)
996 - itkMINCImageIOTest-COM-t1_z+_byte_cor (Failed)
997 - itkMINCImageIOTest-COM-t1_z-_byte_sag (Failed)
998 - itkMINCImageIOTest-COM-t1_z+_byte_sag (Failed)
999 - itkMINCImageIOTest-COM-t1_z-_byte_trans (Failed)
1000 - itkMINCImageIOTest-COM-t1_z+_byte_trans (Failed)
1368 - FrequencyIterators.Even3D (Failed)
1369 - FrequencyIterators.Even2D (Failed)
1469 - itkBSplineScatteredDataPointSetToImageFilterTest05 (Failed)
1550 - itkSymmetricEigenAnalysisImageFilterTestOrderByValue (Failed)
1551 - itkSymmetricEigenAnalysisImageFilterTestOrderByMagnitude (Failed)
1552 - itkSymmetricEigenAnalysisImageFilterTestDoNotOrder (Failed)
1883 - itkSingleLevelSetsv4DenseImage2DTest (Failed)
1888 - itkSingleLevelSetsv4ShiImage2DTest (Failed)
2389 - itkTubeSpatialObjectTest (Failed)
2390 - itkDTITubeSpatialObjectTest (Failed)
2516 - itkSLICImageFilterTest1 (Failed)
Errors while running CTest
make: *** [Makefile:126: test] Error 8
[jw@cn06 build]$

[thewtex]

To share more details on the issues:

ctest -D Experimental

which will report on the dashboard with details.

[LutzWeischerFujitsu]

[jw@cn05 build]$ ctest -D Experimental
Site: cn05
Build name: Linux-c++
Create new tag: 20210505-0854 - Experimental
Configure project
Each . represents 1024 bytes of output
.......... Size of output: 9K
Build project
Each symbol represents 1024 bytes of output.
'!' represents an error and '*' a warning.
.................................................. Size: 49K
........................................... Size of output: 92K
0 Compiler errors
0 Compiler warnings
Test project /data/jw/itk/build
Start 2651: itkTestTransformGetInverse
1/2705 Test #2651: itkTestTransformGetInverse ............................................................................. Passed 169.83 sec
Start 1102: itkLargeMetaImageWriteReadTest4
...
Start 891: itkHDF5ImageIOTest
900/2705 Test #891: itkHDF5ImageIOTest .....................................................................................***Failed 0.22 sec
...
Start 2705: itkMorphologicalWatershedImageFilterTestLevel50
2705/2705 Test #2705: itkMorphologicalWatershedImageFilterTestLevel50 ........................................................ Passed 0.11 sec

99% tests passed, 15 tests failed out of 2705

Label Time Summary:
BigIO = 387.54 secproc (8 tests)
ITKAnisotropicSmoothing = 0.90 secproc (6 tests)
ITKAntiAlias = 0.29 secproc (2 tests)
ITKBiasCorrection = 4.52 secproc (6 tests)
ITKBinaryMathematicalMorphology = 3.86 secproc (27 tests)
ITKClassifiers = 12.35 secproc (14 tests)
ITKColormap = 2.14 secproc (15 tests)
ITKCommon = 46.22 secproc (190 tests)
ITKConnectedComponents = 2.05 secproc (15 tests)
ITKConvolution = 8.28 secproc (34 tests)
ITKCurvatureFlow = 0.85 secproc (3 tests)
ITKDeconvolution = 4.31 secproc (16 tests)
ITKDeformableMesh = 1.69 secproc (5 tests)
ITKDenoising = 29.29 secproc (7 tests)
ITKDiffusionTensorImage = 0.57 secproc (5 tests)
ITKDisplacementField = 4.26 secproc (21 tests)
ITKDistanceMap = 8.80 secproc (28 tests)
ITKEigen = 0.13 secproc (2 tests)
ITKFFT = 4.51 secproc (32 tests)
ITKFastMarching = 82.63 secproc (32 tests)
ITKFiniteDifference = 0.05 secproc (1 test)
ITKGPUAnisotropicSmoothing = 0.04 secproc (1 test)
ITKGPUCommon = 0.09 secproc (1 test)
ITKGPUFiniteDifference = 0.03 secproc (1 test)
ITKGPUImageFilterBase = 0.03 secproc (1 test)
ITKGPUPDEDeformableRegistration = 0.07 secproc (1 test)
ITKGPURegistrationCommon = 0.02 secproc (1 test)
ITKGPUSmoothing = 0.03 secproc (1 test)
ITKGPUThresholding = 0.03 secproc (1 test)
ITKIOBMP = 1.36 secproc (14 tests)
ITKIOBioRad = 0.17 secproc (2 tests)
ITKIOBruker = 0.83 secproc (5 tests)
ITKIOCSV = 0.29 secproc (4 tests)
ITKIOGDCM = 5.18 secproc (33 tests)
ITKIOGE = 1.42 secproc (13 tests)
ITKIOGIPL = 0.36 secproc (4 tests)
ITKIOHDF5 = 0.44 secproc (3 tests)
ITKIOIPL = 0.04 secproc (1 test)
ITKIOImageBase = 7.84 secproc (79 tests)
ITKIOJPEG = 0.38 secproc (4 tests)
ITKIOJPEG2000 = 1.38 secproc (7 tests)
ITKIOLSM = 0.16 secproc (2 tests)
ITKIOMINC = 9.32 secproc (61 tests)
ITKIOMRC = 0.46 secproc (5 tests)
ITKIOMesh = 0.67 secproc (5 tests)
ITKIOMeshBYU = 0.14 secproc (2 tests)
ITKIOMeshBase = 0.33 secproc (2 tests)
ITKIOMeshFreeSurfer = 0.27 secproc (3 tests)
ITKIOMeshGifti = 5.26 secproc (3 tests)
ITKIOMeshOBJ = 1.16 secproc (3 tests)
ITKIOMeshOFF = 0.12 secproc (2 tests)
ITKIOMeshVTK = 0.71 secproc (7 tests)
ITKIOMeta = 2.66 secproc (31 tests)
ITKIONIFTI = 3.95 secproc (22 tests)
ITKIONRRD = 3.33 secproc (31 tests)
ITKIOPNG = 1.51 secproc (15 tests)
ITKIORAW = 0.50 secproc (6 tests)
ITKIOSiemens = 0.03 secproc (1 test)
ITKIOSpatialObjects = 0.61 secproc (6 tests)
ITKIOStimulate = 0.23 secproc (3 tests)
ITKIOTIFF = 5.23 secproc (54 tests)
ITKIOTransformBase = 0.23 secproc (3 tests)
ITKIOTransformHDF5 = 2.73 secproc (5 tests)
ITKIOTransformInsightLegacy = 0.24 secproc (3 tests)
ITKIOTransformMatlab = 0.15 secproc (2 tests)
ITKIOVTK = 1.17 secproc (8 tests)
ITKIOXML = 2.40 secproc (33 tests)
ITKImageAdaptors = 0.70 secproc (7 tests)
ITKImageCompare = 0.76 secproc (8 tests)
ITKImageCompose = 1.21 secproc (12 tests)
ITKImageFeature = 6.42 secproc (40 tests)
ITKImageFilterBase = 0.98 secproc (6 tests)
ITKImageFrequency = 0.36 secproc (4 tests)
ITKImageFunction = 3.52 secproc (29 tests)
ITKImageFusion = 5.89 secproc (24 tests)
ITKImageGradient = 1.80 secproc (16 tests)
ITKImageGrid = 12.20 secproc (74 tests)
ITKImageIntensity = 8.65 secproc (83 tests)
ITKImageLabel = 0.59 secproc (6 tests)
ITKImageNoise = 0.88 secproc (9 tests)
ITKImageSources = 2.43 secproc (15 tests)
ITKImageStatistics = 4.78 secproc (38 tests)
ITKIntegratedTest = 0.92 secproc (12 tests)
ITKKLMRegionGrowing = 11.31 secproc (2 tests)
ITKLabelMap = 13.03 secproc (109 tests)
ITKLabelVoting = 0.76 secproc (8 tests)
ITKLevelSets = 11.93 secproc (29 tests)
ITKLevelSetsv4 = 54.94 secproc (42 tests)
ITKMarkovRandomFieldsClassifiers = 0.27 secproc (3 tests)
ITKMathematicalMorphology = 12.58 secproc (76 tests)
ITKMesh = 7.65 secproc (57 tests)
ITKMetricsv4 = 58.16 secproc (37 tests)
ITKNarrowBand = 0.27 secproc (3 tests)
ITKOptimizers = 1.90 secproc (19 tests)
ITKOptimizersv4 = 3.46 secproc (30 tests)
ITKPDEDeformableRegistration = 46.67 secproc (18 tests)
ITKPath = 1.50 secproc (15 tests)
ITKPolynomials = 0.11 secproc (2 tests)
ITKQuadEdgeMesh = 3.13 secproc (37 tests)
ITKQuadEdgeMeshFiltering = 99.21 secproc (48 tests)
ITKRegionGrowing = 1.06 secproc (10 tests)
ITKRegistrationCommon = 73.24 secproc (66 tests)
ITKRegistrationMethodsv4 = 220.08 secproc (23 tests)
ITKSignedDistanceFunction = 0.20 secproc (3 tests)
ITKSmoothing = 2.00 secproc (18 tests)
ITKSpatialFunction = 0.13 secproc (2 tests)
ITKSpatialObjects = 2.85 secproc (29 tests)
ITKStatistics = 30.43 secproc (112 tests)
ITKSuperPixel = 0.94 secproc (4 tests)
ITKTestKernel = 0.31 secproc (3 tests)
ITKThresholding = 8.04 secproc (69 tests)
ITKTransform = 176.33 secproc (48 tests)
ITKTransformFactory = 0.14 secproc (2 tests)
ITKVTK = 0.05 secproc (1 test)
ITKVideoCore = 0.62 secproc (7 tests)
ITKVideoFiltering = 0.94 secproc (5 tests)
ITKVideoIO = 0.79 secproc (4 tests)
ITKVoronoi = 1.31 secproc (6 tests)
ITKWatersheds = 4.28 secproc (32 tests)
KWStyle = 0.91 secproc (1 test)
RUNS_LONG = 844.03 sec*proc (28 tests)

Total Test time (real) = 1631.40 sec

The following tests FAILED:
891 - itkHDF5ImageIOTest (Failed)
987 - itkMINCImageIOTest3 (Failed)
997 - itkMINCImageIOTest-COM-t1_z-_byte_cor (Failed)
998 - itkMINCImageIOTest-COM-t1_z+_byte_cor (Failed)
999 - itkMINCImageIOTest-COM-t1_z-_byte_sag (Failed)
1000 - itkMINCImageIOTest-COM-t1_z+_byte_sag (Failed)
1001 - itkMINCImageIOTest-COM-t1_z-_byte_trans (Failed)
1002 - itkMINCImageIOTest-COM-t1_z+_byte_trans (Failed)
1370 - FrequencyIterators.Even3D (Failed)
1371 - FrequencyIterators.Even2D (Failed)
1890 - itkSingleLevelSetsv4DenseImage2DTest (Failed)
1895 - itkSingleLevelSetsv4ShiImage2DTest (Failed)
2396 - itkTubeSpatialObjectTest (Failed)
2397 - itkDTITubeSpatialObjectTest (Failed)
2523 - itkSLICImageFilterTest1 (Failed)
Performing coverage

Accumulating results (each . represents one file):

    Covered LOC:         0
    Not covered LOC:     0
    Total LOC:           0
    Percentage Coverage: 0.00%

Submit files
SubmitURL: https://open.cdash.org/submit.php?project=Insight
Uploaded: /data/jw/itk/build/Testing/20210505-0854/Configure.xml
Uploaded: /data/jw/itk/build/Testing/20210505-0854/Build.xml
Uploaded: /data/jw/itk/build/Testing/20210505-0854/Test.xml
Uploaded: /data/jw/itk/build/Testing/20210505-0854/Coverage.xml
Uploaded: /data/jw/itk/build/Testing/20210505-0854/CoverageLog-0.xml
Uploaded: /data/jw/itk/build/Testing/20210505-0854/Done.xml
Submission successful
Errors while running CTest
Output from these tests are in: /data/jw/itk/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
[jw@cn05 build]$

[dzenanz]

Details of test failures can now be seen at:
https://open.cdash.org/viewTest.php?onlyfailed&buildid=7207227

I assume that some of the failures could be ascribed to numerical differences. IO test failures are harder to explain. @gdevenyi any idea why would MINC IO tests fail on ARM64@Fedora33?

[gdevenyi]

libminc seems to pass tests in Debian's arm64 build:
https://buildd.debian.org/status/fetch.php?pkg=libminc&arch=arm64&ver=2.4.03-3&stamp=1605695424&raw=0

But these are ITK level tests, will need to see what its doing.

[gdevenyi]

These tests have hard-coded values, is it possible to run this verbose enough to get the outputs for comparison?

[gdevenyi]

Oh, the HDF5 test fails

- itkHDF5ImageIOTest (Failed)

MINC uses HDF5 for its IO, maybe this is just a downstream consequence?

[LutzWeischerFujitsu]

Tried with the current master on x86, no test errors.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Thank you for your contributions.