ARM 32-bit progress

[manu-st]

I'm opening an issue to track the progress of ARM 32-bit with respects to the regression tests.

Currently I'm getting the following results on Ubuntu 14.04 running on a NVidia Jetson TK1. This is unfortunately not the results you get from a clean checkout, the only difference is that it contains a fix for issue #5422 (PR dotnet/coreclr#3879), but overall it should provide a good base to monitor our progress:

=======================
     Test Results
=======================
# Tests Discovered : 6018
# Passed           : 4313
# Failed           : 1361
# Skipped          : 344
=======================

I've attached the XML results:
coreclrtests.zip

[leemgs]

This is unfortunately not the results you get from a clean checkout,

Even though you could get the 1,361 Failed result, this result is meaningful to me. Actually, we have been meeting a lot of "NYI" functions from CoreCLR.:( It means that we have to do a lot of things for stablization of CoreCLR.

Currently I'm getting the following results on Ubuntu 14.04 running on a NVidia Jetson TK1

BTW, Do you also focus on the Ubuntu/ARM32bit without the Ubuntu/ARM64bit?
http://elinux.org/Jetson_TK1

• CPU: NVIDIA "4-Plus-1" 2.32GHz ARM quad-core Cortex-A15 CPU with Cortex-A15 battery-saving shadow-core
• GPU: NVIDIA Kepler "GK20a" GPU with 192 SM3.2 CUDA cores (upto 326 GFLOPS)
• Memory: ~ 8GiB DDR3

\CC: @myungjoo , @lemmaa
They also interested in the regression test of CoreCLR.

[manu-st]

@leemgs I'm not sure to understand what you mean by Ubuntu/ARM32bit without the Ubuntu/ARM32bit part?

[leemgs]

@manu-silicon Sorry, Typo, I have modified.

[manu-st]

@leemgs My board is the 32-bit variant, so it is just Ubuntu/ARM32bit.

https://en.wikipedia.org/wiki/Tegra#Tegra_K1

[manu-st]

With the fix for PR dotnet/coreclr#3981 the result is:

=======================
     Test Results
=======================
# Tests Discovered : 6019
# Passed           : 4793
# Failed           : 882
# Skipped          : 344
=======================

coreclrtests.zip

[prajwal-aithal]

@manu-silicon I am trying to do the same (run coreclr tests on ARM environment) by following this guide https://github.com/dotnet/coreclr/blob/master/Documentation/building/unix-test-instructions.md. According to this, building tests for ARM results in an error (as building the native code using cmake in Windows does not support arm architecture yet). I can make some changes to the buildtest script to build only the C# based tests for ARM architecture.

I wanted to ask how you are building the test binaries for ARM environment (using buildtest.cmd). Have you made some local changes (to build only the C# code and ignore the CMake build) or is there some other way to do it?

[manu-st]

@prajwal-aithal I've followed the instructions and build the tests on Windows and copy the tests folder on Linux ARM. I built on ARM the native code for CoreFX (just the native code as the rest won't build). I built CoreFX on Linux x64 and copied the assemblies over. I built mscorlib on Linux x64 and copied it over.

Once I had all the paths setup properly, I launched the command to run the tests.

[prajwal-aithal]

@manu-silicon Oh, ok. I will try this then. Thank you :)

[manu-st]

Today's result with FEATURE_STUBS_AS_IL disabled:

=======================
     Test Results
=======================
# Tests Discovered : 6019
# Passed           : 4836
# Failed           : 839
# Skipped          : 344
=======================

coreclrtests.zip

[jyoungyun]

Target: Raspberry Pi2
CoreClr commit id : ff26d68 (around april 9)

=======================
     Test Results
=======================
# Tests Discovered : 7422
# Passed           : 6074
# Failed           : 1017
# Skipped          : 331
=======================

I'm curious why my test discovered count is different with others.
CoreClr_UnitTest_Results_160412_01.zip

[myungjoo]

@jyoungyun Next time, please attach the result XML file as well.

[manu-st]

@jyoungyun The difference in the number of tests is related on how I build first the test on Windows and then copied them over to the Linux ARM device. As @myungjoo is saying, having the XML file would enable us to compare and see what the differences are.

[jyoungyun]

@manu-silicon I attached xml file. Thank you.

[jyoungyun]

Target: Raspberry Pi2
Coreclr commit id : 31fada1 + dotnet/coreclr#4460 + dotnet/coreclr#4503

==========================
      Test Results
==========================
# Tests Discovered : 7421
# Passed           : 6547
# Failed           : 543
# Skipped          : 331
==========================

CoreClr_UnitTest_Results_160427.zip

[masonwheeler]

Wow, so those two fixes cleared up about half of the failing tests? Nice!