[x86/Linux] Cross building for x86 32bit on Linux

[seanshpark]

I would like to enable CoreCLR for 32bit x86 Linux based on the base with current Windows x86 codes.
I'm thinking this task to be in two stages. First is to fix all build errors and then to make it work properly with unit tests. Task items to do will be added to below lists. I hope compile will be done in both 32bit host native and 64bit host cross building.

Fix build error

• fix and add for x86 cmake configuration [x86/Linux] fix and add for x86 cmake configuration coreclr#8219
• Q) compile directive for x86/linux dotnet/coreclr#8222
• Fix compilation in pal [x86/Linux] Fix compilation in pal and context header coreclr#8244
• Fix redefined DISPATCHER_CONTEXT compile error [x86/linux] Fix redefined DISPATCHER_CONTEXT compile error coreclr#8246
• Q) x86/Linux, stdcall calling convention ignored on variadic function dotnet/coreclr#8248
• Fix compile error redifinition RUNTIME_FUNCTION [x86/Linux] Fix compile error redifinition RUNTIME_FUNCTION coreclr#8249
• Fix stdcall calling convention ignored on variadic function [x86/Linux] Fix stdcall calling convention ignored on variadic function coreclr#8252
• Fix unsupported architecture in seh-unwind.cpp [x86/Linux] Fix unsupported architecture in seh-unwind.cpp coreclr#8262
• Fix no matching function for call to 'InternalCreateThread' [x86/Linux] Fix no matching function for call to 'InternalCreateThread' coreclr#8265
• Fix compile error with exception handling [x86/Linux] Fix compile error with exception handling coreclr#8269
• Fix SEH '__try' is not supported on this target [x86/Linux] Fix SEH '__try' is not supported on this target coreclr#8271
• Fix unknown type name 'PTR_RUNTIME_FUNCTION' [x86/Linux] Fix unknown type name 'PTR_RUNTIME_FUNCTION' coreclr#8272
• Fix comparison of two values with different enumeration types [x86/Linux] Fix comparison of two values with different enumeration types coreclr#8273
• Fix unknown type name 'ExInfo' [x86/Linux] Fix unknown type name 'ExInfo' coreclr#8274
• Fix constructor missing in UnmanagedToManagedFrame [x86/Linux] Fix constructor missing in UnmanagedToManagedFrame coreclr#8275
• Supports to build PAL tests under x86/Linux Supports to build PAL tests under x86/Linux coreclr#8292
• Fix compile error in zapcode [x86/Linux] Fix compile error in zapcode coreclr#8294
• X Fix compile error in codeman [x86/Linux] Fix compile error in codeman coreclr#8296
• Fix error variable 'td' is uninitialized [x86/Linux] Fix error variable 'td' is uninitialized coreclr#8299
• Enable field EndAddress only for AMD64 in PAL of RUNTIME_FUNCTION Enable field EndAddress only for AMD64 in PAL of RUNTIME_FUNCTION coreclr#8305
• Renames i386 as amd64 in arch (PAL) [x86/Linux] Renames i386 as amd64 in arch (PAL) coreclr#8307 (parjong)
• Add missing files in x86/Linux PAL [x86/Linux] Add missing files in x86/Linux PAL coreclr#8318 (parjong)
• Fix 'stdcall' here was previously declared without [x86/Linux] Fix 'stdcall' here was previously declared without coreclr#8320
• Q) [x86/Linux] shift count >= width of type error dotnet/coreclr#8331
• X Fix cannot initialize a variable of type 'CLRFLSGETBLOCK' [x86/Linux] Fix cannot initialize a variable of type 'CLRFLSGETBLOCK' coreclr#8332
• Fix ignored attribute warning for x86/Linux build [x86/Linux] Fix ignored attribute warning for x86/Linux build coreclr#8334 (parjong)
• Disable FEATURE_AVX_SUPPORT [x86/Linux] Disable FEATURE_AVX_SUPPORT coreclr#8335 (parjong)
• Fix the mistmatch between gc_thread_stub and GCThreadFunction [x86/Linux] Fix the mistmatch between gc_thread_stub and GCThreadFunction coreclr#8336 (parjong)
• Add virtual dtor for ArrayStubCache class [x86/Linux] Add virtual dtor for ArrayStubCache class coreclr#8339 (parjong)
• Emit compile error on porting issue when PORTABILITY_CHECK is set [x86/Linux] Emit compile error on porting issue when PORTABILITY_CHECK is set coreclr#8340 (parjong)
• X Revise RealCodeHeader and CodeHeader [x86/Linux] Revise RealCodeHeader and CodeHeader coreclr#8344 (parjong)
• Disable FEATURE_AVX_SUPPORT [x86/Linux] Disable FEATURE_AVX_SUPPORT coreclr#8335(parjong)
• Fix the mismatch over LPOVERLAPPED_COMPLETION_ROUTINE [x86/Linux] Fix mismatch over LPOVERLAPPED_COMPLETION_ROUTINE coreclr#8364 (parjong)
• Fix mismatch between sortversioning.h and sortversioning.cpp [x86/Linux] Fix mismatch between sortversioning.h and sortversioning.cpp coreclr#8366 (parjong)
• Uses portable ROTATE_LEFT for x86/Linux [x86/Linux] Uses portable ROTATE_LEFT for x86/Linux coreclr#8367 (parjong)
• Fix inconsistency between the declarations of GetAppDomain [x86/Linux] Fix inconsistency between the declarations of GetAppDomain coreclr#8368 (parjong)
• Fix mismatch between longfilepathwrappers.cpp and longfilepathwrappers.h [x86/Linux] Fix mismatch between longfilepathwrappers.cpp and longfilepathwrappers.h coreclr#8370 (parjong)
• Disable windows-specific code for x86/Linux [x86/Linux] Disable windows-specific code coreclr#8371 (parjong)
• X Declares GetCallerSp for WIN64EXCEPTION (not for TARGET_X86) [x86/Linux] Declare GetCallerSp when WIN64EXCEPTION (not _TARGET_X86_) is defined coreclr#8372 (parjong)
• Disable EHWatsonBucketTracker for non-windows platforms [x86/Linux] Disable EHWatsonBucketTracker for non-windows platforms coreclr#8373 (parjong)
• X Disables FrameHandlerExRecordWithBarrier for non-Windows platforms [x86/Linux] Disables FrameHandlerExRecordWithBarrier for non-Windows platforms coreclr#8374 (parjong)
• Disable WIN64EXECPTION and fix related changes [x86/Linux] Disable WIN64EXECPTION and fix related changes coreclr#8375
• Fix Dacp structure size mismatch [x86/Linux] Fix Dacp structure size mismatch coreclr#8377
• Declare CtorFtnType only when it is used [x86/Linux] Declare CtorFtnType only when it is used coreclr#8378 (parjong)
• Disable PrintSEHChain for non-Windows platforms [x86/Linux] Disable PrintSEHChain for non-Windows platforms coreclr#8379 (parjong)
• X Disable CheckStackBarrier for non-Windows platforms [x86/Linux] Disable CheckStackBarrier for non-Windows platforms coreclr#8380 (parjong)
• X Revises exception handling macros for x86/Linux [x86/Linux] Revises exception handling macros for x86/Linux coreclr#8381 (parjong)
• Disable LeaveCatch for non-Windows platforms [x86/Linux] Mark LeaveCatch as NYI coreclr#8384 (parjong)
• X Do not invoke VerifyValidTransitionFromManagedCode for non Windows [x86/Linux] Do not invoke VerifyValidTransitionFromManagedCode for ... coreclr#8385 (parjong)
• Updates m_LastRedirectIP and m_SpinCount only for non-Unix platforms [x86/Linux] Updates m_LastRedirectIP and m_SpinCount only for non-Unix platforms coreclr#8386 (parjong)
• X Push FrameHandlerExRecord when FEATURE_PAL is not defined [x86/Linux] Push FrameHandlerExRecord when FEATURE_PAL is not defined coreclr#8387 (parjong)
• Fix extraneous parentheses [x86/Linux] Fix extraneous parentheses coreclr#8388
• X Fix cannot jump from this goto statement to its label [x86/Linux] Fix cannot jump from this goto statement to its label coreclr#8389
• Adjust the definition of FnStaticBaseHelper for x86 [x86/Linux] Adjust the definition of FnStaticBaseHelper for x86 coreclr#8390 (parjong)
• Fix error: '&&' within '||' [x86/Linux] Fix error: '&&' within '||' coreclr#8391
• Uses HCIMPL2 in the impl of JIT helpers [x86/Linux] Use portable JIT helpers coreclr#8392 (parjong)
• Exclude jithelp.asm for x86/Linux [x86/Linux] Exclude jithelp.asm for x86/Linux coreclr#8393 (parjong)
• Fix mismatch between signature and implementation of ActivationFunctions [x86/Linux] Fix mismatch between signature and implementation of ActivationFunctions coreclr#8394 (parjong)
• Fix prototype for GCThreadStub [x86/Linux] Fix prototype for GCThreadStub coreclr#8397
• Add parentheses aroung logical operations [x86/Linux] Add parentheses aroung logical operations coreclr#8406 (parjong)
• Port EXCEPTION_REGISTRATION_RECORD [x86/Linux] Port EXCEPTION_REGISTRATION_RECORD coreclr#8408 (parjong)
• Re-enable FrameHandlerExRecord for x86/Linux [x86/Linux] Re-enable FrameHandlerExRecord for x86/Linux coreclr#8409 (parjong)
• Disable Watson-related code for non-Windows platforms [x86/Linux] Disable Watson-related code for non-Windows platforms coreclr#8410 (parjong)
• Re-enable UMThkCallFrame and fix compile errors [x86/Linux] Re-enable UMThkCallFrame and fix compile errors coreclr#8411 (parjong)
• Add Portable PopSEHRecords as NYI [x86/Linux] Add Portable PopSEHRecords as NYI coreclr#8412 (parjong)
• Resolve linking errors due to duplicated functions [x86/Linux] Resolve linking errors due to duplicated functions coreclr#8413 (parjong)
• Mark several Windows-specific functions in excepx86.cpp as NYI [x86/Linux] Mark several Windows-specific functions in excepx86.cpp as NYI  coreclr#8424 (parjong)
• Use UnhandledExceptionHandlerUnix only when WIN64EXCEPTIONS is defined [x86/Linux] Add UnhandledExceptionHandlerUnix Stub coreclr#8425 (parjong)
• X Fix dangling TheUMEntryPrestub reference [x86/Linux] Fix dangling TheUMEntryPrestub reference coreclr#8426 (parjong)
• Fix several unknown pragma compile errors [x86/Linux] Fix several unknown pragma compile errors coreclr#8427 (parjong)
• Fix several parentheses compile warnings [x86/Linux] Fix several parentheses compile warnings coreclr#8428 (parjong)
• Fix unused function warning [x86/Linux] Fix unused function warning coreclr#8429 (parjong)
• Hide Winodws-specfic methods [x86/Linux] Revise COMPlusThrowCallback coreclr#8430 (parjong)
• X Re-enable methods in StubLinkerCPU [x86/Linux] Re-enable methods in StubLinkerCPU coreclr#8431 (parjong)
• Enclose stub-linking methods with FEATURE_STUBS_AS_IL [x86/Linux] Enclose stub-linking methods with FEATURE_STUBS_AS_IL coreclr#8432 (parjong)
• Fix exception handling routine [x86/Linux] Fix exception handling routine coreclr#8433 (parjong)
• Revert UMThkCallFrame-related changes [x86/Linux] Revert UMThkCallFrame-related changes coreclr#8434 (parjong)
• Fix dangling CLR_ImpersonateLoggedOnUser reference [x86/Linux] Fix dangling CLR_ImpersonateLoggedOnUser reference coreclr#8435 (parjong)
• Extract ARRAYSTUBS_AS_IL code from STUBS_AS_IL region [x86/Linux] Extract ARRAYSTUBS_AS_IL code from STUBS_AS_IL region coreclr#8443 (parjong)
• Fix dangling StubLinkerCPU::EmitDelegateInvoke in x86/Linux [x86/Linux] Fix dangling StubLinkerCPU::EmitDelegateInvoke in x86/Linux coreclr#8444 (parjong)
• Enclose ArrayOpStub Exceptions with FEATURE_ARRAYSTUB_AS_IL [x86/Linux] Enclose ArrayOpStub Exceptions with FEATURE_ARRAYSTUB_AS_IL coreclr#8445 (parjong)
• Fix dangling DoubleToNumber and NumberToDouble [x86/Linux] Fix dangling DoubleToNumber and NumberToDouble coreclr#8446 (parjong)
• Use Portable Floating-point Arithmetic Helpers [x86/Linux] Use Portable Floating-point Arithmetic Helpers coreclr#8447 (parjong)
• X Use Portable SHIFT JIT Helpers [x86/Linux] Use Portable SHIFT JIT Helpers coreclr#8448 (parjong)
• Use Portable LMul JIT Helper [x86/Linux] Use Portable LMul JIT Helper coreclr#8449 (parjong)
• Fix no known conversion from 'void ()' to 'void *' [x86/Linux] Fix no known conversion from 'void ()' to 'void *' coreclr#8450
• Fix all paths through this function will call itself [x86/Linux] Fix all paths through this function will call itself coreclr#8451
• Fix indirection of non-volatile null pointer will be deleted [x86/Linux] Fix indirection of non-volatile null pointer will be deleted coreclr#8452
• Fix dangling ClrCaptureContext [x86/Linux] Fix dangling ClrCaptureContext coreclr#8453 (parjong)
• Add a stub for THROW_CONTROL_FOR_THREAD_FUNCTION [x86/Linux] Add a stub for THROW_CONTROL_FOR_THREAD_FUNCTION coreclr#8455 (parjong)
• Add dummy files and port gmsasm.asm [x86/Linux] Port gmsasm.asm coreclr#8456 (parjong)
• Fix inconsistency in GetCLRFunction definitions [x86/Linux] Fix inconsistent GetCLRFunction definitions coreclr#8472 (parjong)

Fix SOS build error (lucenticus)

• Q) What are the DBG_TARGET_XXX used for? dotnet/coreclr#8458
• Disable ARM target support for xplat [x86/Linux][SOS] Disable ARM target support for xplat coreclr#8471
• Fix DataTarget::GetPointerSize for x86 [x86/Linux][SOS] Fix DataTarget::GetPointerSize for x86 coreclr#8473
• Don't include utilcode.h in gcdump(x86).cpp [x86/Linux][SOS] Don't include utilcode.h in gcdump(x86).cpp  coreclr#8475
• Add CALLBACK (aka stdcall) to function declarations [x86/Linux][SOS] Add CALLBACK (aka stdcall) to function declarations coreclr#8476
• Add Dummy Exception Handler [x86/Linux] Add Dummy Exception Handler coreclr#8613

Fix execution error

• Create the TheUMEntryPrestub asm helper: [x86/Linux] Fix constructor missing in UnmanagedToManagedFrame coreclr#8275 (review)
• Port asmhelpers.asm [x86/Linux] Port asmhelpers.asm coreclr#8489
• Port jithelp.asm [x86/Linux] Port jithelp.asm coreclr#8491
• Port StubLinkerCPU::EmitSetup [x86/Linux] Port StubLinkerCPU::EmitSetup coreclr#8494
• Add definitions in CMakeLists.txt file of lldbplugin [x86/Linux][SOS] Add definitions in CMakeLists.txt file of lldbplugin coreclr#8499
• Get correct stack pointer from DT_CONTEXT [x86/Linux][SOS] Get correct stack pointer from DT_CONTEXT coreclr#8500
• Port Several Stubs as NYI [x86/Linux] Port Several Stubs as NYI coreclr#8515

CoreFX

• Initial support of x86 cross build [x86/Linux] Initial support of x86 cross build corefx#14267
• Add RID definition in https://github.com/dotnet/corefx/blob/master/pkg/Microsoft.NETCore.Platforms/runtime.json

For CoreCLR, CoreFX and Core-Setup repos:

• Add to cross targeting support in the build
• Get Product build clean
• Get Linux x86 packages generated
• Enable CI support for build
• Enable pipeline build support

Once the above is done, we should be able to produce .NET Core for Linux for x86 E2E.

[jkotas]

@seanshpark Thanks a lot for getting the Linux x86 port started!

https://github.com/dotnet/coreclr/issues/1264 has earlier discussion on this. Look for replies that contain x86 - there are some useful tidbits.

cc @janvorli @gkhanna79

[gkhanna79]

Thanks for looping me in @jkotas @seanshpark I took the liberty of adding few more specific items that I had followed when bringing up WinArm32 build.

[gkhanna79]

Needless to say, I will be happy to help with bunch of these things.

[janvorli]

@seanshpark awesome! Please feel free to ping me if you have any questions or need help with solving problems with coreclr, its PAL or native parts of CoreFX.

[seanshpark]

Wow, thanks for the support! I got lots of things to study and learn and don't think I can do this by myself by looking at the updated task items :)
I just got the make files rough fixes so that CoreCLR builds with no errors last weekend on my local Linux machine but not executable yet. I'll try to make a PR one by one to fix the builds.
I've found some .asm translation needed to .S and also some inline asembly in .cpp files. And some files that can be copied from amd64.

[lucenticus]

@seanshpark I just would like to share my brief investigation: I've succesfully built with your pull requests + disabled tests (with skiptests) + disabled libsosplugin + some hacks. But when I tried to run simple helloworld.exe, I got undefined symbol error for some symbols located in *.asm files. Probably, this information could be useful for you.

[seanshpark]

I got undefined symbol error for some symbols located in *.asm files

@lucenticus , yes, there are lots of undefined symbols to fix and @parjong is currently working on it. Thanks for the share :) . I'll be working on the SOS.

[seanshpark]

As there will be lots of unit test fix PRs, I think it is better not to add anymore PR items related with execution fixes... :)

[lucenticus]

@seanshpark I've tried to run native 32bit build for CoreFX (using schroot) and found that init-tools.sh script downloads 64bit dotnet package only (https://github.com/dotnet/corefx/blob/master/init-tools.sh#L80). The same behavior has init-tools.sh from CoreCLR. It seems that we have to enable x86 support for https://github.com/dotnet/cli project in case we need native build (without cross)

[janvorli]

@lucenticus enabling support for x86 Linux at this level should be the last step of the bringup, after everything works. It doesn't make sense to do it until you have fully functional coreclr. It is very preliminary to do it now. You can do native part of CoreFX build without it. You just need to run src/Native/build-native.sh instead of the root build.sh

[lucenticus]

@janvorli Thanks a lot for suggestion! I'll use src/Native/build-native.sh script for that.

[gkhanna79]

@seanshpark I just did a build of CoreCLR master on Linux Mint 18 (Ubuntu 16.04 based) as build.sh x86 and the generated binaries are all 64bit instead of x86 (readelf -h ./libcoreclr.so shows it to be ELF64). Is there a different way you have enabled the build?

[seanshpark]

@gkhanna79 , I use like this,

export NETCORE=$HOME/netcore

# build rootfs
sudo ROOTFS_DIR=$NETCORE/rootfs/x86-xenial ./cross/build-rootfs.sh x86 xenial

# build
ROOTFS_DIR=$NETCORE/rootfs/x86-xenial ./build.sh x86 debug cross clang3.8 skipnuget

[seanshpark]

And for a successful build you need to apply parjong:fix/x86_eh_macros (#8381) patch and disable SOS as some PRs are not merged yet. I use like this.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 702d177..df5cd5c 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -531,7 +531,7 @@ if(CLR_CMAKE_PLATFORM_UNIX)
 endif(CLR_CMAKE_PLATFORM_UNIX)

 if(CLR_CMAKE_PLATFORM_UNIX)
-  add_subdirectory(src/ToolBox/SOS/lldbplugin)
+  #add_subdirectory(src/ToolBox/SOS/lldbplugin)
   if(NOT DEFINED CLR_CROSS_COMPONENTS_BUILD)
     add_subdirectory(src/pal)
   endif(NOT DEFINED CLR_CROSS_COMPONENTS_BUILD)
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index d13e8f9..53cd3c3 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -96,7 +96,7 @@ function(add_executable_clr)
 endfunction()

 if(CLR_CMAKE_PLATFORM_UNIX)
-  add_subdirectory(ToolBox/SOS/Strike)
+  #add_subdirectory(ToolBox/SOS/Strike)

   # Include the dummy c++ include files
   include_directories("pal/inc/rt/cpp")

[seanshpark]

x86/Linux is now builds with repo itself and executes simple Hello World program.
you can test with building like this, (with Ubuntu 16.04 x64)

sudo ./cross/build-rootfs.sh x86 xenial
./build.sh x86 debug cross clang3.8 skipnuget

There are several big issues to solve, such as EH and reverse P/Invoke, and now concentrating on those.