ARM/Linux Coreclr breaks with Roslyn.

[myungjoo]

With some stability patches to libunwind that expands the patch of
https://lists.nongnu.org/archive/html/libunwind-devel/2015-05/msg00002.html
for the case of read as well as write, we get the following error.
(Without it, we just get "segmentation fault" without any information)

sh-3.2# ./corerun csc.exe

Assert failure(PID 9230 [0x0000240e], Thread: 9230 [0x240e]): !"Thread::VirtualUnwindToFirstManagedCallFrame: PAL_VirtualUnwind failed"
    File: /work/dotnet/arm32/coreclr/src/vm/stackwalk.cpp Line: 784
    Image: /sdcard/hf/Release/dirty/corerun

**** MessageBox invoked, title 'corerun - Assert Failure (PID 9230, Thread 9230/0x240e)' ****
  !"Thread::VirtualUnwindToFirstManagedCallFrame: PAL_VirtualUnwind failed"

/work/dotnet/arm32/coreclr/src/vm/stackwalk.cpp, Line: 784

Abort - Kill program
Retry - Debug
Ignore - Keep running


Image:
/sdcard/hf/Release/dirty/corerun

********

Aborted (core dumped)
sh-3.2# 

When we reach that point, the stack looks like this:

Assert failure(PID 12916 [0x00003274], Thread: 12916 [0x3274]): !"Thread::VirtualUnwindToFirstManagedCallFrame: PAL_VirtualUnwind failed"
    File: /work/dotnet/arm32/coreclr/src/vm/stackwalk.cpp Line: 784
    Image: /sdcard/hf/Release/dirty/corerun


Program received signal SIGABRT, Aborted.
0xb6d9f8e6 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
(gdb) bt
#0  0xb6d9f8e6 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
dotnet/coreclr#1  0xb6dadf0e in raise () from /lib/arm-linux-gnueabihf/libc.so.6
dotnet/coreclr#2  0xb6db0766 in abort () from /lib/arm-linux-gnueabihf/libc.so.6
dotnet/coreclr#3  0xb6b62952 in PROCEndProcess (hProcess=0xffffff01, uExitCode=1, 
    bTerminateUnconditionally=1)
    at /work/dotnet/arm32/coreclr/src/pal/src/thread/process.cpp:1338
dotnet/coreclr#4  0xb6b62aa6 in TerminateProcess (hProcess=0xffffff01, uExitCode=1)
    at /work/dotnet/arm32/coreclr/src/pal/src/thread/process.cpp:1254
dotnet/coreclr#5  0xb6300edc in _DbgBreakCheck (
    szFile=0xb6b9847c "/work/dotnet/arm32/coreclr/src/vm/stackwalk.cpp", 
    iLine=784, 
    szExpr=0xb6b9851d "!\"Thread::VirtualUnwindToFirstManagedCallFrame: PAL_VirtualUnwind failed\"", fConstrained=0)
    at /work/dotnet/arm32/coreclr/src/utilcode/debug.cpp:492
dotnet/coreclr#6  0xb63011c4 in _DbgBreakCheckNoThrow (
    szFile=0xb6b9847c "/work/dotnet/arm32/coreclr/src/vm/stackwalk.cpp", 
    iLine=784, 
    szExpr=0xb6b9851d "!\"Thread::VirtualUnwindToFirstManagedCallFrame: PAL_VirtualUnwind failed\"", fConstrained=0)
    at /work/dotnet/arm32/coreclr/src/utilcode/debug.cpp:558
dotnet/coreclr#7  0xb63014da in DbgAssertDialog (
    szFile=0xb6b9847c "/work/dotnet/arm32/coreclr/src/vm/stackwalk.cpp", 
    iLine=784, 
---Type <return> to continue, or q <return> to quit---
    szExpr=0xb6b9851d "!\"Thread::VirtualUnwindToFirstManagedCallFrame: PAL_VirtualUnwind failed\"") at /work/dotnet/arm32/coreclr/src/utilcode/debug.cpp:745
dotnet/coreclr#8  0xb63be9b2 in Thread::VirtualUnwindToFirstManagedCallFrame (
    pContext=0xbeffe720) at /work/dotnet/arm32/coreclr/src/vm/stackwalk.cpp:784
dotnet/coreclr#9  0xb6544c14 in DispatchManagedException (ex=...)
    at /work/dotnet/arm32/coreclr/src/vm/exceptionhandling.cpp:4665
dotnet/coreclr#10 0xb672d474 in PreStubWorker (pTransitionBlock=0xbeffec84, pMD=0xb00c8390)
    at /work/dotnet/arm32/coreclr/src/vm/prestub.cpp:1040
dotnet/coreclr#11 0xb6557768 in ThePreStub () at asmhelpers.S:787
dotnet/coreclr#12 0x7dd8252e in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)

The problem is that the saved LR at Frame 11 (ThePreStub) is BROKEN (the PC of Frame 12 is incorrect)

(gdb) info frame 11
Stack frame at 0xbeffec64:
 pc = 0xb6557768 in ThePreStub (asmhelpers.S:787); saved pc = 0x7dd8252e
 called by frame at 0xbeffec64, caller of frame at 0xbeffec40
 source language asm.
 Arglist at 0xbeffec40, args: 
 Locals at 0xbeffec40, Previous frame's sp is 0xbeffec64
 Saved registers:
  r4 at 0xbeffec40, r5 at 0xbeffec44, r6 at 0xbeffec48, r7 at 0xbeffec4c,
  r8 at 0xbeffec50, r9 at 0xbeffec54, r10 at 0xbeffec58, r11 at 0xbeffec5c,
  lr at 0xbeffec60
(gdb) p/x *0xbeffec60
$1 = 0x7dd8252f
(gdb) p/x *0x7dd8252f
Cannot access memory at address 0x7dd8252f
(gdb) 

Looking at the assembly of ThePreStub: (note that PC is located elsewhere in the following instance because I made a watchpoint at 0xbeffec60 this time, which means that 0xbeffec60 is being overwritten by "+10 vstmia r6, {d0-d7]")

<ThePreStub>                         push      {r0, r1, r2, r3}            �
<ThePreStub+2>                       stmdb     sp!, {r4, r5, r6, r7, r8, r9�
<ThePreStub+6>                       sub       sp, dotnet/runtime#3883 ; 0x44              �
<ThePreStub+8>                       add       r6, sp, #0                  �
<ThePreStub+10>                      vstmia    r6, {d0-d7}                 �
>>> <ThePreStub+14>                      push      {r0}                        �
<ThePreStub+16>                      add       r0, sp, dotnet/coreclr#4                  �
<ThePreStub+18>                      tst.w     r0, dotnet/coreclr#7                      �
<ThePreStub+22>                      pop       {r0}                        �
<ThePreStub+24>                      beq.n     0xb6557760 <ThePreStub+30>  �
<ThePreStub+26>                      movs      r0, r0                      �
<ThePreStub+28>                      udf       dotnet/coreclr#1                          �
<ThePreStub+30>              add       r0, sp, dotnet/runtime#3883     ; 0x44              �
<ThePreStub+32>              mov       r1, r12                             �
<ThePreStub+34>              bl        0xb672d098 <PreStubWorker(Transition�
<ThePreStub+38>              mov       r12, r0                             �
<ThePreStub+40>              add       r6, sp, #0                          �
<ThePreStub+42>              vldmia    r6, {d0-d7}                         �
<ThePreStub+46>              add       sp, dotnet/runtime#3883 ; 0x44                      �
<ThePreStub+48>              ldmia.w   sp!, {r4, r5, r6, r7, r8, r9, r10, r�
<ThePreStub+52>              pop       {r0, r1, r2, r3}                    �
<ThePreStub+54>    
(gdb) p/x $d4
$28 = {u8 = {0x2f, 0x25, 0xd8, 0x7d, 0xba, 0x6f, 0xc9, 0xf}, u16 = {0x252f,
    0x7dd8, 0x6fba, 0xfc9}, u32 = {0x7dd8252f, 0xfc96fba},
  u64 = 0xfc96fba7dd8252f, f32 = {0xffffffff, 0x0}, f64 = 0x0}
(gdb) p/x $r6
$29 = 0xbeffec40

Later on, when CORECLR breaks with the exception above, the stored LR of ThePreStub is the value of stored D4 above. It is too strange to understand for now, but there seems to be some sort of buffer overrun, incorrect memory addressing, or sort of such things in coreclr regarding the assembly codes.

My first guess is on the constant "#68" or 0x44, which seems to assume that storing {r4, r5, r6, r7, r8, r9, r10, r11, lr}, and {d0-d7} requires 11 words. Because D registers are "double words", it requires 19 words. However, this doesn't add up exactly because lr is stored where d8 is going to be located if it did vstmia r6, {d0-d8} instead of vstmia r6, {d0-d7}.

I'm currently looking for the code that emitted sub sp, dotnet/runtime#3883.

ps. I'm just using Roslyn as the first "serious" test case for ARM/Linux coreclr. I'm not actually trying to build things inside ARM/Linux box.

[myungjoo]

Another interesting point is that it stores {r4-r11, sp} with stmdb while restoring them with ldmia.w and stores {d0-d7} with vstmia and restores with vldmia. Doesn't this seem to break the symmetricity?

Ah.. the "!" mark does the trick.. now, I have no idea why LR stored in stack is overwritten with some irrelevant values. (strokethrough the guesses before this)

[myungjoo]

Very Strange...

If I compile the following code with all reference library correctly in corefx with mcs, it works.
If I compile it with mcs without any options (no /nostdlib /noconfig /r:ref/...), it shows the same error.

Thus, it is possible that

• Roslyn build (cibuild.sh) refers dlls not recognized by the corefx of today.
  and
• Coreclr today loses (breaks the stack and all sanity stuff) it if the given .exe refers DLLs slightly not matching...

using System;
using System.IO;

namespace filehello
{
    class hello
    {
        static void Main(string[] args)
    {
        FileStream F = new FileStream("test.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite);
        for (int i = 0; i <= 20; i++)
        {
            F.WriteByte((byte)i);
        }

        F.Position = 0;
        for (int i = 0; i <= 20; i++)
        {
            Console.Write(F.ReadByte() + " ");
            }
//      F.Close();
        Console.ReadKey();
    }
    }

}

Anyway, Roslyn's csc.exe gives the same error even all the dlls except for mscorlib.dll are from its build :(

[jkotas]

I have no idea why LR stored in stack is overwritten with some irrelevant values.

It may be a JIT bug.

[myungjoo]

Running GDB for this issue has discovered a bug in GDB itself. I'll continue working on this issue after patching up GDB.

[manu-st]

@myungjoo What kind of bug did you encounter with gdb?

[myungjoo]

The GDB issue is resolved by upgrading the GDB version to 7.10.1: https://sourceware.org/ml/gdb-patches/2015-03/msg00585.html

[manu-st]

@myungjoo I've been trying to debug this on my end with my code but this is hard to debug as I'm debugging this on a large program. Have you been able to reduce the above program to a bare minimum?

[myungjoo]

@manu-silicon No, I've observed this only with csc.exe (Roslyn, single managed thread so far in my case of errors) or xunit.console.netcore.exe (mult managed threads) supplied with a test dll. A simple app but executing for a long time (allocating (incurring GC) a lot of arrays, deeply expanding recursive calls) is running well.

[manu-st]

@myungjoo This is a very similar situation. I was able to figure out that the problem occurs shortly after a new thread is being launched. I'm changing strategy here. I'm going over the CoreCLR tests and look at the failures. From there I'll decide which one to fix.

[myungjoo]

I'm seeing another issues related with this: having a breakpoint at asmhelper.S (bl PreStubWorker or ThePreStub), it seems this strange value of saved LR is quite common. I just don't know how it continues to work normally:

Breakpoint 6, ThePreStub () at asmhelpers.S:799
799 in asmhelpers.S
(gdb) bt
#0  ThePreStub () at asmhelpers.S:799
dotnet/coreclr#1  0x00000006 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) info frame
Stack level 0, frame at 0xbeffecb4:
 pc = 0xb650b6c4 in ThePreStub (asmhelpers.S:799); saved pc = 0x6
 called by frame at 0xbeffecb4
 source language asm.
 Arglist at 0xbeffec38, args: 
 Locals at 0xbeffec38, Previous frame's sp is 0xbeffecb4
 Saved registers:
  r0 at 0xbeffeca4, r1 at 0xbeffeca8, r2 at 0xbeffecac, r3 at 0xbeffecb0,
  r4 at 0xbeffec80, r5 at 0xbeffec84, r6 at 0xbeffec88, r7 at 0xbeffec8c,
  r8 at 0xbeffec90, r9 at 0xbeffec94, r10 at 0xbeffec98, r11 at 0xbeffec9c,
  lr at 0xbeffeca0
(gdb) p/x 0xbeffeca0
$27 = 0xbeffeca0
(gdb) p/x *0xbeffeca0
$28 = 0x7
(gdb) n
PreStubWorker (pTransitionBlock=0xbefff378, pMD=0xb27002a4)
    at /usr/src/debug/coreclr-0.0.1/src/vm/prestub.cpp:954
954 {
(gdb) n
955     PCODE pbRetVal = NULL;
(gdb) n
957     BEGIN_PRESERVE_LAST_ERROR;
(gdb) until
964     MAKE_CURRENT_THREAD_AVAILABLE();
(gdb) n 10
988     CONSISTENCY_CHECK(GetAppDomain()->CheckCanExecuteManagedCode(pMD));
(gdb) n
997     MethodTable *pDispatchingMT = NULL;
(gdb) n 10
1059        return pbRetVal;
(gdb) n
ThePreStub () at asmhelpers.S:801
801 asmhelpers.S: No such file or directory.
(gdb) n
803 in asmhelpers.S
(gdb) n
7   in asmhelpers.S
(gdb) n
ThePreStub () at asmhelpers.S:1
1   in asmhelpers.S
(gdb) n
ThePreStub () at asmhelpers.S:9
9   in asmhelpers.S
(gdb) n
ThePreStub () at asmhelpers.S:804
804 in asmhelpers.S
(gdb) n
0xb50abce8 in ?? ()
(gdb) bt
#0  0xb50abce8 in ?? ()
dotnet/coreclr#1  0xb50b253e in ?? ()

With the saved return address of 0x7, it has finally returned to the managed code. Although that 0xb50abce8 is a managed frame, at least the native frame of ThePreStub should aware of the return address correctly. Maybe ThePreStub's mechanism to bring LR up correctly at the end of its execution is not compatible with both LIBUNWIND/or/exceptionhandler and GDB?

Breaking at the start of ThePreStub, GDB loses the track of the frame after returning from PreStubWorker. I'm not sure this is because of GDB or coreclr. I'll post again if I got some clue.

Breakpoint 7, ThePreStub () at asmhelpers.S:22
22  asmhelpers.S: No such file or directory.
(gdb) x/i $pc
=> 0xb650b6a2 <ThePreStub>: push    {r0, r1, r2, r3}
(gdb) info frame
Stack level 0, frame at 0xbeffecf8:
 pc = 0xb650b6a2 in ThePreStub (asmhelpers.S:22); saved pc = 0xb50b1c34
 called by frame at 0xbeffecf8
 source language asm.
 Arglist at 0xbeffecf8, args: 
 Locals at 0xbeffecf8, Previous frame's sp is 0xbeffecf8
(gdb) p/x $lr
$39 = 0xb50b1c35
(gdb) n
1   in asmhelpers.S
(gdb) x/i $pc
=> 0xb650b6a4 <ThePreStub+2>:   
    stmdb   sp!, {r4, r5, r6, r7, r8, r9, r10, r11, lr}
(gdb) n
27  in asmhelpers.S
(gdb) x/i $pc
=> 0xb650b6a8 <ThePreStub+6>:   sub sp, dotnet/runtime#3883 ; 0x44
(gdb) p/x $lr
$40 = 0xb50b1c35
(gdb) info frame
Stack level 0, frame at 0xbeffecf8:
 pc = 0xb650b6a8 in ThePreStub (asmhelpers.S:27); saved pc = 0xb50b1c34
 called by frame at 0xbeffecf8
 source language asm.
 Arglist at 0xbeffecc4, args: 
 Locals at 0xbeffecc4, Previous frame's sp is 0xbeffecf8
 Saved registers:
  r0 at 0xbeffece8, r1 at 0xbeffecec, r2 at 0xbeffecf0, r3 at 0xbeffecf4,
  r4 at 0xbeffecc4, r5 at 0xbeffecc8, r6 at 0xbeffeccc, r7 at 0xbeffecd0,
  r8 at 0xbeffecd4, r9 at 0xbeffecd8, r10 at 0xbeffecdc, r11 at 0xbeffece0,
  lr at 0xbeffece4
(gdb) p/x *0xbeffece4
$41 = 0xb50b1c35
(gdb) x/i $lr-2
   0xb50b1c33:  blx r3
(gdb) p/x $r3
$42 = 0xb50ab6f5
(gdb) p/x *$r3
$43 = 0x4f8df46
(gdb) p/x *($r3-1)
$44 = 0xf8df46fc
(gdb) p/x *($r3-2)
$45 = 0xdf46fcb6
(gdb) p/x *($r3-3)
$46 = 0x46fcb650
(gdb) p/x *($r3+1 
A syntax error in expression, near `'.
(gdb) p/x *($r3+1)
$47 = 0xf004f8df
(gdb) p/x *($r3+2)
$48 = 0xbbf004f8
(gdb) n
794 in asmhelpers.S
(gdb) x/i $pc
=> 0xb650b6aa <ThePreStub+8>:   add r6, sp, #0
(gdb) n
34  in asmhelpers.S
(gdb) x/i $pc
=> 0xb650b6b0 <ThePreStub+14>:  push    {r0}
(gdb) n
796 in asmhelpers.S
(gdb) x/i $pc
=> 0xb650b6c0 <ThePreStub+30>:  add r0, sp, dotnet/runtime#3883 ; 0x44
(gdb) n
797 in asmhelpers.S
(gdb) x/i $pc
=> 0xb650b6c2 <ThePreStub+32>:  mov r1, r12
(gdb) n

Breakpoint 6, ThePreStub () at asmhelpers.S:799
799 in asmhelpers.S
(gdb) x/i $pc
=> 0xb650b6c4 <ThePreStub+34>:  
    bl  0xb66e7e38 <PreStubWorker(TransitionBlock*, MethodDesc*)>
(gdb) u
PreStubWorker (pTransitionBlock=0xbefff378, pMD=0xbeffed74)
    at /usr/src/debug/coreclr-0.0.1/src/vm/prestub.cpp:954
954 {
(gdb) u
955     PCODE pbRetVal = NULL;
(gdb) u
957     BEGIN_PRESERVE_LAST_ERROR;
(gdb) n 10
986     pMD->CheckRestore();
(gdb) n
988     CONSISTENCY_CHECK(GetAppDomain()->CheckCanExecuteManagedCode(pMD));
(gdb) n
997     MethodTable *pDispatchingMT = NULL;
(gdb) n
999     if (pMD->IsVtableMethod())
(gdb) n
1040        GCX_PREEMP_THREAD_EXISTS(CURRENT_THREAD);
(gdb) n
1041        pbRetVal = pMD->DoPrestub(pDispatchingMT);
(gdb) n
1043        UNINSTALL_UNWIND_AND_CONTINUE_HANDLER;
(gdb) n
1044        UNINSTALL_MANAGED_EXCEPTION_DISPATCHER;
(gdb) n
1050            ThePreStubPatch();
(gdb) n
1053        pPFrame->Pop(CURRENT_THREAD);
(gdb) n
1055        POSTCONDITION(pbRetVal != NULL);
(gdb) n
1057        END_PRESERVE_LAST_ERROR;
(gdb) n
1059        return pbRetVal;
(gdb) n
ThePreStub () at asmhelpers.S:801
801 asmhelpers.S: No such file or directory.
(gdb) p/x $r0
$49 = 0xb50b2591
(gdb) info frame
Stack level 0, frame at 0xbeffecfc:
 pc = 0xb650b6c8 in ThePreStub (asmhelpers.S:801); saved pc = 0xb2700438
 Outermost frame: Cannot access memory at address 0x68633e
 source language asm.
 Arglist at 0xbeffec80, args: 
 Locals at 0xbeffec80, Previous frame's sp is 0xbeffecfc
 Saved registers:
  r0 at 0xbeffecec, r1 at 0xbeffecf0, r2 at 0xbeffecf4, r3 at 0xbeffecf8,
  r4 at 0xbeffecc8, r5 at 0xbeffeccc, r6 at 0xbeffecd0, r7 at 0xbeffecd4,
  r8 at 0xbeffecd8, r9 at 0xbeffecdc, r10 at 0xbeffece0, r11 at 0xbeffece4,
  lr at 0xbeffece8
(gdb) p/x $0xbeffece8
$50 = Value can't be converted to integer.
(gdb) p/x *0xbeffece8
$51 = 0xb2700438
(gdb) p/x *0xbeffece4
$52 = 0xb50b1c35

This was incurred by CHECK_STACK_ALIGNMENT macro, which confuses GDB. I don't think this may confuse others in runtime; however, I'm testing coreclr with the macro nullified.

[manu-st]

@myungjoo I made the following simple C# program that crashes due to a jump to 0x0 in DispatchManagedException.

using System;
using System.Runtime.CompilerServices;

public class Bug426480
{
    public static int s_i = 0;
    public static bool first_time_through = true;

    public static void foo(Exception ex)
    {
        throw ex;
    }

    public static int Main(String[] args)
    {
        try
        {
            foo(new Exception());
        }
        catch (Exception)
        {
        }
        return 1;
    }
}

Debugging it, I came up with the following stack trace:

#0  RtlRestoreContext () at context2.S:337
dotnet/coreclr#1  0xb5cd8eaa in ExceptionTracker::ResumeExecution (pContextRecord=0xbe7fdf40, pExceptionRecord=0x0) at /home/local/Microsoft/coreclr/src/vm/exceptionhandling.cpp:3983
dotnet/coreclr#2  0xb5cd787e in ProcessCLRException (pExceptionRecord=0xbe7fe754, MemoryStackFp=3196054032, pContextRecord=0xbe7fdf40, pDispatcherContext=0xbe7fdf0c)
    at /home/local/Microsoft/coreclr/src/vm/exceptionhandling.cpp:1153
dotnet/coreclr#3  0xb5cdcd48 in UnwindManagedExceptionPass2 (ex=..., unwindStartContext=0xbe7fe240) at /home/local/Microsoft/coreclr/src/vm/exceptionhandling.cpp:4424
dotnet/coreclr#4  0xb5cdd09c in UnwindManagedExceptionPass1 (ex=..., frameContext=0xbe7fe458) at /home/local/Microsoft/coreclr/src/vm/exceptionhandling.cpp:4576
dotnet/coreclr#5  0xb5cdd474 in DispatchManagedException (ex=...) at /home/local/Microsoft/coreclr/src/vm/exceptionhandling.cpp:4684
dotnet/coreclr#6  0xb5c15946 in IL_Throw (obj=0xb0e1cb6c) at /home/local/Microsoft/coreclr/src/vm/jithelpers.cpp:5380

and the failure occurs in context2.S line 201 where we restore the pc register with a null value in my case. Here is the extract of context2.S:

    mov r12, r0 // ideally we would ldmia r0, {r0-r12, sp, lr, pc} here, but that isn't supported on new clang
    add r12, CONTEXT_R0 // so we'll burn r12 as the IPC reg for now -- TODO: is this ok?
    ldm r12, {r0-r11}
    add r12, (CONTEXT_Sp - CONTEXT_R0)
    ldr sp, [r12]
    ldr lr, [r12, #(CONTEXT_Lr - CONTEXT_Sp)]
    ldr pc, [r12, #(CONTEXT_Pc - CONTEXT_Sp)]     <----- Faulty line

[myungjoo]

@manu-silicon In my Linux/ARM system, your code does not incur stack breaking but does not work as expected:

sh-3.2# ./corerun manu-silicon-example.exe

Unhandled Exception: System.Exception: Exception of type 'System.Exception' was thrown.
   at Bug426480.Main(String[] args)
Aborted (core dumped)
sh-3.2# 

The trace of the core dumped:

(gdb) bt
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:44
dotnet/coreclr#1  0xb6d788fe in __GI_raise (sig=sig@entry=6)
    at ../sysdeps/unix/sysv/linux/raise.c:55
dotnet/coreclr#2  0xb6d79612 in __GI_abort () at abort.c:89
dotnet/coreclr#3  0xb6b2469e in PROCEndProcess (hProcess=0xffffff01, uExitCode=1, 
    bTerminateUnconditionally=1)
    at /usr/src/debug/coreclr-0.0.1/src/pal/src/thread/process.cpp:1338
dotnet/coreclr#4  0xb6b247f2 in TerminateProcess (hProcess=0xffffff01, uExitCode=1)
    at /usr/src/debug/coreclr-0.0.1/src/pal/src/thread/process.cpp:1254
dotnet/coreclr#5  0xb64f89e2 in UnwindManagedExceptionPass1 (ex=..., frameContext=0xbeffe738)
    at /usr/src/debug/coreclr-0.0.1/src/vm/exceptionhandling.cpp:4610
dotnet/coreclr#6  0xb64f8ca8 in DispatchManagedException (ex=...)
    at /usr/src/debug/coreclr-0.0.1/src/vm/exceptionhandling.cpp:4684
dotnet/coreclr#7  0xb644831a in IL_Throw (obj=0xb2513478)
    at /usr/src/debug/coreclr-0.0.1/src/vm/jithelpers.cpp:5380
dotnet/coreclr#8  0xb1ae0874 in ?? ()

[manu-st]

I'm debugging against a version of CoreCLR where FEATURE_STUBS_AS_IL is disabled but I do not think it would matter here. Are you running on an emulator? I'm running it onto a Nvidia Jetson TK1.

[myungjoo]

I am testing CoreCLR in a Samsung SM-Z300H phone with custom built Linux distro to support both gnueabihf and gnueabi.

I got caught in a regression in my local repo right now while improving build scripts :(

[manu-st]

I found that my disassembly:

0xb6319b9a <RtlRestoreContext+68>       ldr.w  sp, [r12]
0xb6319b9e <RtlRestoreContext+72>       ldr.w  lr, [r12, dotnet/runtime#3912] ; 0x74
0xb6319ba2 <RtlRestoreContext+76>       ldr.w  pc, [r12, dotnet/runtime#3913] ; 0x78

It is generating #3912 and #3913, but it is certainly incorrect. If I set lr and pc manually from the saved value in the context, the execution proceeds properly. I tried to change the assembly to generate the proper code, but it doesn't do what I want. I'll continue tomorrow.

@myungjoo Could you disassembly RtlRestoreContext from gdb and see if you get the same value as I do? Thanks.