cDAC test results not reported as individual failures

[hoyosjs]

runtime/src/native/managed/cdac/tests/DumpTests/cdac-dump-helix.proj

Line 107 in e5f4006

<_FullCommand>$(_DumpGenCommands) &amp;&amp; $(_DumpInfoCommand) &amp;&amp; $(_TestCommand); _rc=$?; $(_TarCommand); exit $_rc</_FullCommand>

This exit causes premature helix work item exit, and we never report test failures. Runtime follows the following standard:

JIT/Regression/Regression_NoOptimize_d/Regression_NoOptimize_d.sh -usewatcher
test_exit_code=$?
$HELIX_CORRELATION_PAYLOAD/XUnitLogChecker --results-path JIT/Regression/Regression_NoOptimize_d --test-wrapper Regression_NoOptimize_d --dumps-path $HELIX_DUMP_FOLDER
xunitlogchecker_exit_code=$?
result_exit_code=0
if [ $xunitlogchecker_exit_code -ne 0 ]; then result_exit_code=$xunitlogchecker_exit_code; fi
if [ $test_exit_code -ne 0 ]; then result_exit_code=$test_exit_code; fi
set_return() { return $1
}
set_return $result_exit_code

export _commandExitCode=$?

instead we are generating these crazy commands:

mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/heap/r2r/BasicThreads && DOTNET_DbgMiniDumpType=2 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/heap/r2r/BasicThreads/BasicThreads.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/BasicThreads/BasicThreads.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/heap/r2r/GCRoots && DOTNET_DbgMiniDumpType=2 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/heap/r2r/GCRoots/GCRoots.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/GCRoots/GCRoots.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/heap/r2r/ServerGC && DOTNET_DbgMiniDumpType=2 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/heap/r2r/ServerGC/ServerGC.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/ServerGC/ServerGC.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/heap/r2r/TypeHierarchy && DOTNET_DbgMiniDumpType=2 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/heap/r2r/TypeHierarchy/TypeHierarchy.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/TypeHierarchy/TypeHierarchy.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/AsyncContinuation && DOTNET_DbgMiniDumpType=4 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/AsyncContinuation/AsyncContinuation.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/AsyncContinuation/AsyncContinuation.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/ComWrappers && DOTNET_DbgMiniDumpType=4 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/ComWrappers/ComWrappers.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/ComWrappers/ComWrappers.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/LocalVariables && DOTNET_DbgMiniDumpType=4 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/LocalVariables/LocalVariables.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/LocalVariables/LocalVariables.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/MultiModule && DOTNET_DbgMiniDumpType=4 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/MultiModule/MultiModule.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/MultiModule/MultiModule.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/SyncBlock && DOTNET_DbgMiniDumpType=4 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/SyncBlock/SyncBlock.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/SyncBlock/SyncBlock.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/ExceptionHandlingInfo && DOTNET_DbgMiniDumpType=4 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/ExceptionHandlingInfo/ExceptionHandlingInfo.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/ExceptionHandlingInfo/ExceptionHandlingInfo.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/StackWalk && DOTNET_DbgMiniDumpType=4 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/full/r2r/StackWalk/StackWalk.dmp DOTNET_ReadyToRun=1 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/StackWalk/StackWalk.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/full/jit/ExceptionHandlingInfo && DOTNET_DbgMiniDumpType=4 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/full/jit/ExceptionHandlingInfo/ExceptionHandlingInfo.dmp DOTNET_ReadyToRun=0 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/ExceptionHandlingInfo/ExceptionHandlingInfo.dll || true && mkdir -p $HELIX_WORKITEM_PAYLOAD/dumps/local/full/jit/StackWalk && DOTNET_DbgMiniDumpType=4 DOTNET_DbgMiniDumpName=$HELIX_WORKITEM_PAYLOAD/dumps/local/full/jit/StackWalk/StackWalk.dmp DOTNET_ReadyToRun=0 $HELIX_CORRELATION_PAYLOAD/dotnet exec $HELIX_WORKITEM_PAYLOAD/debuggees/StackWalk/StackWalk.dll || true && echo '{"os":"linux","arch":"x64","debuggees":[{"name":"BasicThreads","dumpDir":"heap","r2rDir":"r2r"},{"name":"GCRoots","dumpDir":"heap","r2rDir":"r2r"},{"name":"ServerGC","dumpDir":"heap","r2rDir":"r2r"},{"name":"TypeHierarchy","dumpDir":"heap","r2rDir":"r2r"},{"name":"AsyncContinuation","dumpDir":"full","r2rDir":"r2r"},{"name":"ComWrappers","dumpDir":"full","r2rDir":"r2r"},{"name":"LocalVariables","dumpDir":"full","r2rDir":"r2r"},{"name":"MultiModule","dumpDir":"full","r2rDir":"r2r"},{"name":"SyncBlock","dumpDir":"full","r2rDir":"r2r"},{"name":"ExceptionHandlingInfo","dumpDir":"full","r2rDir":"r2r"},{"name":"StackWalk","dumpDir":"full","r2rDir":"r2r"},{"name":"ExceptionHandlingInfo","dumpDir":"full","r2rDir":"jit"},{"name":"StackWalk","dumpDir":"full","r2rDir":"jit"}]}' > $HELIX_WORKITEM_PAYLOAD/dumps/local/dump-info.json && $HELIX_CORRELATION_PAYLOAD/dotnet exec --runtimeconfig $HELIX_WORKITEM_PAYLOAD/tests/Microsoft.Diagnostics.DataContractReader.DumpTests.runtimeconfig.json --depsfile $HELIX_WORKITEM_PAYLOAD/tests/Microsoft.Diagnostics.DataContractReader.DumpTests.deps.json $HELIX_WORKITEM_PAYLOAD/tests/xunit.console.dll $HELIX_WORKITEM_PAYLOAD/tests/Microsoft.Diagnostics.DataContractReader.DumpTests.dll -xml testResults.xml -nologo; _rc=$?; tar -czf $HELIX_WORKITEM_UPLOAD_ROOT/dumps.tar.gz -C $HELIX_WORKITEM_PAYLOAD/dumps .; exit $_rc
export _commandExitCode=$?

We should generate a similar shell script or dotnet run file instead of these inscrutable single line commands. Then we need to remember the workitem exit code - if our xplat tests depend on the dump tarball we also should consider as part of the exit code.

[hoyosjs]

@rcj1 - this is you don't see results. It's a simple fix.

[dotnet-policy-service]

Tagging subscribers to this area: @steveisok, @tommcdon, @dotnet/dotnet-diag
See info in area-owners.md if you want to be subscribed.