Adding debug logs for OleDb test failures

[saurabh500]

Adding some diagnostics to get more information from the SEHException to debug OleDb Test failures in OuterLoop.
Related issue at https://github.com/dotnet/corefx/issues/37823

[maryamariyan]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 4 pipeline(s).

[maryamariyan]

Hoping to see repro on the PR :)

[maryamariyan]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 4 pipeline(s).

[danmoseley]

Everything you are logging here should appear in the output of exception.ToString() shouldn't it? Why doesn't the test/test harness write that out whenever the test fails? Or is it eaten by design for some reason?

[saurabh500]

@danmosemsft I was under the impression that it should be written as part of the ToString message. I sent out this PR to make sure that even if the details are not being written by the test harness, then we can capture it in the logs.
If there are failures after this change, and there is nothing in the logs about the internal details about the SEHException, then this issue would be much harder to debug.

I am open to any other ideas to chase down the underlying problem here.

[danmoseley]

@saurabh500 I didn't mean to suggest this wasn't a good change. Just that if we weren't logging all this already we should fix that, and as far as I can tell we should be already. Let's see what you catch..

[saurabh500]

Got it. Yeah, lets see what we discover. Will keep you posted.

[danmoseley]

https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F37825~2Fmerge/test~2Ffunctional~2Fcli~2Finnerloop~2F/20190520.4/workItem/System.Data.OleDb.Tests
https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F37825~2Fmerge/test~2Ffunctional~2Fcli~2Finnerloop~2F/20190520.4/workItem/System.Data.OleDb.Tests/wilogs

AV on Windows 8.1 in Prepare_InsertMultipleItems_UseTableDirectToVerify

0xC0000005
   at System.Data.Common.UnsafeNativeMethods+ICommandWithParameters.SetParameterInfo(IntPtr, IntPtr[], System.Data.OleDb.tagDBPARAMBINDINFO[])
   at System.Data.OleDb.OleDbCommand.ApplyParameterBindings(ICommandWithParameters, System.Data.OleDb.tagDBPARAMBINDINFO[])
   at System.Data.OleDb.OleDbCommand.CreateAccessor()
   at System.Data.OleDb.OleDbCommand.InitializeCommand(System.Data.CommandBehavior, Boolean)
   at System.Data.OleDb.OleDbCommand.ExecuteCommand(System.Data.CommandBehavior, System.Object ByRef)

Same on Windows 7 plus a different test failing with COMException

    System.Data.OleDb.Tests.OleDbCommandTests.Parameters_AddNullParameter_Throws [FAIL]
      System.Runtime.InteropServices.COMException : Bad variable type. (0x80020008 (DISP_E_BADVARTYPE))
      Stack Trace:
           at System.Data.Common.SafeNativeMethods.VariantClear(IntPtr pObject)
        /_/src/System.Data.OleDb/src/DbPropSet.cs(141,0): at System.Data.OleDb.DBPropSet.ReleaseHandle()
        F:\workspace\_work\1\s\src\System.Private.CoreLib\shared\System\Runtime\InteropServices\SafeHandle.cs(211,0): at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
        /_/src/System.Data.OleDb/src/OleDbDataReader.cs(1850,0): at System.Data.OleDb.OleDbDataReader.GetPropertyOnRowset(Guid propertySet, Int32 propertyID)
        /_/src/System.Data.OleDb/src/OleDbDataReader.cs(1824,0): at System.Data.OleDb.OleDbDataReader.GetPropertyValue(Int32 propertyId)
        /_/src/System.Data.OleDb/src/OleDbDataReader.cs(1510,0): at System.Data.OleDb.OleDbDataReader.CreateAccessors(Boolean allowMultipleAccessor)
        /_/src/System.Data.OleDb/src/OleDbDataReader.cs(298,0): at System.Data.OleDb.OleDbDataReader.BuildMetaInfo()
        /_/src/System.Data.OleDb/src/OleDbCommand.cs(647,0): at System.Data.OleDb.OleDbCommand.ExecuteReaderInternal(CommandBehavior behavior, String method)
        /_/src/System.Data.OleDb/src/OleDbCommand.cs(586,0): at System.Data.OleDb.OleDbCommand.ExecuteReader(CommandBehavior behavior)
        /_/src/System.Data.OleDb/tests/OleDbCommandTests.cs(158,0): at System.Data.OleDb.Tests.OleDbCommandTests.<>c.<Parameters_AddNullParameter_Throws>b__7_0(OleDbCommand command, String tableName)
        /_/src/System.Data.OleDb/tests/OleDbCommandTests.cs(234,0): at System.Data.OleDb.Tests.OleDbCommandTests.RunTest(Action`2 testAction, String memberName)
        /_/src/System.Data.OleDb/tests/OleDbCommandTests.cs(151,0): at System.Data.OleDb.Tests.OleDbCommandTests.Parameters_AddNullParameter_Throws()

[saurabh500]

I have a local repro and the problem seems to lie with the side by side install of the JET and ACE oledb driver.
ACE driver is x64 and JET driver is x86.

While file reads succeed with SxS installation, the file writes lead to SEHException.
The side by side installation was forced on the CI VMs using /passive switch for the x84 installer.

I uninstalled the x64 and x86 version and I can see that the tests that were failing earlier are now passing.

To fix the test runs the following need to be done

1. The drivers should be uninstalled on the CI machine and then the x64 driver should be reinstalled.
2. The tests should be disabled on x86 (assuming that the OS being used for testing is the same for x64 and x86)
3. The tests should only check for the presence of the Microsoft.ACE.OLEDB.12.0 driver and should be skipped in case the CI has both JET and ACE drivers (x86 and x64 respectively), or probably the tests should fail if both drivers are detected in the CI, on the same OS.

Let me know if this is an acceptable approach. I can make the code changes in the test.
Also where can I request changes in the CI OS?

[saurabh500]

@danmosemsft I think the AVs maybe a side effect of both drivers being installed.

[danmoseley]

Will customers hit this? Or only if they use /passive?

@maryamariyan can help adjust CI setup

[maryamariyan]

cc: @MattGal
To reiterate:
JET driver was originally requested to be added as part of https://github.com/dotnet/core-eng/issues/6125
If I am understanding correctly, here are the action items:

• to reopen https://github.com/dotnet/core-eng/issues/6125 and request to remove JET driver from the CI machines (refer to step 1 of Adding debug logs for OleDb test failures #37825 (comment)).
• to skip tests on x86 platform
• to skip tests if both JET and ACE are installed on the machine

@saurabh500 If I am missing any other action item or misunderstood anything above please let me know.

[saurabh500]

@maryamariyan LGTM
@danmosemsft The customers can see this if they have the JET driver installed with /passive option. However coexistence of both drivers isn't a supported scenario based on the threads I am seeing online.

[MattGal]

@maryamariyan / @saurabh500 : can you decide exactly which course of action you'd like to take and make a PR in https://dev.azure.com/dnceng/internal/_git/dotnet-helix-machines/ to selectively undo the changes? It'd be a great test of whether folks can self-serve update their machine configurations.

See the original PR to get an idea where to update.

[danmoseley]

Will that fix ystem.Runtime.InteropServices.COMException : Bad variable type. (0x80020008 (DISP_E_BADVARTYPE)) also?

[saurabh500]

@danmosemsft

Will that fix ystem.Runtime.InteropServices.COMException : Bad variable type. (0x80020008 (DISP_E_BADVARTYPE)) also?

Not sure. I dont have the error message in the repro that I have. However all outer loop tests are passing with a singe driver.
Let's tackle the problem of running the tests on x64 with a single driver in the OS, and then if the issues that you have found, continue to occur then I could investigate them.

[maryamariyan]

Closing as #38024 got merged.