build with latest version of MS Build Tools 2022 (17.10.1) causes call to get_frame() to seg fault

[RussBerg]

This took me a while to figure out but it appears to fail when trying to lock the mutex in start_seq

PyVCAM/src/pyvcam/pvcmodule.cpp

Line 768 in 9398c28

std::lock_guard<std::mutex> lock(g_camInstanceMutex);

The build of the module all completes and installs without error, it just failed to work.

Downgrading to MS Build Tools 2022 17.2.1 fixes this.

This was tested on a windows 11 machine 64bit, you can download the Build Tools for Spring 2022 LTSC | 17.2
from here
https://learn.microsoft.com/en-us/visualstudio/releases/2022/release-history#updating-your-installation-to-a-specific-release

[tomhanak]

I have tried to reproduce the issue with full Visual Studio installation versions 17.9.6 and 17.10.2. Everything seems to work well with Python 3.7 and 3.12.

Could you please provide:

• Minimal code sample? Or did you use one of our tests like tests/seq_mode.py?
• PVCAM version?
• Camera model?

[RussBerg]

• Minimal code sample? Or did you use one of our tests like tests/seq_mode.py?
  I was testing with single_image_polling.py from tests
• PVCAM version?
  2.1.6
• Camera model?
  Retiga R6

I even tried wiping the computer and starting from scratch and got the same result, the only thing that worked was the different version of build tools.

Other info:

• Python: 3.11
• PVCam Release: 3.10.2-7PMQI
• PVCam SDK: PVCam SDK (for PVCam 3.10.1.1) (I tried every conceivable combination of driver and SDK)

[tomhanak]

Unfortunately, I'm still unable to reproduce the issue even with exactly the same environment and camera you have.

But quick Google query has found that something similar is happening to others:
https://stackoverflow.com/a/78599923/6523522

Could you please try the recommended Microsoft solution?
I.e. update VCRedist x86 and x64 (Microsoft Visual C++ 2015-2022 Redistributable) to the latest version. Mine is currently 14.40.33810.

[RussBerg]

tried adding the flag D_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR to extra_compile_args in setup.py and that worked 👍 , doesn't quite make sense to me though that you didn't need the flag if using the same build env (14.40.33810)

[RussBerg]

If you add me I can create a PR for the flag addition or you can, the change was

 elif is_windows:
     extra_compile_args = ['-D_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR']
     include_dirs.append('{}/inc/'.format(pvcam_sdk_path))

[tomhanak]

I'm glad that defining the macro works for you.
But as stated by Microsoft in the VS 2022 17.10 STL release notes, that's a workaround only, not the solution:

... You must follow this rule: When you mix binaries built by different supported versions of the toolset, the Redistributable version must be at least as new as the latest toolset used by any app component.

You can define _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR as an escape hatch.

--- microsoft/STL changelog

Thus we won't apply the proposed patch. It might be reconsidered later when more users report the problem and nothing else helps...

Back to your problem:

• Did you try to update C++ Redistributable?
• Do you run the Python apps on the same computer where you built PyVCAM? (I guess you did not built a wheel for PyVCAM on one PC and did not transfer .whl file to another PC to install it and test there...)
• If you use Python's virtual environment, do you use the same venv for building PyVCAM and running the tests? Can you try to create new clean venv, build PyVCAM and run the test sample?
• Are you sure there isn't older MSVCP140.dll anywhere in the PATH before the system finds the version used to build PyVCAM? (the order of paths in PATH matters...)

[RussBerg]

Thanks for your interest Tom but I can't devote any more time to this problem as the computer in question is being used by others on my team to develop software that uses the camera, so now that I have a work around I/we have to move on.

To answer your questions though:

• Did you try to update C++ Redistributable? yes to 14.40.33810
• Do you run the Python apps on the same computer where you built PyVCAM? Yes
• If you use Python's virtual environment, do you use the same venv for building PyVCAM and running the tests? Yes
• Can you try to create new clean venv, build PyVCAM and run the test sample? Nope, not now anyway maybe down the road when the system is more available
• Are you sure there isn't older MSVCP140.dll anywhere in the PATH before the system finds the version used to build PyVCAM? No Im not sure, I doubt it though because after running into the problem I had the system wiped and OS reinstalled, then built PyVCAM and got the same result

[krixen55]

I am still having this issue and i cannot find anything that works. I downgraded to 17.2 ms build tools, re-ran build/install of pyvcam and it still gets hung up on get_frame(). I also tried the extra arg but that also did not work.

I am using:

python: 3.10.10
pvcam: 3.10.1.1
ms build tools 17.2

[AdvancedResearchConsulting]

Has there been any resolution to this issue? I'm running into this problem as well.
my system crash at this line
self.__exposure_bytes = pvc.start_live(self.__handle, self.__rois, exp_time, self.__mode, buffer_frame_count, stream_to_disk_path)
no error handling, just a complete application closure, indicating a memory access issue or thread.

[sidy3d]

@krixen55 @AdvancedResearchConsulting
As stated by others in the above comments and in the Microsoft documentation the issue is that you are using too old of a version of the MSVC compiler and redistributable compared to what PVCAM was compiled with:

Note: Programs that aren't following the documented restrictions on binary compatibility may encounter null dereferences in mutex machinery. You must follow this rule:

When you mix binaries built by different supported versions of the toolset, the Redistributable version must be at least as new as the latest toolset used by any app component.

You should update your build tools and redistributable to the latest version. This is a better solution than using build tools that are no longer supported (≤17.6) and defining _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR.

[AdvancedResearchConsulting]

@krixen55 I finally had a chance to test your recommentation and want to report that Using build tools 17.12.4 on MSVC 2022 both for my PyVCAM driver compile and my entire application compile still threw the same error.
So - at least for me, this didn't fix anything.

[sidy3d]

Have you tried also updating the MSVC redistributable to the latest version?
https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170