Getting vim-ipython to run on Windows 7 using Anaconda2.0.0 (32-bit) (Python2.7, IPython2.1.0, etc. distro)

[10p10110111xxi11p00100100]

Hello, another Windows user here with Windows-related (?) problems,

In short: installation of vim-ipython is successful, but calling :IPython fails at specific Python libraries (ctypes, zmq?). Similar bugs were reported before #34, #21, #20, but I am not certain if this is the same kind of bug or a problem of the Python distribution (Anaconda) and incorrect packaging of IPython, zmq, etc.

OS: Win7 (64 bit)
Vim: gVim 7.4 (2013 Aug 10, compiled Aug 10 2013 14:38:33)
Python stack: Python 2.7.6 |Anaconda 2.0.0 (32-bit)| (default, May 27 2014, 15:05:55) [MSC v.1500 32 bit (Intel)]
IPython: IPython 2.1.0
IPython and Python are installed together by Anaconda Python distribution and registered in Windows Registry.
vim-ipython: installed with Vundle (automatically gets newest Git repo or updates to the newest; based on Pathogen)

Checking if Windows Registry is correctly used:
Running "import os; print os.__file__" in IPython or Python.exe (on CMD.exe) returns:

C:\Anaconda\lib\os.pyc

Running ":py import os; print os.__file__" on gVim commandline returns:

C:\Anaconda\lib\os.pyc

The problem occurs when opening a Python file (.py) with gVim and then running ":IPython" on gVim commandline:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 83, in km_from_string
  File "C:\Anaconda\lib\site-packages\IPython\kernel\__init__.py", line 4, in <module>
    from . import zmq
  File "C:\Anaconda\lib\site-packages\IPython\kernel\zmq\__init__.py", line 14, in <module>
    check_for_zmq('2.1.11', 'IPython.kernel.zmq')
  File "C:\Anaconda\lib\site-packages\IPython\utils\zmqrelated.py", line 35, in check_for_zmq
    import zmq
  File "C:\Anaconda\lib\site-packages\zmq\__init__.py", line 41, in <module>
    _libzmq = ctypes.cdll.LoadLibrary(bundled[0])
  File "C:\Anaconda\Lib\ctypes\__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "C:\Anaconda\Lib\ctypes\__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
WindowsError: [Error 126] The specified module could not be found

Running Python.exe on CMD.exe:

>>>import ctypes
>>> ctypes.__version__
'1.1.0'

>>> import zmq
>>> zmq.__version__
'14.3.0'
>>> zmq.__path__
['C:\\Anaconda\\lib\\site-packages\\zmq']

>>> import IPython
>>> IPython.__version__
'2.1.0'
>>> IPython.__path__
['C:\\Anaconda\\lib\\site-packages\\IPython']

What can I do to be able to run :IPython in gVim commandline? Please be detailed, as I do not have a lot of experience installing and configuring Microsoft Visual Studio or Windows SDK. I could install Windows SDK and get mt.exe running, but it returned only errors when trying to "sign" the files. From previous issue reports it is not clear which files (gVim.exe, python27.dll, .pyd, .pyc) should be "signed" with which signatures. Is there a different 32 and 64 bit version of mt.exe?

[ivanov]

unfortunately I don't have enough Windows experience to help here: you can try mentioning this issue in the other windows issues to see if one of the helpful participants there can lend you a hand and walk you through it.

[ivanov]

I added a new red tag to this issue, maybe that'll draw someone's attention, too

[dhj-io]

I am having the same problem, but have dug a little deeper -- I have confirmed that when loading libzmq (in zmq/__init__.py) the the following file is specified as the target to load: 'c:\\Python27\\lib\\site-packages\\zmq\\libzmq.pyd' ... This is the target whether running zmq from python directly or through vim-ipython. Also, the os.path.isfile() function confirms the file is present whether loading from either environment. So it seems like the same ctypes.cdll.LoadLibrary() function is working from python but not when loading from vim-ipython. I will dig one step deeper and look at ctypes/__init__.py.

UPDATE:
Apparently this is a dependency issue, and not a direct loading issue. Error 126 is described at stackoverflow ... So, it's one of those items that zmq depends on that isn't being found, but that happens in C land so it is somewhat obscure.

One interesting point. When running python through the command line python loads the following libraries:
kernel32
msvcrt
msvcr90.dll
user32
mvscr90.dll (yes, twice)
python27

However, when running through vim it loads these libraries (differences in bold):
kernel32
msvcrt
msvcr90.dll
shell32

ipython run from the command line loads a combination of all of these libraries -- the "vim environment" libraries through shell32, then the user32, msvcr90.dll and python27. Next step -- see what is loading these first.

UPDATE:

It is the second msvcr90.dll that is causing the problem. I guess when it is imported from python there is a static version of the library that is pre-loaded?
Definitely related to issue: #21
I used the x86 dependency walker to confirm that libzmq.pyd depends on msvcp90.dll and msvcr90.dll

Two attempts to resolve:
dig up an msvcp90 and msvcr90 dll files from an old winsxs directory. -- not detected in system or system32 directories, and in SysWOW64 gives errors
Windows popup: R6034 error, attempt to load C runtime library incorrectly.
Within vim: Error 1114, a dynamic link library (DLL) initialization routine failed

Replace libzmq.pyd with a zmq dll directly from zmq -- got the x86 version. This has ZMQ linked against several different versions of the VC libraries. The binaries work correctly and the dependency walker identifies that v110 and v100 do not have immediate dependency errors.

Placing either of these dll files in site-packages/zmq instead of the libzmq.pyd (keeping the same name as the original libzmq-v1[1|0]0-mt-4_0_4.dll) allows them to be loaded as the "official" libzmq library. Unfortunately both python and vim give errors when this is attempted.

In python DLL load failed: The specified module could not be found -- although it does attempt to load the dll.
In vim: No module named zmq.blockingkernelmanager (same as if zmq is not installed)

Is this what is referred to as DLL hell?

UPDATE: ... last one for me. Also tried the Windows Visual C++ Redistributable packages (regular and SP1) and no change at all. It's as if dependency walker and the vim environment just don't register those new dlls. Python loads zmq just fine. Curiously, dependency walker shows the Python.exe is finding MSVCR90.dll in the winsxs directory (same one that I tried to copy to SysWOW64 when it gave errors about loading incorrectly), but no MSVCP90.dll dependency as in the libzmq.py file.

If we can get libzmq to recognize the C:\Windows\winsxs directory as a source for dll files then it might be able to load all of the MSVC?90.dll files that it needs. Of course it could be that those are specific to the python.exe file against which they are linked (and why copying them to other dll directories didn't work).

Well, this will be my last update for this extremely long comment. Hopefully someone else will take the time to read it and know enough about python, windows (and zmq?) to identify what is going wrong.

I am using updated versions:
ctypes == '1.1.0'
zmq == '14.4.1'
IPython == '2.3.1'
vim == '7.4.430'
python == '2.7.8' (x86 32 bit)

[dhj-io]

So, the problem is very well defined by @edouardp in problem #21 ... The python.exe includes a dependency that allows the correct .dll to be loaded without a problem. If you edit the dependency specification to your vim.exe or gvim.exe file it works without a problem (of course there is generally no reason for vim to include that dependency). Here is a gist that compares the manifest specifications: https://gist.github.com/dhj-io/f18771fadf2c9e32c0c0

This blog gives a great description of what's up with the winsxs system.

You can add / edit manifest specifications using these tools:

XN Resource Editor Portable
HeavenTools Resource Tuner
Beware "ResEdit" for windows. It installs a bunch of spyware.

I tried to alter manifest files directly for libzmq.pyd, but it did not work. That seems like the most appropriate place to make a change (as it is the pyzmq bundled libzmq.pyd file that has the dependency, but not the manifest specification). I am going to go back to the .dll files directly from zmq to see if I can modify them.

UPDATE:
Curiously enough the .dll file from zeromq.org .that is linked against vc90 includes the dependency specification. Python reports that the dll is loaded (when it is moved to the site-packages/zmq directory and the .pyd file removed). However, there is still an error of "no module named zmq.blockingkernelmanager" in vim and regular python reports an error on import of zmq (the specified module could not be found). Perhaps it is a version difference (4.0.4 .dll vs 4.0.5 .pyd).

[10p10110111xxi11p00100100]

Thank you, @dhj-io! I finally made "vim-ipython" work as advertised on my system (Python distro=Anaconda, Windows 7 (64-bit), vim-ipython: installed through Vundle). As it was not so straightforward, I outline my steps:

• After obtaining "XNResourceEditorPortable.exe", I opened "gvim.exe" in "XN Resource Editor"
• Exported and then deleted "XP Theme Manifest\1\English (United States)" leaf node
• Modified the exported manifest by adding additional <dependency>:

<dependency>
  <dependentAssembly>
    <assemblyIdentity
      type="win32"
      name="Microsoft.VC90.CRT"
      version="9.0.21022.8"
      processorArchitecture="x86"
      publicKeyToken="1fc8b3b9a1e18e3b">
    </assemblyIdentity>
  </dependentAssembly>
</dependency>

• "Import User Resource..." to import this modified manifest
• Save the modified file (one can keep the unmodified "gvim.exe" as backup with another name) as "gvim.exe"

Opening .py file, running :IPython, and typing any Python expression and <Ctrl-S> on that line works as intended.

So, unlike mt.exe, XN Resource Editor worked on 64-bit Windows 7.

[dhj-io]

Glad I could help! I wish there was an upstream fix to make it work out of the box (without the vim/gvim resource mod). I think the best long term fix would be a mod to the zmq .dll or .pyd files to include the dependency there. Agree with the bug close -- this is a python dependencies issue not a vim-ipython issue. Modifying vim/gvim resource manifest works well enough!

[aronla]

I'm having the exact same issue, but can't get the solution above to work. Could this be the reason: when using the "XN Resource Editor" and importing the user resource, I can't get it to say "XP Theme Manifest" anymore? Does the node have to have this exact name?

I'd appreciate any help, this is driving me nuts! :)

[chiefenne]

I am having the same status as @aronla.
Almost same nuts driving issue :)