PyInstaller with mip "NameError: name 'cbclib' is not define"

[simontt19]

Describe the bug
I am writing a program that solve MIP and it works well when I directly run the code. But after I use pyinstaller to pack my code into an executable file and then when I run that file this error comes out("NameError: name 'cbclib' is not define").

To Reproduce
The whole logic is complicated, so I only extract the relevant one.

1. The script as below

from mip import Model
import tkinter as tk
import sys
def run():
    m = Model("load_planing") 
    r3 = str(m)
    r3_label.configure(text=r3)   

if __name__ == "__main__":
    window = tk.Tk()
    window.title('Multi-SOC Optimizer')
    window.geometry('640x480')
    window.configure(background='white')
    calculate_btn = tk.Button(window, text='Start!',command=run)
    calculate_btn.pack(side=tk.TOP)
    r3_label = tk.Label(window)
    r3_label.pack(side=tk.TOP)
    window.mainloop()
    sys.exit()

2. Directly run the script on python is fine.

3. Use pyinstaller to pack the logic. pyinstaller -F -n mip_debug mip_debug.py

4. Run the exe file after packed. => error

Expected behavior
Running the exe file's outcome should be same as directly run on python.

Desktop (please complete the following information):

• Operating System, version: 64-bit operating system, win10
• Python version:3.8.3
• Python-MIP version (we recommend you to test with the latest version): 1.13.0

[nickpgill]

I have the same error! Did you manage to fix it?

[labaude]

Same here with slightly different versions. Any resolution?

[rschwarz]

This is not actually a bug with this library. Rather, you need to specify that the required .dll file should be bundled with pyinstaller by using an --add-binary argument.

[sebheger]

@nickpgill @labaude @simontt19 Does the comment of @rschwarz fixes your problem?

[nickpgill]

Sorry for the delay in replying... I tried to make use of the suggestion from @rschwarz but I was not able to figure out how to use the --add-binary argument. If you can give me a clue here, then I will follow this up and see if I can make it work...

…

On Tue, 5 Apr 2022 at 20:39, Sebastian Heger ***@***.***> wrote: @nickpgill <https://github.com/nickpgill> @labaude <https://github.com/labaude> @simontt19 <https://github.com/simontt19> Does the comment of @rschwarz <https://github.com/rschwarz> fixes your problem? — Reply to this email directly, view it on GitHub <#198 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIC6SVOBT67BVJGHBVBGUXDVDSJHNANCNFSM474RQ6ZQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[rschwarz]

I have not tried it with python-mip and Cbc, but here's how I called pyinstaller (on Windows) for a past project that used PySCIPOpt:

pyinstaller --onefile ^
            --add-binary "%SCIPOPTDIR%\bin\libscip.dll;." ^
            --add-binary "%SCIPOPTDIR%\bin\tbb.dll;." ^
            --windowed ^
            mypackage/myapp.py

where SCIPOPTDIR was previously defined via set SCIPOPTDIR=C:\Program Files\SCIPOptSuite 7.0.2. The result was a portable .exe that included the two relevant .dlls.

I'm no longer sure what the ;. suffix after the .dll path means 😞

[nickpgill]

Thanks Robert, I appreciate your help with this. I'll see if I can make this work! Nick

…

On Mon, 25 Apr 2022 at 09:36, Robert Schwarz ***@***.***> wrote: I have not tried it with python-mip and Cbc, but here's how I called pyinstaller (on Windows) for a past project that used PySCIPOpt: pyinstaller --onefile ^ --add-binary "%SCIPOPTDIR%\bin\libscip.dll;." ^ --add-binary "%SCIPOPTDIR%\bin\tbb.dll;." ^ --windowed ^ mypackage/myapp.py where SCIPOPTDIR was previously defined via set SCIPOPTDIR=C:\Program Files\SCIPOptSuite 7.0.2. The result was a portable .exe that included the two relevant .dlls. I'm no longer sure what the ;. suffix after the .dll path means 😞 — Reply to this email directly, view it on GitHub <#198 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIC6SVLCY4DHLWQQ5JZ32ZLVGZKRPANCNFSM474RQ6ZQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[sebheger]

@nickpgill Will you share some insights with us so that I can see if there is some issue on python-mip side or that I can move it to the discussion section. Otherwise, I will close this issue.

[zanghyu]

you can write a hook-mip.py file there in a folder named hooks (create a folder hooks and touch a file hooks/hook-mip.py), and the code of hook-mip.py is:

from PyInstaller.utils.hooks import collect_all  
def hook(hook_api):  
    packages = ['mip']  
    for package in packages:  
        datas, binaries, hiddenimports = collect_all(package)  
        hook_api.add_datas(datas)  
        hook_api.add_binaries(binaries)
        hook_api.add_imports(*hiddenimports)

then, you can use pyinstaller as: pyinstaller -D xxx.py --additional-hooks-dir=./hooks/, then it will work.

[nickpgill]

I have come back to this thread 2 1/2 years later because I got the same error when I tried to use pyinstaller to make a windows build via github actions. However I used the answer of @zanghyu and it now works perfectly. So thank you!

[foobacca]

This still doesn't work on Linux, even with the script provided by @zanghyu . The issue appears to be that the file mip/libraries/cbc-c-linux-x86-64.so is not found. Digging into pyinstaller hooks, we can see that:

• collect_all() calls collect_dynamic_libs() with mostly default args - see here
• collect_dynamic_libs() looks for libraries using the arg search_patterns which defaults to PY_DYLIB_PATTERNS - see function def starting here
• PY_DYLIB_PATTERNS is defined here

# Patterns of dynamic library filenames that might be bundled with some installed Python packages.
PY_DYLIB_PATTERNS = [
    '*.dll',
    '*.dylib',
    'lib*.so',
]

So it finds:

├── cbc-c-darwin-x86-64.dylib
└── win64
    ├── cbc-c-windows-x86-64.dll
    ├── libbz2-1.dll
    ├── libgcc_s_seh-1.dll
    ├── libstdc++-6.dll
    └── libwinpthread-1.dll

but fails to find:

├── cbc-c-linux-x86-64.so

It would find it if it was libcbc-c-linux-x86-64.so - I'm not sure why PyInstaller uses lib*.so rather than *.so

Will see if I can adjust the code provided by @zanghyu to add the extra .so to binaries

[foobacca]

I seem to have fixed our project. The slightly expanded hook-mip.py file is now:

from PyInstaller.utils.hooks import collect_all, collect_dynamic_libs


# this function came from this comment:
# https://github.com/coin-or/python-mip/issues/198#issuecomment-1640309437
def hook(hook_api):
    packages = ['mip']
    for package in packages:
        datas, binaries, hiddenimports = collect_all(package)
        binary_paths = set(b[0] for b in binaries)
        # find any xyz.so libraries that don't start with "lib"
        binaries += [b for b in collect_dynamic_libs(package, search_patterns=["*.so"]) if b[0] not in binary_paths]
        hook_api.add_datas(datas)
        hook_api.add_binaries(binaries)
        hook_api.add_imports(*hiddenimports)

This extra step would not be required if the linux .so file was named libcbc-xyz.so rather than cbc-xyz.so

I don't have a strong opinion on what is the right way to go here, but leaving the notes in case others hit something similar.