unable to import deepmd package in jupyter notebook

[minghuisvn]

I installed deepmk-kit by conda command "conda install deepmd-kit=*=cpu lammps-dp==*cpu -c deepmodeling". When i import deepmd in python's interactive face, there appeared following message.

>>> import deepmd
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /home/linazhao/anaconda3/envs/dp/lib/python3.7/site-packages/tensorflow/python/compat/v2_compat.py:61: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term

And when i tried to import deepmd in jupyter notebook, the error ocurred indicating "ModuleNotFoundError: No module named 'deepmd'". I examined the packages installed in conda environment, and indentified the existence of the deepmd-kit installed from deepmodeling channel. But why it cannot be imported in the jupyter notebook?

[njzjz]

You may check your environment variables. https://stackoverflow.com/a/13178935/9567349

[minghuisvn]

Thanks for your reply! But I still could't use deepmd in jupyter notebook, by adding environment path using sys.path.append(). I tried anothor way to install deepmd python interface, as follows:

(base) linazhao@LinaZhao:~/mhsvn/projectenv$ virtualenv -p python3 deepmdenv
created virtual environment CPython3.7.4.final.0-64 in 168ms
  creator CPython3Posix(dest=/home/linazhao/mhsvn/projectenv/deepmdenv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/linazhao/.l      ocal/share/virtualenv)
    added seed packages: pip==21.0.1, setuptools==52.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
(base) linazhao@LinaZhao:~/mhsvn/projectenv$ source deepmdenv/bin/activate
(deepmdenv) (base) linazhao@LinaZhao:~/mhsvn/projectenv$ pip  install --upgrade pip
Requirement already satisfied: pip in ./deepmdenv/lib/python3.7/site-packages (21.0.1)
(deepmdenv) (base) linazhao@LinaZhao:~/mhsvn/projectenv$ pip install --upgrade tensorflow==2.3.0
Collecting tensorflow==2.3.0
  Using cached tensorflow-2.3.0-cp37-cp37m-manylinux2010_x86_64.whl (320.4 MB)
Collecting h5py<2.11.0,>=2.10.0
  Using cached h5py-2.10.0-cp37-cp37m-manylinux1_x86_64.whl (2.9 MB)
Collecting termcolor>=1.1.0
  Using cached termcolor-1.1.0-py3-none-any.whl
Collecting grpcio>=1.8.6
  Using cached grpcio-1.36.1-cp37-cp37m-manylinux2014_x86_64.whl (4.1 MB)
Collecting keras-preprocessing<1.2,>=1.1.1
  Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting protobuf>=3.9.2
  Using cached protobuf-3.15.5-cp37-cp37m-manylinux1_x86_64.whl (1.0 MB)
Collecting absl-py>=0.7.0
  Using cached absl_py-0.12.0-py3-none-any.whl (129 kB)
Requirement already satisfied: wheel>=0.26 in ./deepmdenv/lib/python3.7/site-packages (from tensorflow==2.3.0) (0.36.2)
Collecting wrapt>=1.11.1
  Using cached wrapt-1.12.1-cp37-cp37m-linux_x86_64.whl
Collecting google-pasta>=0.1.8
  Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting tensorflow-estimator<2.4.0,>=2.3.0
  Using cached tensorflow_estimator-2.3.0-py2.py3-none-any.whl (459 kB)
Collecting opt-einsum>=2.3.2
  Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting tensorboard<3,>=2.3.0
  Using cached tensorboard-2.4.1-py3-none-any.whl (10.6 MB)
Collecting six>=1.12.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting astunparse==1.6.3
  Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting scipy==1.4.1
  Using cached scipy-1.4.1-cp37-cp37m-manylinux1_x86_64.whl (26.1 MB)
Collecting gast==0.3.3
  Using cached gast-0.3.3-py2.py3-none-any.whl (9.7 kB)
Collecting numpy<1.19.0,>=1.16.0
  Using cached numpy-1.18.5-cp37-cp37m-manylinux1_x86_64.whl (20.1 MB)
Requirement already satisfied: setuptools>=41.0.0 in ./deepmdenv/lib/python3.7/site-packages (from tensorboard<3,>=2.3.0->      tensorflow==2.3.0) (52.0.0)
Collecting tensorboard-plugin-wit>=1.6.0
  Using cached tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
Collecting werkzeug>=0.11.15
  Using cached Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Using cached google_auth_oauthlib-0.4.3-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8
  Using cached Markdown-3.3.4-py3-none-any.whl (97 kB)
Collecting requests<3,>=2.21.0
  Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting google-auth<2,>=1.6.3
  Using cached google_auth-1.27.1-py2.py3-none-any.whl (136 kB)
Collecting pyasn1-modules>=0.2.1
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
  Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting cachetools<5.0,>=2.0.0
  Using cached cachetools-4.2.1-py3-none-any.whl (12 kB)
Collecting requests-oauthlib>=0.7.0
  Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting importlib-metadata
  Using cached importlib_metadata-3.7.2-py3-none-any.whl (11 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting chardet<5,>=3.0.2
  Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.3-py2.py3-none-any.whl (137 kB)
Collecting idna<3,>=2.5
  Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting oauthlib>=3.0.0
  Using cached oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)
Collecting typing-extensions>=3.6.4
  Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting zipp>=0.5
  Using cached zipp-3.4.1-py3-none-any.whl (5.2 kB)
Installing collected packages: urllib3, pyasn1, idna, chardet, certifi, zipp, typing-extensions, six, rsa, requests, pyasn      1-modules, oauthlib, cachetools, requests-oauthlib, importlib-metadata, google-auth, werkzeug, tensorboard-plugin-wit, pro      tobuf, numpy, markdown, grpcio, google-auth-oauthlib, absl-py, wrapt, termcolor, tensorflow-estimator, tensorboard, scipy,       opt-einsum, keras-preprocessing, h5py, google-pasta, gast, astunparse, tensorflow
Successfully installed absl-py-0.12.0 astunparse-1.6.3 cachetools-4.2.1 certifi-2020.12.5 chardet-4.0.0 gast-0.3.3 google-      auth-1.27.1 google-auth-oauthlib-0.4.3 google-pasta-0.2.0 grpcio-1.36.1 h5py-2.10.0 idna-2.10 importlib-metadata-3.7.2 ker      as-preprocessing-1.1.2 markdown-3.3.4 numpy-1.18.5 oauthlib-3.1.0 opt-einsum-3.3.0 protobuf-3.15.5 pyasn1-0.4.8 pyasn1-mod      ules-0.2.8 requests-2.25.1 requests-oauthlib-1.3.0 rsa-4.7.2 scipy-1.4.1 six-1.15.0 tensorboard-2.4.1 tensorboard-plugin-w      it-1.8.0 tensorflow-2.3.0 tensorflow-estimator-2.3.0 termcolor-1.1.0 typing-extensions-3.7.4.3 urllib3-1.26.3 werkzeug-1.0      .1 wrapt-1.12.1 zipp-3.4.1
(deepmdenv) (base) linazhao@LinaZhao:~/mhsvn/projectenv$ pip install deepmd-kit
Collecting deepmd-kit
  Using cached deepmd_kit-1.3.1-cp37-cp37m-manylinux2010_x86_64.whl (475 kB)
Requirement already satisfied: numpy in ./deepmdenv/lib/python3.7/site-packages (from deepmd-kit) (1.18.5)
Requirement already satisfied: scipy in ./deepmdenv/lib/python3.7/site-packages (from deepmd-kit) (1.4.1)
Collecting dargs
  Using cached dargs-0.1.3-py3-none-any.whl (9.6 kB)
Collecting pyyaml
  Using cached PyYAML-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (636 kB)
Installing collected packages: pyyaml, dargs, deepmd-kit
Successfully installed dargs-0.1.3 deepmd-kit-1.3.1 pyyaml-5.4.1
(deepmdenv) (base) linazhao@LinaZhao:~/mhsvn/projectenv$ dp -h
2021-03-10 10:51:54.597387: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library       'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PA      TH: /usr/local/cuda-10.0/lib64:/usr/local/cuda-10.0/lib64:/usr/local/cuda-10.0/lib64:/usr/local/cuda-10.0/lib:/usr/local/c      uda/extras/CUPTI/lib64
2021-03-10 10:51:54.597434: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not       have a GPU set up on your machine.
WARNING:tensorflow:From /home/linazhao/mhsvn/projectenv/deepmdenv/lib/python3.7/site-packages/tensorflow/python/compat/v2_      compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in       a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Traceback (most recent call last):
  File "/home/linazhao/mhsvn/projectenv/deepmdenv/bin/dp", line 5, in <module>
    from deepmd.main import main
  File "/home/linazhao/mhsvn/projectenv/deepmdenv/lib/python3.7/site-packages/deepmd/__init__.py", line 1, in <module>
    from .env import set_mkl
  File "/home/linazhao/mhsvn/projectenv/deepmdenv/lib/python3.7/site-packages/deepmd/env.py", line 61, in <module>
    op_module = get_module("libop_abi")
  File "/home/linazhao/mhsvn/projectenv/deepmdenv/lib/python3.7/site-packages/deepmd/env.py", line 58, in get_module
    module = tf.load_op_library(module_path + "{}.{}".format(module_name, ext))
  File "/home/linazhao/mhsvn/projectenv/deepmdenv/lib/python3.7/site-packages/tensorflow/python/framework/load_library.py"      , line 58, in load_op_library
    lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /home/linazhao/mhsvn/projectenv/deepmdenv/lib/python3.7/site-packag      es/deepmd/libop_abi.so: undefined symbol: _ZNK10tensorflow8OpKernel11TraceStringERKNS_15OpKernelContextEb
(deepmdenv) (base) linazhao@LinaZhao:~/mhsvn/projectenv$

It failed somehow.But what things do lead to this error?

[njzjz]

You need to install the latest tensorflow with pip install tensorflow==2.4.1. The document is wrong.