cpp-package missing

[cantor-set]

Description

While trying to compile cpp-package from the main branch, following the instructions from here and runing the following compilation step:

mxnet/build#> cmake -DUSE_CPP_PACKAGE=1 -DBUILD_CPP_EXAMPLES=1 ..

Results on:

CMake Warning:
  Manually-specified variables were not used by the project:

    USE_CPP_PACKAGE

The CPP package never gets build.

Error Message

-- The C compiler identification is GNU 8.3.0
-- The CXX compiler identification is GNU 8.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_CROSSCOMPILING FALSE
-- CMAKE_HOST_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_NAME Linux
-- CMake version '3.13.4' using generator 'Unix Makefiles'
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda/bin/nvcc
-- The CUDA compiler identification is NVIDIA 10.2.89
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- Intel MKL-DNN compat: set DNNL_BUILD_EXAMPLES to MKLDNN_BUILD_EXAMPLES with value OFF
-- Intel MKL-DNN compat: set DNNL_BUILD_TESTS to MKLDNN_BUILD_TESTS with value OFF
-- Intel MKL-DNN compat: set DNNL_ENABLE_JIT_PROFILING to MKLDNN_ENABLE_JIT_PROFILING with value OFF
-- Intel MKL-DNN compat: set DNNL_LIBRARY_TYPE to MKLDNN_LIBRARY_TYPE with value STATIC
-- Intel MKL-DNN compat: set DNNL_ARCH_OPT_FLAGS to MKLDNN_ARCH_OPT_FLAGS with value ``
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found Git: /usr/bin/git (found version "2.20.1")
-- Primitive cache is enabled
-- Using intgemm
-- Compiling with OpenMP
-- Found OpenMP_C: -fopenmp
-- Found OpenMP_CXX: -fopenmp
-- Found OpenMP: TRUE
After choosing blas, linking to dnnl
-- Could NOT find Gperftools (missing: GPERFTOOLS_LIBRARIES GPERFTOOLS_INCLUDE_DIR)
-- Found OpenCV: /usr (found version "3.2.0") found components: core highgui imgproc imgcodecs
-- OpenCV 3.2.0 found (/usr/share/OpenCV)
-- OpenCV_LIBS=opencv_core;opencv_highgui;opencv_imgproc;opencv_imgcodecs
-- Performing Test OPENMP_HAVE_WERROR_FLAG
-- Performing Test OPENMP_HAVE_WERROR_FLAG - Success
-- Performing Test OPENMP_HAVE_STD_GNUPP11_FLAG
-- Performing Test OPENMP_HAVE_STD_GNUPP11_FLAG - Success
-- Performing Test OPENMP_HAVE_STD_CPP11_FLAG
-- Performing Test OPENMP_HAVE_STD_CPP11_FLAG - Success
-- Found PythonInterp: /usr/bin/python (found version "2.7.16")
-- Cannot find llvm-lit.
-- Please put llvm-lit in your PATH, set OPENMP_LLVM_LIT_EXECUTABLE to its full path, or point OPENMP_LLVM_TOOLS_DIR to its directory.
CMake Warning at 3rdparty/openmp/cmake/OpenMPTesting.cmake:22 (message):
The check targets will not be available!
Call Stack (most recent call first):
3rdparty/openmp/cmake/OpenMPTesting.cmake:40 (find_standalone_test_dependencies)
3rdparty/openmp/CMakeLists.txt:49 (include)

-- Performing Test LIBOMP_HAVE_FNO_EXCEPTIONS_FLAG
-- Performing Test LIBOMP_HAVE_FNO_EXCEPTIONS_FLAG - Success
-- Performing Test LIBOMP_HAVE_FNO_RTTI_FLAG
-- Performing Test LIBOMP_HAVE_FNO_RTTI_FLAG - Success
-- Performing Test LIBOMP_HAVE_X_CPP_FLAG
-- Performing Test LIBOMP_HAVE_X_CPP_FLAG - Success
-- Performing Test LIBOMP_HAVE_WCAST_QUAL_FLAG
-- Performing Test LIBOMP_HAVE_WCAST_QUAL_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_FUNCTION_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_FUNCTION_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_LOCAL_TYPEDEF_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_LOCAL_TYPEDEF_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VALUE_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VALUE_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VARIABLE_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VARIABLE_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_SWITCH_FLAG
-- Performing Test LIBOMP_HAVE_WNO_SWITCH_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_COVERED_SWITCH_DEFAULT_FLAG
-- Performing Test LIBOMP_HAVE_WNO_COVERED_SWITCH_DEFAULT_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_DEPRECATED_REGISTER_FLAG
-- Performing Test LIBOMP_HAVE_WNO_DEPRECATED_REGISTER_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_SIGN_COMPARE_FLAG
-- Performing Test LIBOMP_HAVE_WNO_SIGN_COMPARE_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_GNU_ANONYMOUS_STRUCT_FLAG
-- Performing Test LIBOMP_HAVE_WNO_GNU_ANONYMOUS_STRUCT_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_UNKNOWN_PRAGMAS_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNKNOWN_PRAGMAS_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_MISSING_FIELD_INITIALIZERS_FLAG
-- Performing Test LIBOMP_HAVE_WNO_MISSING_FIELD_INITIALIZERS_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_MISSING_BRACES_FLAG
-- Performing Test LIBOMP_HAVE_WNO_MISSING_BRACES_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_COMMENT_FLAG
-- Performing Test LIBOMP_HAVE_WNO_COMMENT_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_SELF_ASSIGN_FLAG
-- Performing Test LIBOMP_HAVE_WNO_SELF_ASSIGN_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_VLA_EXTENSION_FLAG
-- Performing Test LIBOMP_HAVE_WNO_VLA_EXTENSION_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_FORMAT_PEDANTIC_FLAG
-- Performing Test LIBOMP_HAVE_WNO_FORMAT_PEDANTIC_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WSTRINGOP_OVERFLOW_FLAG
-- Performing Test LIBOMP_HAVE_WSTRINGOP_OVERFLOW_FLAG - Success
-- Performing Test LIBOMP_HAVE_MSSE2_FLAG
-- Performing Test LIBOMP_HAVE_MSSE2_FLAG - Success
-- Performing Test LIBOMP_HAVE_FTLS_MODEL_FLAG
-- Performing Test LIBOMP_HAVE_FTLS_MODEL_FLAG - Success
-- Performing Test LIBOMP_HAVE_MMIC_FLAG
-- Performing Test LIBOMP_HAVE_MMIC_FLAG - Failed
-- Performing Test LIBOMP_HAVE_M32_FLAG
-- Performing Test LIBOMP_HAVE_M32_FLAG - Success
-- Performing Test LIBOMP_HAVE_X_FLAG
-- Performing Test LIBOMP_HAVE_X_FLAG - Success
-- Performing Test LIBOMP_HAVE_WARN_SHARED_TEXTREL_FLAG
-- Performing Test LIBOMP_HAVE_WARN_SHARED_TEXTREL_FLAG - Success
-- Performing Test LIBOMP_HAVE_AS_NEEDED_FLAG
-- Performing Test LIBOMP_HAVE_AS_NEEDED_FLAG - Success
-- Performing Test LIBOMP_HAVE_VERSION_SCRIPT_FLAG
-- Performing Test LIBOMP_HAVE_VERSION_SCRIPT_FLAG - Success
-- Performing Test LIBOMP_HAVE_STATIC_LIBGCC_FLAG
-- Performing Test LIBOMP_HAVE_STATIC_LIBGCC_FLAG - Success
-- Performing Test LIBOMP_HAVE_Z_NOEXECSTACK_FLAG
-- Performing Test LIBOMP_HAVE_Z_NOEXECSTACK_FLAG - Success
-- Performing Test LIBOMP_HAVE_FINI_FLAG
-- Performing Test LIBOMP_HAVE_FINI_FLAG - Success
-- Found Perl: /usr/bin/perl (found version "5.28.1")
-- Performing Test LIBOMP_HAVE_VERSION_SYMBOLS
-- Performing Test LIBOMP_HAVE_VERSION_SYMBOLS - Success
-- Performing Test LIBOMP_HAVE___BUILTIN_FRAME_ADDRESS
-- Performing Test LIBOMP_HAVE___BUILTIN_FRAME_ADDRESS - Success
-- Performing Test LIBOMP_HAVE_WEAK_ATTRIBUTE
-- Performing Test LIBOMP_HAVE_WEAK_ATTRIBUTE - Success
-- Looking for include files windows.h, psapi.h
-- Looking for include files windows.h, psapi.h - not found
-- Looking for EnumProcessModules in psapi
-- Looking for EnumProcessModules in psapi - not found
-- LIBOMP: Operating System -- Linux
-- LIBOMP: Target Architecture -- x86_64
-- LIBOMP: Build Type -- Release
-- LIBOMP: Library Kind -- SHARED
-- LIBOMP: Library Type -- normal
-- LIBOMP: Fortran Modules -- FALSE
-- LIBOMP: Build -- 20140926
-- LIBOMP: Use Stats-gathering -- FALSE
-- LIBOMP: Use Debugger-support -- FALSE
-- LIBOMP: Use ITT notify -- TRUE
-- LIBOMP: Use OMPT-support -- TRUE
-- LIBOMP: Use OMPT-optional -- TRUE
-- LIBOMP: Use Adaptive locks -- TRUE
-- LIBOMP: Use quad precision -- TRUE
-- LIBOMP: Use TSAN-support -- FALSE
-- LIBOMP: Use Hwloc library -- FALSE
-- Looking for sqrt in m
-- Looking for sqrt in m - found
-- Looking for __atomic_load_1
-- Looking for __atomic_load_1 - not found
-- Looking for __atomic_load_1 in atomic
-- Looking for __atomic_load_1 in atomic - found
-- check-libomp does nothing.
-- check-ompt does nothing.
-- check-openmp does nothing.
USE_LAPACK is ON
CMake Warning at 3rdparty/googletest/googletest/CMakeLists.txt:47 (project):
VERSION keyword not followed by a value or was followed by a value that
expanded to nothing.

-- Found GTest: gtest
-- Found CUDNN: /usr/local/cuda/lib64/libcudnn.so
-- Could NOT find CUTENSOR (missing: CUTENSOR_LIBRARY CUTENSOR_INCLUDE)
CMake Warning (dev) at 3rdparty/dmlc-core/cmake/Utils.cmake:196 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.

For compatibility with older versions of CMake, option is clearing the
normal variable 'DMLC_FORCE_SHARED_CRT'.
Call Stack (most recent call first):
3rdparty/dmlc-core/CMakeLists.txt:23 (dmlccore_option)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for fopen64
-- Looking for fopen64 - not found
-- Looking for C++ include cxxabi.h
-- Looking for C++ include cxxabi.h - found
-- Looking for nanosleep
-- Looking for nanosleep - found
-- Looking for backtrace
-- Looking for backtrace - found
-- backtrace facility detected in default set of libraries
-- Found Backtrace: /usr/include
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Using unsigned short
-- Check if the system is big endian - little endian
-- /home/xxxx/src/mxnet2/3rdparty/dmlc-core/cmake/build_config.h.in -> include/dmlc/build_config.h
-- Performing Test SUPPORT_MSSE2
-- Performing Test SUPPORT_MSSE2 - Success
-- Autodetected CUDA architecture(s): 6.1
-- CUDA: Using the following NVCC architecture flags -gencode;arch=compute_61,code=sm_61
-- Found CUDAToolkit: /usr/local/cuda/include (found version "10.2.89")
-- Found NVML: /usr/local/cuda/include
-- Found NVML (include: /usr/local/cuda/include, library: /lib/x86_64-linux-gnu/libnvidia-ml.so)
-- Performing Test SUPPORT_MSSE3
-- Performing Test SUPPORT_MSSE3 - Success
-- Determining F16C support
-- Performing Test COMPILER_SUPPORT_MF16C
-- Performing Test COMPILER_SUPPORT_MF16C - Success
-- Using 64-bit integer for tensor size
-- Found Python3: /usr/bin/python3.7 (found version "3.7.3") found components: Interpreter
-- CUDA: Adding NVCC options: --fatbin-options --compress-all
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:

USE_CPP_PACKAGE

-- Build files have been written to: mxnet/build

To Reproduce

Steps to reproduce

1. git clone --recursive https://github.com/apache/incubator-mxnet mxnet
2. mkdir -p mxnet/build
3. cd mxnet
4. cp config/linux_gpu.cmake config.cmake
5. cd build
6. cmake -DUSE_CPP_PACKAGE=1 -DBUILD_CPP_EXAMPLES=1 ..

What have you tried to solve it?

1. Clean installation. The python package build properly, with GPU support

Environment

Environment Information

----------Python Info----------
Version      : 3.8.0
Compiler     : GCC 8.3.0
Build        : ('default', 'Nov 24 2020 16:09:45')
Arch         : ('64bit', 'ELF')
------------Pip Info-----------
Version      : 20.1.1
Directory    : /home/xxxx/.local/share/virtualenvs/deep_learning2-IV6Tqu2b/lib/python3.8/site-packages/pip
----------MXNet Info-----------
Version      : 2.0.0
Directory    : /home/xxxx/src/mxnet/python/mxnet
Commit hash file "/home/xxxx/src/mxnet/python/mxnet/COMMIT_HASH" not found. Not installed from pre-built package or built from source.
Library      : ['/home/xxxx/src/mxnet/python/mxnet/../../build/libmxnet.so']
Build features:
✔ CUDA
✔ CUDNN
✖ NCCL
✖ TENSORRT
✖ CUTENSOR
✔ CPU_SSE
✔ CPU_SSE2
✔ CPU_SSE3
✔ CPU_SSE4_1
✔ CPU_SSE4_2
✖ CPU_SSE4A
✔ CPU_AVX
✖ CPU_AVX2
✔ OPENMP
✖ SSE
✔ F16C
✖ JEMALLOC
✔ BLAS_OPEN
✖ BLAS_ATLAS
✖ BLAS_MKL
✖ BLAS_APPLE
✔ LAPACK
✔ MKLDNN
✔ OPENCV
✖ DIST_KVSTORE
✔ INT64_TENSOR_SIZE
✔ SIGNAL_HANDLER
✖ DEBUG
✖ TVM_OP
----------System Info----------
Platform     : Linux-4.19.0-12-amd64-x86_64-with-glibc2.28
system       : Linux
node         : xxxx
release      : 4.19.0-12-amd64
version      : #1 SMP Debian 4.19.152-1 (2020-10-18)
----------Hardware Info----------
machine      : x86_64
processor    : 
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       43 bits physical, 48 bits virtual
CPU(s):              24
On-line CPU(s) list: 0-23
Thread(s) per core:  2
Core(s) per socket:  12
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               113
Model name:          AMD Ryzen 9 3900X 12-Core Processor
Stepping:            0
CPU MHz:             2205.896
CPU max MHz:         3800.0000
CPU min MHz:         2200.0000
BogoMIPS:            7600.19
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-23
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd sev ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0166 sec, LOAD: 0.5280 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0461 sec, LOAD: 0.0528 sec.
Error open Gluon Tutorial(cn): https://zh.gluon.ai, <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1108)>, DNS finished in 0.01643514633178711 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0315 sec, LOAD: 0.3105 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0193 sec, LOAD: 0.2280 sec.
Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.015947341918945312 sec.
----------Environment----------
KMP_DUPLICATE_LIB_OK="True"
KMP_INIT_AT_FORK="FALSE"

[github-actions]

Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue.
Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly.
If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on contributing to MXNet and our development guides wiki.

[leezu]

CPP Package is currently not available in MXNet 2. Contribution to add it back would be very welcome. You can copy over the code from v1.x branch and may need to do a few changes for compatibility due to removal of deprecated APIs in MXNet 2.

[chinakook]

I have developed a MXNet 2 compatible mxnet-cpp package called mxnet-cpp-mini which inference support only. I'll release it if someone want to use it too.