python build (with cython) seems broken in linux (python3)

[sl1pkn07]

Description

python setup.py --with-cython seems broken on linux

Environment info (Required)

└───╼  python diagnose.py 
----------Python Info----------
Version      : 3.6.5
Compiler     : GCC 7.3.1 20180406
Build        : ('default', 'Apr 14 2018 13:17:30')
Arch         : ('64bit', 'ELF')
------------Pip Info-----------
Version      : 9.0.1
Directory    : /usr/lib/python3.6/site-packages/pip
----------MXNet Info-----------
No MXNet installed.
----------System Info----------
Platform     : Linux-4.16.4-1-ARCH-x86_64-with-arch
system       : Linux
node         : sL1pKn07
release      : 4.16.4-1-ARCH
version      : #1 SMP PREEMPT Tue Apr 24 13:21:29 UTC 2018
----------Hardware Info----------
machine      : x86_64
processor    : 
Arquitectura:                        x86_64
modo(s) de operación de las CPUs:    32-bit, 64-bit
Orden de los bytes:                  Little Endian
CPU(s):                              48
Lista de la(s) CPU(s) en línea:      0-47
Hilo(s) de procesamiento por núcleo: 2
Núcleo(s) por «socket»:              12
«Socket(s)»                          2
Modo(s) NUMA:                        2
ID de fabricante:                    GenuineIntel
Familia de CPU:                      6
Modelo:                              79
Nombre del modelo:                   Genuine Intel(R) CPU 0000 @ 2.20GHz
Revisión:                            0
CPU MHz:                             1988.153
CPU MHz máx.:                        2400,0000
CPU MHz mín.:                        1200,0000
BogoMIPS:                            4392.39
Virtualización:                      VT-x
Caché L1d:                           32K
Caché L1i:                           32K
Caché L2:                            256K
Caché L3:                            30720K
CPU(s) del nodo NUMA 0:              0-11,24-35
CPU(s) del nodo NUMA 1:              12-23,36-47
Indicadores:                         fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0245 sec, LOAD: 1.0120 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.2953 sec, LOAD: 0.0879 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.2812 sec, LOAD: 0.8269 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.1800 sec, LOAD: 0.8772 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0218 sec, LOAD: 0.6655 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.2025 sec, LOAD: 0.9752 sec.

Package used (Python/R/Scala/Julia):
python 3.6
cython (builded with python3) 0.28.2

Compiler (gcc/clang/mingw/visual studio):

MXNet commit hash:
f0d2776bb9bb3c4f164f1704f0ca3e2b28952c0f

Build config:
cmake ../incubator-mxnet
-DCMAKE_BUILD_TYPE=None
-DCMAKE_INSTALL_PREFIX=/usr
-DCMAKE_INSTALL_LIBDIR=lib
-DCMAKE_INSTALL_DOCDIR=share/doc/mxnet
-DCUDA_HOST_COMPILER=/opt/cuda/bin/gcc
-DCUDA_CUDA_LIBRARY=/opt/cuda/lib64/stub/libcuda.so
-DBUILD_SHARED_LIBS=ON
-DBUILD_TESTING=OFF
-DBUILD_CPP_EXAMPLES=OFF
-DUSE_CPP_PACKAGE=ON
-DUSE_OLDCMAKECUDA=ON
-DUSE_CUDNN=ON
-DUSE_NCCL=ON
-DUSE_OPENCV=ON
-DUSE_OPENMP=ON
-DUSE_LAPACK=ON
-DUSE_JEMALLOC=OFF
-DUSE_GPERFTOOLS=OFF
-DWITH_EXAMPLE=OFF
-DWITH_TEST=OFF \

Error Message:

Compiling mxnet/cython/symbol.pyx because it changed.
Compiling mxnet/cython/ndarray.pyx because it changed.
[1/2] Cythonizing mxnet/cython/ndarray.pyx

Error compiling Cython file:
------------------------------------------------------------
...
# Licensed to the Apache Software Foundation (ASF) under one
^
------------------------------------------------------------

mxnet/cython/ndarray.pyx:1:0: 'mxnet/_cy3/.ndarray' is not a valid module name
Traceback (most recent call last):
  File "setup.py", line 107, in <module>
    ext_modules=config_cython(),
  File "setup.py", line 96, in config_cython
    return cythonize(ret)
  File "/usr/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1026, in cythonize
    cythonize_one(*args)
  File "/usr/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1146, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: mxnet/cython/ndarray.pyx

Minimum reproducible example

Steps to reproduce

MXNET_LIBRARY_PATH="${pkgdir}/usr/lib/libmxnet.so" python setup.py install --root="${pkgdir}"/ --optimize=1 --with-cython

What have you tried to solve it?

archlinux [AUR] mxnet package fix with use this: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=mxnet#n47

but i'm not sure if is ok

[asitstands]

The same with #10295. The problem is with cython 0.28. It is ok with 0.27.

[apeforest]

It seems this issue has been fixed by Open
[MXNET-545] Fix broken cython build #10951 .@sl1pkn07 Could you please verify it? Thx!

[sl1pkn07]

hi @apeforest i cant test the PR because i cant apply it in the top of master (7aac0b5)

greetings

[karan6181]

Throwing same error with the latest "cython-0.29.6". Tried uninstalling cython, and then try to install again(pip version), but it's throwing same issue.

[asitstands]

This issue may have been fixed in the latest master. Would you check the latest master?
Also see this FAQ entry.

[karan6181]

I have tried with latest master on Ubuntu EC2 instance and it's working for me.

Below is the sample output:

Using /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages
Finished processing dependencies for mxnet==1.5.0

@sl1pkn07 Could you please try it again and see if it's failing for you or not and we can close this issue if it's not reproducible. Thanks!

[asitstands]

@karan6181 Would you check if the cython modules are actually used in runtime? mx.nd._internal.NDArrayBase should be mxnet._cy3.ndarray.NDArrayBase or mxnet._cy2.ndarray.NDArrayBase.