[Bug] ci_cpu docker image build fail

[cissoidx]

Expected behavior

The docker image (ci_cpu) builds successfully.

Actual behavior

failed with the following error

40.25 [ 69%] Built target nnpack_reference_layers
40.88 [ 71%] Generating src/x86_64-fma/2d-fourier-16x16.py.o
43.07 [ 72%] Generating src/x86_64-fma/2d-winograd-8x8-3x3.py.o
43.44 [ 73%] Generating src/x86_64-fma/blas/s8gemm.py.o
43.56 [ 74%] Generating src/x86_64-fma/blas/c8gemm.py.o
43.76 [ 75%] Generating src/x86_64-fma/blas/s4c6gemm.py.o
43.94 [ 77%] Generating src/x86_64-fma/blas/conv1x1.py.o
44.04 [ 78%] Generating src/x86_64-fma/blas/sgemm.py.o
44.18 [ 79%] Generating src/x86_64-fma/max-pooling.py.o
44.25 [ 80%] Generating src/x86_64-fma/relu.py.o
44.30 [ 81%] Generating src/x86_64-fma/softmax.py.o
44.57 [ 83%] Generating src/x86_64-fma/blas/sdotxf.py.o
44.78 [ 84%] Generating src/x86_64-fma/blas/shdotxf.py.o
44.81 Traceback (most recent call last):
44.81   File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
44.81     return _run_code(code, main_globals, None,
44.81   File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
44.81     exec(code, run_globals)
44.81   File "/NNPACK/deps/peachpy/peachpy/x86_64/__main__.py", line 282, in <module>
44.81     main()
44.81   File "/NNPACK/deps/peachpy/peachpy/x86_64/__main__.py", line 269, in main
44.81     execute_script(writers, options.input[0])
44.81   File "/NNPACK/deps/peachpy/peachpy/x86_64/__main__.py", line 200, in execute_script
44.81     execute_script(writers, source_filename)
44.81   File "/NNPACK/deps/peachpy/peachpy/x86_64/__main__.py", line 200, in execute_script
44.81     execute_script(writers, source_filename)
44.81   File "/NNPACK/deps/peachpy/peachpy/x86_64/__main__.py", line 204, in execute_script
44.81     exec(code, globals())
44.81   File "/NNPACK/src/x86_64-fma/blas/shdotxf.py", line 4, in <module>
44.81     from fp16.avx import fp16_alt_xmm_to_fp32_xmm
44.81 ModuleNotFoundError: No module named 'fp16.avx'
44.81 make[2]: *** [CMakeFiles/nnpack.dir/build.make:188: src/x86_64-fma/blas/shdotxf.py.o] Error 1
44.81 make[1]: *** [CMakeFiles/Makefile2:175: CMakeFiles/nnpack.dir/all] Error 2
44.81 make: *** [Makefile:136: all] Error 2
------
Dockerfile.ci_cpu:82
--------------------
  80 |     # NNPACK deps
  81 |     COPY install/ubuntu_install_nnpack.sh /install/ubuntu_install_nnpack.sh
  82 | >>> RUN bash /install/ubuntu_install_nnpack.sh
  83 |
  84 |     # ANTLR deps
--------------------
ERROR: failed to solve: process "/bin/sh -c bash /install/ubuntu_install_nnpack.sh" did not complete successfully: exit code: 2
ERROR: docker build failed.

Environment

Ubuntu 22.04
Latest TVM code at 1f4c568

Steps to reproduce

./docker/build.sh tvm.ci_cpu


### Triage

Please refer to the list of label tags [here](https://github.com/apache/tvm/wiki/Issue-Triage-Labels) to find the relevant tags and add them below in a bullet format (example below).

* needs-triage

[cissoidx]

the demo_cpu also fails

Building container (tvm.demo_cpu)...
[+] Building 1.7s (2/2) FINISHED                                                                                      docker:default
 => [internal] load build definition from Dockerfile.demo_cpu                                                                   0.0s
 => => transferring dockerfile: 1.37kB                                                                                          0.0s
 => ERROR [internal] load metadata for docker.io/tlcpack/ci-cpu:v0.55                                                           1.7s
------
 > [internal] load metadata for docker.io/tlcpack/ci-cpu:v0.55:
------
Dockerfile.demo_cpu:20
--------------------
  18 |     # Minimum docker image for demo purposes
  19 |     # prebuilt-image: tvmai/demo-cpu
  20 | >>> FROM tlcpack/ci-cpu:v0.55
  21 |
  22 |     COPY utils/apt-install-and-clear.sh /usr/local/bin/apt-install-and-clear
--------------------
ERROR: failed to solve: tlcpack/ci-cpu:v0.55: docker.io/tlcpack/ci-cpu:v0.55: not found
ERROR: docker build failed.

[yongwww]

Thanks for reporting this error. Seems the image build failed with the same error for several days. cc: @lhutton1 @tqchen

[lhutton1]

Looks related to this issue: Maratyszcza/NNPACK#219

[yongwww]

if we don't have a quick fix for this, probably we could consider disabling nnpack.

[agoston-mc]

During my testing, I could do a workaround by updating the ubuntu_install_nnpack.sh file (pastebin) so that it uses a previous commit of FP16 during the NNPACK cmake as a temporary solution, while they work it out in NNPACK.

[tqchen]

i think it makes sense to disable and deprecate nnpack given it is more stale,