Rebuild ci-arm, ci-cpu, and ci-gpu container

[areusch]

This is a tracking issue for the process of updating the TVM ci- containers to reflect the following PRs:

#8169

Steps:

• Pick the latest revision of main and use that specific git hash for following steps. 61a6ea1
  • merge [BYOC][NNAPI]: Add testing package to ci_cpu image #8088
  • merge [CI] [ComputeLibrary] Use pre-built binaries instead of compiled #8245
  • merge Fix bulleted lists in TVM documentation #8268
  • merge Turn on Compute library testing in CI for AArch64 #8291
  • merge Update ONNX versions #8304
  • merge [Docker] Update tensorflow/tflite/xgboost versions #8306
  • merge Install curl in ubuntu_install_core.sh #8310
  • merge Fix ordering of tf and tflite installs in ci_cpu #8312
  • merge [Docker] Fix ordering of tf and tflite installs in ci_qemu #8315
  • merge [DOCKER] fix sphinx install versions #8316
  • merge [Docker][QEMU] Update gpg server #8319
• Build docker containers, tagging them as e.g. <username>/ci-<container>:v0.<ver>
  • ci-arm
  • ci-cpu
  • ci-gpu
  • ci-qemu
• Push docker containers to Docker Hub
• Create a draft PR modifying Jenkinsfile to point all containers at <username>/ci-<container>:v0.<ver>
  • Update Docker CI #8193
• Force-push the PR to ci-docker-staging branch
• Jenkins will notice the push and start a build here
• Debug the build and repeat these steps until the build passes
• Push the valid containers to tlcpack/ci-<container>:v0.<ver>
• Update the PR to point Jenkinsfile to the new containers.
• Merge the PR.

[areusch]

cc @d-smirnov @u99127

[areusch]

also #8088 needs to be a part of this

[areusch]

expanding to ci-gpu and including #8268

[Lunderberg]

Found while implementing #8029, there are a few version incompatibilities in tlcpack/ci_gpu:v0.75.

• Current version of tensorflow==2.3.1 is incompatible with cuda 11.0. Based on this table, we'll want to switch to tensorflow==2.4.0. The warning can be reproduced by running import tensorflow in a python shell.

• WARNING:root:scikit-learn version 0.24.2 is not supported. Minimum required version: 0.17. Maximum required version: 0.19.2. Disabling scikit-learn conversion API. The warning can be reproduced by running import coremltools in a python shell.

  The current version of coremltools==4.1 has a maximum version of scikit-learn<=0.19.2. Unfortunately, the current version of gluoncv==0.10.1 has a conflicting minimum version of scikit-learn>=0.23.2, so it won't be a simple update of a single package.

[areusch]

i won't have time to get to this during this week. if someone else wants to push the images, go for it. #8193 could also potentially join this merge train.

if this isn't closed by next Monday, I'll handle it then.

[mbrookhart]

I can probably take this. It looks like there are a number of things we need to merge to main before regenerating the tickets?

It looks like the follow up to #8169 was merged as #8245. Do we need to merge #8088 and a CI-only version of #8193 before generating the images? It looks like #8268 failed a flaky test in the TF frontend, @areusch can you rebase?

@Lunderberg Do you want to make a PR for the TF update?

[areusch]

@mbrookhart rebased

[Lunderberg]

@mbrookhart #8306 is now open, includes the TF version update.

Regarding coremltools, we're already installing the latest version coremltools==4.1, and even the beta version coremltools==5.0b1 gives warnings about the scikit-learn and tensorflow versions. That said, all the tests in tests/python/frontend/coreml and tests/python/contrib/test_coreml_codegen.py pass despite the warnings. If somebody with an IOS device available could test whether tests/python/contrib/test_coreml_runtime.py passes, I think that would be a good additional check.

[mbrookhart]

I've merged everything I think we want, but unfortunately, none of the docker images are building.

ci_arm fails with this: /install/ubuntu_download_arm_compute_lib_binaries.sh: line 54: curl: command not found, I think that's an issue with #8245 . @d-smirnov @leandron , can you debug and fix? you probably need to install curl in the download script or use wget.

Everything else is failing with:

E: Failed to fetch https://apt.llvm.org/bionic/dists/llvm-toolchain-bionic/main/source/Sources.gz  File has unexpected size (2207 != 2205). Mirror sync in progress? [IP: 199.232.194.49 443]
   Hashes of expected file:
    - Filesize:2205 [weak]
    - SHA256:ad346b2d1b4cffcf27cad9cdd14f8d5dd047cfc10cd09e6e17671d78eeac3501
    - SHA1:a4cb9420123a78efb36e79f4cbe8acda7c27ecf7 [weak]
    - MD5Sum:ae153a86b6a5f0f7ee88f5c97ab13a36 [weak]
   Release file created at: Tue, 22 Jun 2021 23:51:39 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.
The command '/bin/sh -c bash /install/ubuntu1804_install_llvm.sh' returned a non-zero code: 100

I wonder if there's something wrong with the llvm release at the moment?

[mbrookhart]

I'm trying to rebuild the images one at a time instead of in parallel on this machine to see if that fixes the LLVM issue

[u99127]

I've merged everything I think we want, but unfortunately, none of the docker images are building.

ci_arm fails with this: /install/ubuntu_download_arm_compute_lib_binaries.sh: line 54: curl: command not found, I think that's an issue with #8245 . @d-smirnov @leandron , can you debug and fix? you probably need to install curl in the download script or use wget.

I've had a look and probably have a fix for it - it appears that curl is installed in individual install files rather than moving into a common place which is what I'll look to do .

Everything else is failing with:

E: Failed to fetch https://apt.llvm.org/bionic/dists/llvm-toolchain-bionic/main/source/Sources.gz  File has unexpected size (2207 != 2205). Mirror sync in progress? [IP: 199.232.194.49 443]
   Hashes of expected file:
    - Filesize:2205 [weak]
    - SHA256:ad346b2d1b4cffcf27cad9cdd14f8d5dd047cfc10cd09e6e17671d78eeac3501
    - SHA1:a4cb9420123a78efb36e79f4cbe8acda7c27ecf7 [weak]
    - MD5Sum:ae153a86b6a5f0f7ee88f5c97ab13a36 [weak]
   Release file created at: Tue, 22 Jun 2021 23:51:39 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.
The command '/bin/sh -c bash /install/ubuntu1804_install_llvm.sh' returned a non-zero code: 100

That happens when the LLVM mirrors are updating to get new packages. Nothing much can be done but wait, sadly. @leandron probably knows more.

Ramana

I wonder if there's something wrong with the llvm release at the moment?