Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cd/Jenkinsfile_cd_pipeline
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ pipeline {

parameters {
// Release parameters
string(defaultValue: "cpu,native,cu101,cu102,cu110", description: "Comma separated list of variants", name: "MXNET_VARIANTS")
string(defaultValue: "cpu,native,cu101,cu102,cu110,cu112", description: "Comma separated list of variants", name: "MXNET_VARIANTS")
booleanParam(defaultValue: false, description: 'Whether this is a release build or not', name: "RELEASE_BUILD")
}

Expand Down
2 changes: 1 addition & 1 deletion cd/Jenkinsfile_release_job
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ pipeline {
// any disruption caused by different COMMIT_ID values chaning the job parameter configuration on
// Jenkins.
string(defaultValue: "mxnet_lib/static", description: "Pipeline to build", name: "RELEASE_JOB_TYPE")
string(defaultValue: "cpu,native,cu101,cu102,cu110", description: "Comma separated list of variants", name: "MXNET_VARIANTS")
string(defaultValue: "cpu,native,cu101,cu102,cu110,cu112", description: "Comma separated list of variants", name: "MXNET_VARIANTS")
booleanParam(defaultValue: false, description: 'Whether this is a release build or not', name: "RELEASE_BUILD")
}

Expand Down
5 changes: 3 additions & 2 deletions cd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ MXNet aims to support a variety of frontends, e.g. Python, Java, Perl, R, etc. a

The CD process is driven by the [CD pipeline job](Jenkinsfile_cd_pipeline), which orchestrates the order in which the artifacts are delivered. For instance, first publish the libmxnet library before publishing the pip package. It does this by triggering the [release job](Jenkinsfile_release_job) with a specific set of parameters for each delivery channel. The release job executes the specific release pipeline for a delivery channel across all MXNet *variants*.

A variant is a specific environment or features for which MXNet is compiled. For instance CPU, GPU with CUDA v10.1, CUDA v10.2 with MKL-DNN support, etc.
A variant is a specific environment or features for which MXNet is compiled. For instance CPU, GPU with CUDA v10.1, CUDA v10.2 with MKL-DNN support, etc.

Currently, below variants are supported. All of these variants except native have MKL-DNN backend enabled.

Expand All @@ -34,6 +34,7 @@ Currently, below variants are supported. All of these variants except native hav
* *cu101*: CUDA 10.1
* *cu102*: CUDA 10.2
* *cu110*: CUDA 11.0
* *cu112*: CUDA 11.2

*For more on variants, see [here](https://github.com/apache/incubator-mxnet/issues/8671)*

Expand Down Expand Up @@ -119,7 +120,7 @@ The "first mile" of the CD process is posting the mxnet binaries to the [artifac

##### Timeout

We shouldn't set global timeouts for the pipelines. Rather, the `step` being executed should be rapped with a `timeout` function (as in the pipeline example above). The `max_time` is a global variable set at the [release job](Jenkinsfile_release_job) level.
We shouldn't set global timeouts for the pipelines. Rather, the `step` being executed should be rapped with a `timeout` function (as in the pipeline example above). The `max_time` is a global variable set at the [release job](Jenkinsfile_release_job) level.

##### Node of execution

Expand Down
2 changes: 1 addition & 1 deletion cd/utils/artifact_repository.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ If not set, derived through the value of sys.platform (https://docs.python.org/3

**Variant**

Manually configured through the --variant argument. The current variants are: cpu, native, cu101, cu102, cu110.
Manually configured through the --variant argument. The current variants are: cpu, native, cu101, cu102, cu110, cu112.

As long as the tool is being run from the MXNet code base, the runtime feature detection tool (https://github.com/larroy/mxnet/blob/dd432b7f241c9da2c96bcb877c2dc84e6a1f74d4/docs/api/python/libinfo/libinfo.md) can be used to detect whether the library has been compiled with MKL (library has MKL-DNN feature enabled) and/or CUDA support (compiled with CUDA feature enabled).

Expand Down
3 changes: 3 additions & 0 deletions cd/utils/mxnet_base_image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ case ${mxnet_variant} in
cu110*)
echo "nvidia/cuda:11.0-cudnn8-runtime-ubuntu16.04"
;;
cu112*)
echo "nvidia/cuda:11.2-cudnn8-runtime-ubuntu16.04"
;;
cpu)
echo "ubuntu:16.04"
;;
Expand Down
10 changes: 10 additions & 0 deletions ci/docker/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,16 @@ services:
BASE_IMAGE: nvidia/cuda:11.0-cudnn8-devel-centos7
cache_from:
- ${DOCKER_CACHE_REGISTRY}/build.centos7_gpu_cu110:latest
centos7_gpu_cu112:
image: ${DOCKER_CACHE_REGISTRY}/build.centos7_gpu_cu112:latest
build:
context: .
dockerfile: Dockerfile.build.centos7
target: base
args:
BASE_IMAGE: nvidia/cuda:11.2.0-cudnn8-devel-centos7
cache_from:
- ${DOCKER_CACHE_REGISTRY}/build.centos7_gpu_cu112:latest
###################################################################################################
# Dockerfile.build.ubuntu based images. On Ubuntu we test more recent
# toolchain and dependency versions compared to CentOS7. We attempt to update
Expand Down
1 change: 1 addition & 0 deletions tools/pip/doc/CPU_ADDITIONAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
Prerequisites
-------------
This package supports Linux, Mac OSX, and Windows platforms. You may also want to check:
- [mxnet-cu112](https://pypi.python.org/pypi/mxnet-cu112/) with CUDA-11.2 support.
- [mxnet-cu110](https://pypi.python.org/pypi/mxnet-cu110/) with CUDA-11.0 support.
- [mxnet-cu102](https://pypi.python.org/pypi/mxnet-cu102/) with CUDA-10.2 support.
- [mxnet-cu101](https://pypi.python.org/pypi/mxnet-cu101/) with CUDA-10.1 support.
Expand Down
1 change: 1 addition & 0 deletions tools/pip/doc/CU101_ADDITIONAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
Prerequisites
-------------
This package supports Linux and Windows platforms. You may also want to check:
- [mxnet-cu112](https://pypi.python.org/pypi/mxnet-cu112/) with CUDA-11.2 support.
- [mxnet-cu110](https://pypi.python.org/pypi/mxnet-cu110/) with CUDA-11.0 support.
- [mxnet-cu102](https://pypi.python.org/pypi/mxnet-cu102/) with CUDA-10.2 support.
- [mxnet](https://pypi.python.org/pypi/mxnet/).
Expand Down
1 change: 1 addition & 0 deletions tools/pip/doc/CU102_ADDITIONAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
Prerequisites
-------------
This package supports Linux and Windows platforms. You may also want to check:
- [mxnet-cu112](https://pypi.python.org/pypi/mxnet-cu112/) with CUDA-11.2 support.
- [mxnet-cu110](https://pypi.python.org/pypi/mxnet-cu110/) with CUDA-11.0 support.
- [mxnet-cu101](https://pypi.python.org/pypi/mxnet-cu101/) with CUDA-10.1 support.
- [mxnet](https://pypi.python.org/pypi/mxnet/).
Expand Down
1 change: 1 addition & 0 deletions tools/pip/doc/CU110_ADDITIONAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
Prerequisites
-------------
This package supports Linux and Windows platforms. You may also want to check:
- [mxnet-cu112](https://pypi.python.org/pypi/mxnet-cu112/) with CUDA-11.2 support.
- [mxnet-cu102](https://pypi.python.org/pypi/mxnet-cu102/) with CUDA-10.2 support.
- [mxnet-cu101](https://pypi.python.org/pypi/mxnet-cu101/) with CUDA-10.1 support.
- [mxnet](https://pypi.python.org/pypi/mxnet/).
Expand Down
44 changes: 44 additions & 0 deletions tools/pip/doc/CU112_ADDITIONAL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
<!--- Licensed to the Apache Software Foundation (ASF) under one -->
<!--- or more contributor license agreements. See the NOTICE file -->
<!--- distributed with this work for additional information -->
<!--- regarding copyright ownership. The ASF licenses this file -->
<!--- to you under the Apache License, Version 2.0 (the -->
<!--- "License"); you may not use this file except in compliance -->
<!--- with the License. You may obtain a copy of the License at -->

<!--- http://www.apache.org/licenses/LICENSE-2.0 -->

<!--- Unless required by applicable law or agreed to in writing, -->
<!--- software distributed under the License is distributed on an -->
<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
<!--- KIND, either express or implied. See the License for the -->
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->

Prerequisites
-------------
This package supports Linux and Windows platforms. You may also want to check:
- [mxnet-cu110](https://pypi.python.org/pypi/mxnet-cu110/) with CUDA-11.0 support.
- [mxnet-cu102](https://pypi.python.org/pypi/mxnet-cu102/) with CUDA-10.2 support.
- [mxnet-cu101](https://pypi.python.org/pypi/mxnet-cu101/) with CUDA-10.1 support.
- [mxnet](https://pypi.python.org/pypi/mxnet/).
- [mxnet-native](https://pypi.python.org/pypi/mxnet-native/) CPU variant without MKLDNN.

To download CUDA, check [CUDA download](https://developer.nvidia.com/cuda-downloads). For more instructions, check [CUDA Toolkit online documentation](http://docs.nvidia.com/cuda/index.html).

To use this package on Linux you need the `libquadmath.so.0` shared library. On
Debian based systems, including Ubuntu, run `sudo apt install libquadmath0` to
install the shared library. On RHEL based systems, including CentOS, run `sudo
yum install libquadmath` to install the shared library. As `libquadmath.so.0` is
a GPL library and MXNet part of the Apache Software Foundation, MXNet must not
redistribute `libquadmath.so.0` as part of the Pypi package and users must
manually install it.

To install for other platforms (e.g. Windows, Raspberry Pi/ARM) or other versions, check [Installing MXNet](https://mxnet.apache.org/versions/master) for instructions on building from source.

Installation
------------
To install:
```bash
pip install mxnet-cu112
```
1 change: 1 addition & 0 deletions tools/pip/doc/NATIVE_ADDITIONAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
Prerequisites
-------------
This package supports Linux and Windows platforms. You may also want to check:
- [mxnet-cu112](https://pypi.python.org/pypi/mxnet-cu112/) with CUDA-11.2 support.
- [mxnet-cu110](https://pypi.python.org/pypi/mxnet-cu110/) with CUDA-11.0 support.
- [mxnet-cu102](https://pypi.python.org/pypi/mxnet-cu102/) with CUDA-10.2 support.
- [mxnet-cu101](https://pypi.python.org/pypi/mxnet-cu101/) with CUDA-10.1 support.
Expand Down
4 changes: 3 additions & 1 deletion tools/pip/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,9 @@ def skip_markdown_comments(md):
if variant == 'CPU':
libraries.append('openblas')
else:
if variant.startswith('CU110'):
if variant.startswith('CU112'):
libraries.append('CUDA-11.2')
elif variant.startswith('CU110'):
libraries.append('CUDA-11.0')
elif variant.startswith('CU102'):
libraries.append('CUDA-10.2')
Expand Down
4 changes: 2 additions & 2 deletions tools/staticbuild/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ You need to install `patchelf` first, for example via `apt install patchelf` on
Ubuntu systems.

```
tools/staticbuild/build.sh cu102
tools/staticbuild/build.sh cu112
```
This would build the mxnet package based on CUDA 10.2. Currently, we support variants cpu, native, cu101, cu102 and cu110. All of these variants expect native have MKL-DNN backend enabled.
This would build the mxnet package based on CUDA 11.2. Currently, we support variants cpu, native, cu101, cu102, cu110, and cu112. All of these variants expect native have MKL-DNN backend enabled.

```
tools/staticbuild/build.sh cpu
Expand Down