Skip to content

Commit

Permalink
🔙 bring back cuda 10.0 for tensorflow 1.x (#52)
Browse files Browse the repository at this point in the history
  • Loading branch information
haobibo authored Jul 19, 2020
1 parent 4728ac4 commit 8ace9d5
Show file tree
Hide file tree
Showing 3 changed files with 125 additions and 3 deletions.
37 changes: 35 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,19 +47,26 @@ jobs:
script:
- build_image_squash "${TRAVIS_JOB_NAME}" "docker_base/Dockerfile"

- name: "base-cuda_10.0"
stage: cuda
script:
- build_image_squash "${TRAVIS_JOB_NAME}" "docker_base/cuda10.0.Dockerfile"
--build-arg "repository=${REPOSITORY}"
- docker rmi "${REPOSITORY}:base"

- name: "base-cuda_10.1"
stage: cuda
script:
- build_image_squash "${TRAVIS_JOB_NAME}" "docker_base/cuda10.1.Dockerfile"
--build-arg "repository=${REPOSITORY}"
- alias_image "${TRAVIS_JOB_NAME}" "cuda"
- docker rmi "${REPOSITORY}:base"

- name: "base-cuda_10.2"
stage: cuda
script:
- build_image_squash "${TRAVIS_JOB_NAME}" "docker_base/cuda10.2.Dockerfile"
--build-arg "repository=${REPOSITORY}"
- alias_image "${TRAVIS_JOB_NAME}" "cuda"
- docker rmi "${REPOSITORY}:base"

- name: "base-cuda_11.0"
Expand Down Expand Up @@ -127,6 +134,18 @@ jobs:
--build-arg "ARG_LATEX_CJK=true"
- docker rmi "${REPOSITORY}:cuda"

- name: "jupyter-full-cuda_10.0"
stage: jpy
script:
- build_image_squash "${TRAVIS_JOB_NAME}" "docker_jpy/Dockerfile"
--build-arg "repository=${REPOSITORY}"
--build-arg "base=base-cuda_10.0"
--build-arg "ARG_EXTEND_JUPYTER=true"
--build-arg "ARG_NODEJS=true"
--build-arg "ARG_LATEX_BASE=true"
--build-arg "ARG_LATEX_CJK=true"
- docker rmi "${REPOSITORY}:cuda"


- name: "r-mini"
stage: qpod
Expand Down Expand Up @@ -217,6 +236,20 @@ jobs:
--build-arg "ARG_PY_BIOINFO=true"
- docker rmi "${REPOSITORY}:jupyter-full-cuda"

- name: "py-cuda_10.0"
stage: qpod
script:
- build_image_squash "${TRAVIS_JOB_NAME}" "docker_template/Dockerfile"
--build-arg "repository=${REPOSITORY}"
--build-arg "base=jupyter-full-cuda_10.0"
--build-arg "ARG_MKL=true"
--build-arg "ARG_PY_DATABASE=true"
--build-arg "ARG_PY_DATASCIENCE=true"
--build-arg "ARG_PY_NLP=true"
--build-arg "ARG_PY_CV=true"
--build-arg "ARG_PY_BIOINFO=true"
- docker rmi "${REPOSITORY}:jupyter-full-cuda"

- name: "go"
stage: qpod
script:
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ you can easily add one in the `install_XX.list` in the `work` folder.
| `octave` | [![octave](https://images.microbadger.com/badges/image/qpod/qpod:octave.svg)](https://microbadger.com/images/qpod/qpod:octave) | `jupyter-full` | Image with Jupyter environment and Octave and LaTex installed.
| `full`, `latest` | [![full](https://images.microbadger.com/badges/image/qpod/qpod:full.svg)](https://microbadger.com/images/qpod/qpod:full) [![latest](https://images.microbadger.com/badges/image/qpod/qpod.svg)](https://microbadger.com/images/qpod/qpod) | `jupyter-full` | All features and packages (Pythoncuda, R, RStudio, OpenJDK, NodeJS, Go, Julia, LaTex, Jupyter extensions) for CPU included in this image. |
| **👆The above Images do NOT have NVIDIA cuda/cudnn features installed.** | | | **👇The Following Images have NVIDA cuda/cudnn features installed. Work for Linux only.** |
| `cuda`, `base-cuda_10.1` | [![cuda](https://images.microbadger.com/badges/image/qpod/qpod:cuda.svg)](https://microbadger.com/images/qpod/qpod:cuda) [![base-cuda_10.1](https://images.microbadger.com/badges/image/qpod/qpod:base-cuda_10.1.svg)](https://microbadger.com/images/qpod/qpod:base-cuda_10.1) | `base` | This image add version 10.1 of NVIDIA cuda and cudnn libs, including runtime and devel. We use this version by default because popular Deep Learning packages hosted on `pypi` is build against `cuda 10.1`. |
| `cuda`, `base-cuda_10.2` | [![cuda](https://images.microbadger.com/badges/image/qpod/qpod:cuda.svg)](https://microbadger.com/images/qpod/qpod:cuda) [![base-cuda_10.1](https://images.microbadger.com/badges/image/qpod/qpod:base-cuda_10.1.svg)](https://microbadger.com/images/qpod/qpod:base-cuda_10.1) | `base` | This image add version 10.2 of NVIDIA cuda and cudnn libs, including runtime and devel. We use this version by default because packages like pytorch hosted on `pypi` is build against `cuda 10.2`. |
| `jupyter-mini-cuda` | [![jupyter-mini-cuda](https://images.microbadger.com/badges/image/qpod/qpod:jupyter-mini-cuda.svg)](https://microbadger.com/images/qpod/qpod:jupyter-mini-cuda) | `cuda` | A minimal Jupyter environment with NVIDIA cuda installed. No popular data science or AI Python installed. This might not be very useful, unless you just want to test Jupyter and cuda. |
| `jupyter-full-cuda` | [![jupyter-full-cuda](https://images.microbadger.com/badges/image/qpod/qpod:jupyter-full-cuda.svg)](https://microbadger.com/images/qpod/qpod:jupyter-full-cuda) | `cuda` | `jupyter-mini-cuda` plus NodeJS, LaTex, and Jupyter extensions. Might not be very useful as above but will server as base of other images. |
| `py-cuda` | [![py-cuda](https://images.microbadger.com/badges/image/qpod/qpod:py-cuda.svg)](https://microbadger.com/images/qpod/qpod:py-cuda) | `jupyter-full-cuda` | This is the recommended image for Python based Deep Learning environment, which includes popular Python data science and AI packages. (`tensorflow` already included, use pip to install `pytorch` easily) |
Expand Down
89 changes: 89 additions & 0 deletions docker_base/cuda10.0.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Distributed under the terms of the Modified BSD License.

# CUDA base: https://gitlab.com/nvidia/container-images/cuda/-/tree/master/dist/ubuntu18.04/10.0/base/Dockerfile
# CUDA runtime: https://gitlab.com/nvidia/container-images/cuda/-/tree/master/dist/ubuntu18.04/10.0/runtime/Dockerfile
# CUDNN runtime: https://gitlab.com/nvidia/container-images/cuda/-/tree/master/dist/ubuntu18.04/10.0/runtime/cudnn7/Dockerfile
# CUDA devel: https://gitlab.com/nvidia/container-images/cuda/-/tree/master/dist/ubuntu18.04/10.0/devel/Dockerfile
# CUDNN devel https://gitlab.com/nvidia/container-images/cuda/-/tree/master/dist/ubuntu18.04/10.0/devel/cudnn7/Dockerfile

ARG repository
FROM ${repository}:base

LABEL maintainer="haobibo@gmail.com"

ARG ARG_CUDA_RUNTIME=true
ARG ARG_CUDNN_RUNTIME=true
ARG ARG_CUDA_DEVEL=true
ARG ARG_CUDNN_DEVEL=true

ENV CUDA_VER 10.0
ENV CUDA_VERSION ${CUDA_VER}.130
ENV CUDA_PKG_VERSION 10-0=$CUDA_VERSION-1
ENV NCCL_VERSION 2.4.8
ENV CUDNN_VERSION 7.6.5.32
ENV NVIDIA_REQUIRE_CUDA "cuda>=${CUDA_VER}"

ENV NVIDIA_VISIBLE_DEVICES=all \
NVIDIA_DRIVER_CAPABILITIES=compute,utility \
NVIDIA_REQUIRE_CUDA="cuda>=10.0 brand=tesla,driver>=384,driver<385 brand=tesla,driver>=410,driver<411" \
PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH} \
LIBRARY_PATH=/usr/local/cuda/lib64/stubs \
LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64

LABEL com.nvidia.cudnn.version="${CUDNN_VERSION}"

# Installing CUDA base
RUN wget -qO- "https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub" | apt-key add - \
&& echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/cuda.list \
&& echo "deb https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/nvidia-ml.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
cuda-cudart-$CUDA_PKG_VERSION cuda-compat-10-0 \
&& ln -s cuda-$CUDA_VER /usr/local/cuda \
&& echo "/usr/local/nvidia/lib" >> /etc/ld.so.conf.d/nvidia.conf \
&& echo "/usr/local/nvidia/lib64" >> /etc/ld.so.conf.d/nvidia.conf

# If installing CUDA runtime
RUN ${ARG_CUDA_RUNTIME:-false} \
&& apt-get install -y --no-install-recommends \
cuda-libraries-$CUDA_PKG_VERSION cuda-nvtx-$CUDA_PKG_VERSION libnccl2=$NCCL_VERSION-1+cuda$CUDA_VER \
&& apt-mark hold libnccl2 \
|| true

# If installing CUDNN runtime
RUN ${ARG_CUDNN_RUNTIME:false} \
&& apt-get install -y --no-install-recommends \
libcudnn7=$CUDNN_VERSION-1+cuda$CUDA_VER \
&& apt-mark hold libcudnn7 \
|| true

# If installing CUDA devel
RUN ${ARG_CUDA_DEVEL:false} \
&& apt-get install -y --no-install-recommends \
cuda-libraries-dev-$CUDA_PKG_VERSION cuda-nvml-dev-$CUDA_PKG_VERSION \
cuda-minimal-build-$CUDA_PKG_VERSION cuda-command-line-tools-$CUDA_PKG_VERSION \
libnccl-dev=$NCCL_VERSION-1+cuda$CUDA_VER \
|| true

# If installing CUDNN devel
RUN ${ARG_CUDNN_DEVEL:false} \
&& apt-get install -y --no-install-recommends \
libcudnn7=$CUDNN_VERSION-1+cuda$CUDA_VER libcudnn7-dev=$CUDNN_VERSION-1+cuda$CUDA_VER \
|| true

# Install Utilities `nvtop`
RUN cd /tmp \
&& apt-get -y update --fix-missing && apt-get -qq install -y --no-install-recommends libncurses5-dev \
&& git clone https://github.com/Syllo/nvtop.git \
&& mkdir -p nvtop/build && cd nvtop/build \
&& LIB_PATH=`find / -name "libnvidia-ml*" 2>/dev/null` \
&& cmake .. -DCMAKE_LIBRARY_PATH="`dirname $LIB_PATH`" .. \
&& make && make install \
&& apt-get -qq remove -y libncurses5-dev

# Clean up and display components version information...
RUN source /opt/utils/script-utils.sh \
&& conda install -yq python=3.7 \
&& install__clean && cd \
&& echo "@ Version of image: building finished at:" `date` `uname -a` \
&& echo "@ System environment variables:" `printenv`

0 comments on commit 8ace9d5

Please sign in to comment.