Is this a right way to descrease size of my docker images?
I am running a deep learning model in Docker container which needs pytorch and Azure ML service.
AML requirement is Ubuntu 18.04 (which by default has only python3.6 and only way to install python3.7+ is from the source from what i was able to find)
transformers in pytorch have a requirement of python 3.7+
and i need pytorch with Cuda so i choose abinali/pytorch https://github.com/anibali/docker-pytorch/blob/master/dockerfiles/1.5.0-cuda10.2-ubuntu18.04/Dockerfile
The issue is that the size of the image is around 4GB +. so I wanted to move the PyTorch installation out of the image and run install when the docker is running the container(which increases container running uptime).
Error when running basic torch commands
File &quot;test.py&quot;, line 1, in &lt;module&gt;
    import torch
  File &quot;/home/user/miniconda/lib/python3.8/site-packages/torch/__init__.py&quot;, line 135, in &lt;module&gt;
    _load_global_deps()
  File &quot;/home/user/miniconda/lib/python3.8/site-packages/torch/__init__.py&quot;, line 93, in _load_global_deps
    ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
  File &quot;/home/user/miniconda/lib/python3.8/ctypes/__init__.py&quot;, line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/user/miniconda/lib/python3.8/site-packages/torch/lib/../../../../libnvToolsExt.so.1: invalid ELF header

so my current docker file is :
From nvidia/cuda:10.2-base-ubuntu18.04

RUN apt-get update &amp;&amp; apt-get install -y \
    curl \
    ca-certificates \
    sudo \
    git \
    bzip2 \
    libx11-6 \
 &amp;&amp; rm -rf /var/lib/apt/lists/*


# Create a working directory

# COPY . /app
# WORKDIR /app/
RUN mkdir /app
COPY . /app
WORKDIR /app


# Create a non-root user and switch to it
RUN adduser --disabled-password --gecos '' --shell /bin/bash user \
 &amp;&amp; chown -R user:user /app
RUN echo &quot;user ALL=(ALL) NOPASSWD:ALL&quot; &gt; /etc/sudoers.d/90-user
USER user

# All users can use /home/user as their home directory
ENV HOME=/home/user
RUN chmod 777 /home/user


# Install Miniconda and Python 3.8
ENV CONDA_AUTO_UPDATE_CONDA=false
ENV PATH=/home/user/miniconda/bin:$PATH
RUN curl -sLo ~/miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-py38_4.8.2-Linux-x86_64.sh \
 &amp;&amp; chmod +x ~/miniconda.sh \
 &amp;&amp; ~/miniconda.sh -b -p ~/miniconda \
 &amp;&amp; rm ~/miniconda.sh \
 &amp;&amp; conda install -y python==3.8.1 \
 &amp;&amp; conda clean -ya



CMD [&quot;sh&quot;, &quot;-c&quot;, &quot;conda install -y -c pytorch cudatoolkit=10.2 \&quot;pytorch=1.5.0=py3.8_cuda10.2.89_cudnn7.6.5_0\&quot; \&quot;torchvision=0.6.0=py38_cu102\&quot; &amp;&amp; conda clean -ya &amp;&amp; python test.py&quot;]


Installing runtime dependencies via CMD is not a typical way to reduce Docker image size. As the OP noted, this imposes a cost in container startup.
There are a few changes that can be made to the Dockerfile to reduce image size.

Use a base image that has miniconda installed but not cuda/cudnn. The OP installs pytorch, cuda, and cudnn via conda, so then there will be duplicate installations of cudnn and cuda in the image (one from the nvidia/cuda base image and the other from conda). See the Azure ML documentation for a list of publicly available base images provided by Azure ML.
Use --no-install-recommends in the apt-get install command. This will prevent apt-get from installing recommended dependencies, which are not required for the packages' use.
Use the --chown option in COPY to change ownership of the copied files during the COPY. Using RUN chown ... after a COPY will cause the size of /app to count twice towards the Docker image's total size.

Here is a Dockerfile that implements my suggestions. This creates a Docker image that is 3.43 GB.
FROM continuumio/miniconda3

RUN apt-get update \
    &amp;&amp; apt-get install -y --no-install-recommends \
        curl \
        ca-certificates \
        sudo \
        git \
        bzip2 \
        libx11-6 \
    &amp;&amp; rm -rf /var/lib/apt/lists/*

# Create a non-root user
RUN adduser --disabled-password --gecos '' --shell /bin/bash user \
    &amp;&amp; echo &quot;user ALL=(ALL) NOPASSWD:ALL&quot; &gt; /etc/sudoers.d/90-user

# All users can use /home/user as their home directory
ENV HOME=/home/user
RUN chmod 777 /home/user

# Install pytorch.
ENV CONDA_AUTO_UPDATE_CONDA=&quot;false&quot;
RUN conda install --yes python=3.8 \
    &amp;&amp; conda install --yes --channel pytorch \
        cudatoolkit=10.2 \
        pytorch=1.5.0=py3.8_cuda10.2.89_cudnn7.6.5_0 \
        torchvision=0.6.0=py38_cu102 \
    &amp;&amp; conda clean --all --yes

USER user
WORKDIR /app
COPY --chown=user:user . .

CMD [&quot;python&quot;, &quot;test.py&quot;]

