How to run a pytorch model server inside docker?
I am trying to create a docker image to make it run as a server for serving a model in pytorch.
I converted the .pt model file to .MAR file in my local machine
and i copied the .MAR file inside the docker image. I created a dockerfile:
FROM ubuntu:18.04
ENV TZ=Asia/Shanghai
ENV DEBIAN_FRONTEND noninteractive 
RUN apt-get update \
    &amp;&amp; apt-get install --yes --no-install-recommends \
    tzdata

RUN ln -snf /usr/share/zoneinfo/${TZ} /etc/localtime\ 
    &amp;&amp; echo ${TZ} &gt; /etc/timezone \
    &amp;&amp; dpkg-reconfigure -f noninteractive tzdata

RUN apt-get install python3 python3-dev python3-pip openjdk-11-jre-headless git wget curl -y
RUN python3 -m pip install torch torchvision torch-model-archiver torchserve==0.2.0
COPY densenet161.mar /model_store/
CMD torchserve --start --model-store model_store --models densenet161=densenet161.mar
EXPOSE 8080

I was able to create the image but I was not able to access that container
when I tried to open the image and run the code it works
docker exec -it 4b126bd87f21 sh

# torchserve --start --ncs --model-store model_store --models densenet161.mar

The server is running. When I run the docker image it is not working. Docker container is running but I was not able to access the server.
I don't know what is the problem.

I hope I understand the problem.
When you do docker run torchserve:local ...., by default it runs the CMD which is torchserve --start --model-store model_store --models densenet161=densenet161.mar but since the command runs in the background, your newly created docker container will immediately exit. Due to this same problem, i.e. to prevent docker exit it is possible to add tail -f /dev/null.
Look at the official docker entry point of torchserve https://github.com/pytorch/serve/blob/master/docker/dockerd-entrypoint.sh#L12
They tail it at the end to prevent docker exit
