title: Spark Streaming with Neo4j hangs while running with Docker
body: <p>I have created a docker image of my application when I simply run it from the bash script, it works properly. However, when I run it as part of the docker-compose file the application hangs on the message: </p>
  
  <pre><code>18/06/27 13:17:18 INFO StateStoreCoordinatorRef: Registered StateStoreCoordinator endpoint
  </code></pre>
  
  <p>And even after I wait for a while streaming heartbeat times out. What may be the reason for such a Spark Streaming+Neo4j application performance with Docker and how it can be improved?</p>
  
  <p>The docker-compose file for my application:</p>
  
  <pre><code>version: '3.3'
  services:
    consumer-demo:
      build:
        context: .
        dockerfile: Dockerfile
        args:
          - ARG_CLASS=consumer
          - HOST=neo4jdb
      volumes:
        - ./:/workdir
      working_dir: /workdir
      restart: always
  </code></pre>
  
  <p>Overall docker-compose file for all the applications:</p>
  
  <pre><code>version: '3.3'
  services:
    kafka:
      image: spotify/kafka
      ports:
       - "9092:9092"
      networks:
        - docker_elk
      environment:
      - ADVERTISED_HOST=localhost
    neo4jdb:
      image: neo4j:latest
      container_name: neo4jdb
      ports:
        - "7474:7474"
        - "7473:7473"
        - "7687:7687"
      networks:
        - docker_elk
      volumes:
        - /var/lib/neo4j/import:/var/lib/neo4j/import
        - /var/lib/neo4j/data:/data
        - /var/lib/neo4j/conf:/conf
      environment:
        - NEO4J_dbms_active__database=graphImport.db
    elasticsearch:
      image: elasticsearch:latest
      ports:
        - "9200:9200"
        - "9300:9300"
      networks:
        - docker_elk
      volumes:
          - esdata1:/usr/share/elasticsearch/data
    kibana:
      image: kibana:latest
      ports:
        - "5601:5601"
      networks:
        - docker_elk
  volumes:
    esdata1:
      driver: local
  
  networks:
    docker_elk:
      driver: bridge
  </code></pre>
  
  <p>The bash script using which an application works properly:</p>
  
  <pre><code>#!/usr/bin/env bash
  if [ "$1" = "consumer" ]
  then
      java -cp "jars/spark_consumer.jar" consumer.SparkConsumer 
  else
      echo "Wrong parameter. It should be consumer or producer, but it is $1"
  fi
  </code></pre>
  
  <p>Application Dockerfile which may be the reason of slowdown of the application execution:</p>
  
  <pre><code>FROM java:8
  ARG ARG_CLASS
  ARG HOST
  ENV MAIN_CLASS $ARG_CLASS
  ENV SCALA_VERSION 2.11.8
  ENV SBT_VERSION 1.1.1
  ENV SPARK_VERSION 2.2.0
  ENV SPARK_DIST spark-$SPARK_VERSION-bin-hadoop2.6
  ENV SPARK_ARCH $SPARK_DIST.tgz
  ENV HOSTNAME bolt://$HOST:7687
  VOLUME /workdir
  
  WORKDIR /opt
  
  # Install Scala
  RUN \
    cd /root &amp;&amp; \
    curl -o scala-$SCALA_VERSION.tgz http://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz &amp;&amp; \
    tar -xf scala-$SCALA_VERSION.tgz &amp;&amp; \
    rm scala-$SCALA_VERSION.tgz &amp;&amp; \
    echo &gt;&gt; /root/.bashrc &amp;&amp; \
    echo 'export PATH=~/scala-$SCALA_VERSION/bin:$PATH' &gt;&gt; /root/.bashrc
  
  # Install SBT
  RUN \
    curl -L -o sbt-$SBT_VERSION.deb https://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb &amp;&amp; \
    dpkg -i sbt-$SBT_VERSION.deb &amp;&amp; \
    rm sbt-$SBT_VERSION.deb
  
  
  # Install Spark
  RUN \
      cd /opt &amp;&amp; \
      curl -o $SPARK_ARCH http://d3kbcqa49mib13.cloudfront.net/$SPARK_ARCH &amp;&amp; \
      tar xvfz $SPARK_ARCH &amp;&amp; \
      rm $SPARK_ARCH &amp;&amp; \
      echo 'export PATH=$SPARK_DIST/bin:$PATH' &gt;&gt; /root/.bashrc
  
  EXPOSE 9851 9852 4040 9092 9200 9300 5601 7474 7687 7473
  
  CMD /workdir/runDemo.sh "$MAIN_CLASS" 
  </code></pre>