title: Docker build from Dockerfile with more memory
body: <p><strong>How to <code>docker build</code> from Dockerfile with more memory?</strong></p>
  
  <p>This is a different question from this <a href="https://stackoverflow.com/questions/26913110/allow-more-memory-when-docker-build-a-dockerfile">Allow more memory when docker build a Dockerfile</a> </p>
  
  <p>When installing the software natively, there is enough memory to successfully build and install the <a href="https://github.com/marian-nmt/marian" rel="noreferrer"><code>marian</code> tool</a></p>
  
  <p>But when building the Docker image using the <code>Dockerfile</code>
  <a href="https://github.com/marian-nmt/marian/blob/master/scripts/docker/Dockerfile.cpu" rel="noreferrer">https://github.com/marian-nmt/marian/blob/master/scripts/docker/Dockerfile.cpu</a> , it fails with multiple memory exhausted errors </p>
  
  <pre><code>virtual memory exhausted: Cannot allocate memory
  </code></pre>
  
  <p>[out]:</p>
  
  <pre><code>Step : RUN cmake $MARIANPATH &amp;&amp; make -j
   ---&gt; Running in 4867d166d17a
  -- The C compiler identification is GNU 5.4.0
  -- The CXX compiler identification is GNU 5.4.0
  -- Check for working C compiler: /usr/bin/cc
  -- Check for working C compiler: /usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /usr/bin/c++
  -- Check for working CXX compiler: /usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  CUDA_TOOLKIT_ROOT_DIR not found or specified
  -- Cannot find CUDA libraries. Compiling without them.
  -- Could NOT find CUDA (missing:  CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY) 
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Looking for pthread_create
  -- Looking for pthread_create - not found
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE  
  -- Boost version: 1.58.0
  -- Found the following Boost libraries:
  --   system
  --   filesystem
  --   program_options
  --   timer
  --   iostreams
  --   python
  --   thread
  --   chrono
  --   regex
  --   date_time
  --   atomic
  -- Found Python
  -- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.12", minimum required is "2.7") 
  -- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8") 
  -- Found Git: /usr/bin/git (found version "2.7.4") 
  -- Git version: 5abc774
  -- Found SparseHash: /usr/include  
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /marian/build
  Scanning dependencies of target fast_align
  Scanning dependencies of target extract_lex
  Scanning dependencies of target libcnpy
  Scanning dependencies of target atools
  Scanning dependencies of target libyaml-cpp-amun
  [  2%] Building CXX object src/amun/3rd_party/extract_lex/CMakeFiles/extract_lex.dir/exception.cpp.o
  [  2%] Building CXX object src/amun/3rd_party/extract_lex/CMakeFiles/extract_lex.dir/utils.cpp.o
  [  3%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/fast_align.dir/src/ttables.cc.o
  [  6%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/fast_align.dir/src/fast_align.cc.o
  [  7%] Building CXX object src/amun/3rd_party/extract_lex/CMakeFiles/extract_lex.dir/extract-lex-main.cpp.o
  [ 10%] Building CXX object src/amun/3rd_party/CMakeFiles/libcnpy.dir/cnpy/cnpy.cpp.o
  [ 10%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/atools.dir/src/atools.cc.o
  [ 10%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/atools.dir/src/alignment_io.cc.o
  [ 11%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/regex_yaml.cpp.o
  [ 12%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scanner.cpp.o
  [ 14%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/convert.cpp.o
  [ 15%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/stream.cpp.o
  [ 16%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/memory.cpp.o
  [ 17%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/node.cpp.o
  [ 19%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/directives.cpp.o
  [ 20%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/null.cpp.o
  [ 21%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitfromevents.cpp.o
  [ 23%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/singledocparser.cpp.o
  [ 24%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitterstate.cpp.o
  [ 25%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scantag.cpp.o
  [ 26%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/simplekey.cpp.o
  [ 28%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/binary.cpp.o
  [ 29%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/parser.cpp.o
  [ 30%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/nodeevents.cpp.o
  [ 32%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emit.cpp.o
  [ 33%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/exp.cpp.o
  [ 35%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/nodebuilder.cpp.o
  [ 38%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scanscalar.cpp.o
  [ 39%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitter.cpp.o
  [ 41%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/tag.cpp.o
  [ 42%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scantoken.cpp.o
  [ 43%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/ostream_wrapper.cpp.o
  Scanning dependencies of target libcommon
  [ 37%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/node_data.cpp.o
  [ 44%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/contrib/graphbuilder.cpp.o
  [ 46%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitterutils.cpp.o
  [ 34%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/contrib/graphbuilderadapter.cpp.o
  [ 47%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/parse.cpp.o
  [ 48%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/scorer.cpp.o
  [ 50%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/git_version.cpp.o
  [ 51%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/base_matrix.cpp.o
  [ 52%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/sentence.cpp.o
  [ 53%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/config.cpp.o
  [ 55%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/utils.cpp.o
  [ 56%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/vocab.cpp.o
  [ 65%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/filter.cpp.o
  [ 66%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/god.cpp.o
  [ 67%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/exception.cpp.o
  [ 69%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/search.cpp.o
  [ 69%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/hypothesis.cpp.o
  [ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/translation_task.cpp.o
  [ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/printer.cpp.o
  [ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/sentences.cpp.o
  [ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/history.cpp.o
  [ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/logging.cpp.o
  [ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/processor/bpe.cpp.o
  [ 71%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/output_collector.cpp.o
  [ 73%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/types.cpp.o
  [ 74%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/loader.cpp.o
  virtual memory exhausted: Cannot allocate memory
  virtual memory exhausted: Cannot allocate memory
  virtual memory exhausted: Cannot allocate memory
  </code></pre>
  
  <p>And eventually it leads to:</p>
  
  <pre><code>src/amun/CMakeFiles/libcommon.dir/build.make:254: recipe for target 'src/amun/CMakeFiles/libcommon.dir/common/loader.cpp.o' failed
  virtual memory exhausted: Cannot allocate memory
  make[2]: *** [src/amun/CMakeFiles/cpumode.dir/cpu/decoder/encoder_decoder.cpp.o] Error 1
  src/amun/CMakeFiles/cpumode.dir/build.make:110: recipe for target 'src/amun/CMakeFiles/cpumode.dir/cpu/decoder/encoder_decoder.cpp.o' failed
  [ 79%] Built target libcnpy
  virtual memory exhausted: Cannot allocate memory
  src/amun/CMakeFiles/libcommon.dir/build.make:326: recipe for target 'src/amun/CMakeFiles/libcommon.dir/common/printer.cpp.o' failed
  make[2]: *** [src/amun/CMakeFiles/libcommon.dir/common/printer.cpp.o] Error 1
  CMakeFiles/Makefile2:340: recipe for target 'src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/all' failed
  make[1]: *** [src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/all] Error 2
  CMakeFiles/Makefile2:182: recipe for target 'src/amun/CMakeFiles/libcommon.dir/all' failed
  make[1]: *** [src/amun/CMakeFiles/libcommon.dir/all] Error 2
  make[1]: *** [src/amun/CMakeFiles/cpumode.dir/all] Error 2
  CMakeFiles/Makefile2:110: recipe for target 'src/amun/CMakeFiles/cpumode.dir/all' failed
  make: *** [all] Error 2
  </code></pre>
  
  <p>Looking at the <code>docker build --help</code>, there are options to control memory usage:</p>
  
  <pre><code>$ docker build --help
  
  Usage:  docker build [OPTIONS] PATH | URL | -
  
  Build an image from a Dockerfile
  
  Options:
        --build-arg list             Set build-time variables (default [])
        --cache-from stringSlice     Images to consider as cache sources
        --cgroup-parent string       Optional parent cgroup for the container
        --compress                   Compress the build context using gzip
        --cpu-period int             Limit the CPU CFS (Completely Fair Scheduler) period
        --cpu-quota int              Limit the CPU CFS (Completely Fair Scheduler) quota
    -c, --cpu-shares int             CPU shares (relative weight)
        --cpuset-cpus string         CPUs in which to allow execution (0-3, 0,1)
        --cpuset-mems string         MEMs in which to allow execution (0-3, 0,1)
        --disable-content-trust      Skip image verification (default true)
    -f, --file string                Name of the Dockerfile (Default is 'PATH/Dockerfile')
        --force-rm                   Always remove intermediate containers
        --help                       Print usage
        --isolation string           Container isolation technology
        --label list                 Set metadata for an image (default [])
    -m, --memory string              Memory limit
        --memory-swap string         Swap limit equal to memory plus swap: '-1' to enable unlimited swap
        --network string             Set the networking mode for the RUN instructions during build (default "default")
        --no-cache                   Do not use cache when building the image
        --pull                       Always attempt to pull a newer version of the image
    -q, --quiet                      Suppress the build output and print image ID on success
        --rm                         Remove intermediate containers after a successful build (default true)
        --security-opt stringSlice   Security options
        --shm-size string            Size of /dev/shm, default value is 64MB
    -t, --tag list                   Name and optionally a tag in the 'name:tag' format (default [])
        --ulimit ulimit              Ulimit options (default [])
  </code></pre>
  
  <p>But I couldn't figure out the correct syntax of where exactly to put the <code>-m</code> option -_-|||</p>
  
  <p>It isn't before the Dockerfile:</p>
  
  <pre><code># Before Docker file.
  $ docker build -m 4g Dockerfile.cpu -t ibot-cpu .
  
  "docker build" requires exactly 1 argument(s).
  See 'docker build --help'.
  
  Usage:  docker build [OPTIONS] PATH | URL | -
  
  Build an image from a Dockerfile
  </code></pre>
  
  <p>It isn't after Dockerfile before -t</p>
  
  <pre><code># Before -t
  $ docker build Dockerfile.cpu -m 4g -t ibot-cpu .
  
  "docker build" requires exactly 1 argument(s).
  See 'docker build --help'.
  
  Usage:  docker build [OPTIONS] PATH | URL | -
  
  Build an image from a Dockerfile
  </code></pre>
  
  <p>It isn't after -t before the local path </p>
  
  <pre><code># Before local path
  $ docker build Dockerfile.cpu -t ibot-cpu -m 4g .
  
  "docker build" requires exactly 1 argument(s).
  See 'docker build --help'.
  
  Usage:  docker build [OPTIONS] PATH | URL | -
  
  Build an image from a Dockerfile
  </code></pre>
  
  <p>It isn't after the local path at the end too...</p>
  
  <pre><code># At the end...
  $ docker build Dockerfile.cpu -t ibot-cpu . -m 4g
  
  "docker build" requires exactly 1 argument(s).
  See 'docker build --help'.
  
  Usage:  docker build [OPTIONS] PATH | URL | -
  
  Build an image from a Dockerfile
  </code></pre>
  
  <p><strong>How to <code>docker build</code> from Dockerfile with more memory?</strong></p>
  
  <p>My docker version:</p>
  
  <pre><code>docker version
  Client:
   Version:      17.03.1-ce
   API version:  1.27
   Go version:   go1.7.5
   Git commit:   c6d412e
   Built:        Tue Mar 28 00:40:02 2017
   OS/Arch:      darwin/amd64
  
  Server:
   Version:      17.04.0-ce
   API version:  1.28 (minimum version 1.12)
   Go version:   go1.7.5
   Git commit:   4845c56
   Built:        Wed Apr  5 18:45:47 2017
   OS/Arch:      linux/amd64
   Experimental: false
  </code></pre>