title: standard_init_linux.go:219: exec user process caused: no such file or directory
body: <p>I am trying to move my rust server from Heroku to Google Cloud or AWS. Even though I like the simplicity of having a <code>git push</code> build and deploy to Heroku with just a buildpack specified, the service is not cost effective for me.</p>
  <p>I identified Google Cloud Run and AWS Elastic Beanstalk as potential alternatives.</p>
  <p>First, I need to build a docker image with a static binary.</p>
  <p>Thus, I added this Dockerfile:</p>
  <pre><code>FROM rust AS build
  WORKDIR /usr/src
  RUN rustup target add x86_64-unknown-linux-musl
  RUN apt-get update &amp;&amp; apt-get upgrade -y &amp;&amp; apt-get install -y build-essential git clang llvm-dev libclang-dev libssl-dev pkg-config libpq-dev musl-tools brotli
  RUN USER=root cargo new loxe-api
  WORKDIR /usr/src/loxe-api
  COPY Cargo.toml Cargo.lock ./
  COPY data ./data
  COPY migrations ./migrations
  ENV RUSTFLAGS=&quot;-C target-feature=+crt-static&quot; # this should be set by the target, but just to be sure
  RUN cargo build --release
  COPY src ./src
  ENV PKG_CONFIG_ALLOW_CROSS=1
  ENV OPENSSL_INCLUDE_DIR=&quot;/usr/include/openssl&quot;
  RUN cargo install --target x86_64-unknown-linux-musl --path .
  
  FROM scratch
  COPY --from=build /usr/local/cargo/bin/loxe-api .
  COPY data ./data
  COPY migrations ./migrations
  USER 1000
  CMD [&quot;./loxe-api&quot;]
  </code></pre>
  <p>The image gets build without errors, but if I run it via docker run, I'm getting this error:</p>
  <blockquote>
  <p>standard_init_linux.go:219: exec user process caused: no such file or
  directory</p>
  </blockquote>
  <p>By replacing the base image of the final step with rust, I checked whether the binary and other files are in fact in the image. They are, I can see them via ls, but I also cannot execute <code>loxe-api</code> when entering the shell.</p>
  <p>dockerd logs this:</p>
  <pre><code>INFO[2020-07-05T13:04:42.368119033-07:00] shim containerd-shim started                  address=/containerd-shim/bf85e63468a9c1b3b9fe418b5a186673f0609bfff20c4832789ae87433e82473.sock debug=false pid=27032
  INFO[2020-07-05T13:04:42.913438974-07:00] shim reaped                                   id=8cadeee800649ceca8a52d9a75cc9071b923d01a5d2a37497bf8b9a6e719267a
  INFO[2020-07-05T13:04:42.925442900-07:00] ignoring event                                module=libcontainerd namespace=moby topic=/tasks/delete type=&quot;*events.TaskDelete&quot;
  </code></pre>
  <p>Here's dependencies-section of the Cargo.toml:</p>
  <pre><code>[dependencies]
  actix = &quot;0.9&quot;
  actix-cors = &quot;0.2&quot;
  actix-identity = &quot;0.2&quot;
  actix-multipart = &quot;0.2&quot;
  actix-rt = &quot;1.0&quot;
  actix-web = &quot;2.0&quot;
  argonautica = &quot;0.2&quot;
  brotli = &quot;3.3&quot;
  bytes = { version = &quot;0.5&quot;, features = [&quot;serde&quot;]  }
  chrono = { version = &quot;0.4&quot;, features = [&quot;serde&quot;]  }
  derive_more = &quot;0.99&quot;
  diesel = { version = &quot;1.4&quot;, features = [&quot;postgres&quot;, &quot;uuidv07&quot;, &quot;r2d2&quot;, &quot;chrono&quot;, &quot;serde_json&quot;]  }
  diesel_migrations = &quot;1.4&quot;
  dotenv = &quot;0.15&quot;
  env_logger = &quot;0.7&quot;
  futures = &quot;0.3&quot;
  indexmap = { version = &quot;1.3&quot;, features = [&quot;serde-1&quot;] }
  lazy_static = &quot;1.4&quot;
  log = &quot;0.4&quot;
  openssl = { version = &quot;0.10&quot;, features = [&quot;vendored&quot;] }
  openssl-probe = &quot;0.1.2&quot;
  percent-encoding = &quot;2.1&quot;
  r2d2 = &quot;0.8&quot;
  rand = &quot;0.7&quot;
  redis = &quot;0.15&quot;
  rusoto_core = { version = &quot;0.44&quot; }
  rusoto_s3 = { version = &quot;0.44&quot; }
  sanitize-filename = &quot;0.2&quot;
  sendgrid = { version = &quot;0.10&quot;, features = [&quot;rustls&quot;] }
  serde = { version = &quot;1.0&quot;, features = [&quot;derive&quot;]  }
  serde_json = &quot;1.0&quot;
  stripe-rust = &quot;0.12&quot;
  uuid = { version = &quot;0.8&quot;, features = [&quot;serde&quot;, &quot;v4&quot;]  }
  wana_kana = &quot;2.0&quot;
  </code></pre>
  <p>Further investigation from inside the container:</p>
  <pre><code>$ ldd /loxe-api
          linux-vdso.so.1 (0x00007ffcc219d000)
          libpq.so.5 =&gt; /usr/lib/x86_64-linux-gnu/libpq.so.5 (0x00007f2d3792d000)
          libssl.so.1.1 =&gt; /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f2d3789b000)
          libcrypto.so.1.1 =&gt; /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f2d375b2000)
          libgssapi_krb5.so.2 =&gt; /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f2d37565000)
          libldap_r-2.4.so.2 =&gt; /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f2d37511000)
          libpthread.so.0 =&gt; /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2d374f0000)
          libc.so.6 =&gt; /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2d3732d000)
          libdl.so.2 =&gt; /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2d37328000)
          libkrb5.so.3 =&gt; /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f2d37248000)
          libk5crypto.so.3 =&gt; /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f2d37214000)
          libcom_err.so.2 =&gt; /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f2d3720e000)
          libkrb5support.so.0 =&gt; /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f2d371ff000)
          libkeyutils.so.1 =&gt; /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f2d371f6000)
          libresolv.so.2 =&gt; /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f2d371dc000)
          liblber-2.4.so.2 =&gt; /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f2d371cb000)
          libsasl2.so.2 =&gt; /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f2d371ae000)
          libgnutls.so.30 =&gt; /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f2d37002000)
          /lib/ld64.so.1 =&gt; /lib64/ld-linux-x86-64.so.2 (0x00007f2d37983000)
          libp11-kit.so.0 =&gt; /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f2d36ed1000)
          libidn2.so.0 =&gt; /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f2d36eb2000)
          libunistring.so.2 =&gt; /usr/lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f2d36d2e000)
          libtasn1.so.6 =&gt; /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f2d36b1b000)
          libnettle.so.6 =&gt; /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f2d36ae3000)
          libhogweed.so.4 =&gt; /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f2d36aaa000)
          libgmp.so.10 =&gt; /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f2d36a25000)
          libffi.so.6 =&gt; /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f2d36a1b000)
  </code></pre>
  <p>I also tried to build and execute from my host system.
  I ran <code>cargo run --target x86_64-unknown-linux-musl --release</code> which built the binary, but ultimately resulted in this error:</p>
  <blockquote>
  <p>error: could not execute process
  <code>target/x86_64-unknown-linux-musl/release/loxe-api</code> (never executed)</p>
  </blockquote>
  <p>I checked for the existence of the binary via <code>du -h target/x86_64-unknown-linux-musl/release/loxe-api</code>:</p>
  <pre><code>35M     target/x86_64-unknown-linux-musl/release/loxe-api
  </code></pre>
  <p>Further unsuccessful attempts with out-of-the-box and some modified cross, clux/muslrust and emk/rust-musl-builder repos have been made.</p>
  <p>Building and running a new cargo default project with the x86_64-unknown-linux-musl target works. I assume, libclang, brotli, or some of the argonautica libraries may make this not work.</p>
  <hr />
  <p>Here's a simplified Dockerfile that yields the same result.</p>
  <pre><code>FROM rust AS build
  WORKDIR /usr/src
  
  RUN rustup target add x86_64-unknown-linux-musl
  RUN apt-get update &amp;&amp; apt-get upgrade -y &amp;&amp; apt-get install -y build-essential git clang llvm-dev libclang-dev libssl-dev pkg-config libpq-dev musl-tools brotli
  
  RUN USER=root cargo new loxe-api
  WORKDIR /usr/src/loxe-api
  COPY Cargo.toml Cargo.lock ./
  COPY data ./data
  COPY migrations ./migrations
  COPY src ./src
  ENV PKG_CONFIG_ALLOW_CROSS=1
  ENV OPENSSL_INCLUDE_DIR=&quot;/usr/include/openssl&quot;
  ENV RUSTFLAGS=&quot;-C target-feature=+crt-static&quot;
  RUN cargo install --target x86_64-unknown-linux-musl --path .
  
  FROM debian
  COPY --from=build /usr/local/cargo/bin/loxe-api .
  COPY .env ./.env
  COPY data ./data
  COPY migrations ./migrations
  USER 1000
  CMD [&quot;./loxe-api&quot;]
  </code></pre>