title: Pushing binary data to Amazon S3 using Node.js
body: <p>I'm trying to take an image and upload it to an Amazon S3 bucket using Node.js. In the end, I want to be able to push the image up to S3, and then be able to access that S3 URL and see the image in a browser. I'm using a Curl query to do an HTTP POST request with the image as the body.</p>
  
  <p><code>curl -kvX POST --data-binary "@test.jpg" 'http://localhost:3031/upload/image'</code></p>
  
  <p>Then on the Node.js side, I do this:</p>
  
  <pre><code>exports.pushImage = function(req, res) {
      var image = new Buffer(req.body);
      var s3bucket = new AWS.S3();
      s3bucket.createBucket(function() {
          var params = {Bucket: 'My/bucket', Key: 'test.jpg', Body: image};
          // Put the object into the bucket.
          s3bucket.putObject(params, function(err) {
              if (err) {
                  res.writeHead(403, {'Content-Type':'text/plain'});
                  res.write("Error uploading data");
                  res.end()
              } else {
                  res.writeHead(200, {'Content-Type':'text/plain'});
                  res.write("Success");
                  res.end()
              }
          });
      });
  };
  </code></pre>
  
  <p>My file is 0 bytes, as shown on Amazon S3. How do I make it so that I can use Node.js to push the binary file up to S3? What am I doing wrong with binary data and buffers?</p>
  
  <p><strong>UPDATE:</strong></p>
  
  <p>I found out what I needed to do. The curl query is the first thing that should be changed. This is the working one:</p>
  
  <p><code>curl -kvX POST -F foobar=@my_image_name.jpg 'http://localhost:3031/upload/image'</code></p>
  
  <p>Then, I added a line to convert to a Stream. This is the working code:</p>
  
  <pre><code>exports.pushImage = function(req, res) {
      var image = new Buffer(req.body);
      var s3bucket = new AWS.S3();
      s3bucket.createBucket(function() {
          var bodyStream = fs.createReadStream(req.files.foobar.path);
          var params = {Bucket: 'My/bucket', Key: 'test.jpg', Body: bodyStream};
          // Put the object into the bucket.
          s3bucket.putObject(params, function(err) {
              if (err) {
                  res.writeHead(403, {'Content-Type':'text/plain'});
                  res.write("Error uploading data");
                  res.end()
              } else {
                  res.writeHead(200, {'Content-Type':'text/plain'});
                  res.write("Success");
                  res.end()
              }
          });
      });
  };
  </code></pre>
  
  <p>So, in order to upload a file to an API endpoint (using Node.js and Express) and have the API push that file to Amazon S3, first you need to perform a POST request with the "files" field populated. The file ends up on the API side, where it resides probably in some tmp directory. Amazon's S3 putObject method requires a Stream, so you need to create a read stream by giving the 'fs' module the path where the uploaded file exists.</p>
  
  <p>I don't know if this is the proper way to upload data, but it works. Does anyone know if there is a way to POST binary data inside the request body and have the API send that to S3? I don't quite know what the difference is between a multi-part upload vs a standard POST to body.</p>