title: How to import a text file on AWS S3 into pandas without writing to disk
body: <p>I have a text file saved on S3 which is a tab delimited table. I want to load it into pandas but cannot save it first because I am running on a heroku server. Here is what I have so far.</p>
  
  <pre><code>import io
  import boto3
  import os
  import pandas as pd
  
  os.environ["AWS_ACCESS_KEY_ID"] = "xxxxxxxx"
  os.environ["AWS_SECRET_ACCESS_KEY"] = "xxxxxxxx"
  
  s3_client = boto3.client('s3')
  response = s3_client.get_object(Bucket="my_bucket",Key="filename.txt")
  file = response["Body"]
  
  
  pd.read_csv(file, header=14, delimiter="\t", low_memory=False)
  </code></pre>
  
  <p>the error is</p>
  
  <pre><code>OSError: Expected file path name or file-like object, got &lt;class 'bytes'&gt; type
  </code></pre>
  
  <p>How do I convert the response body into a format pandas will accept?</p>
  
  <pre><code>pd.read_csv(io.StringIO(file), header=14, delimiter="\t", low_memory=False)
  
  returns
  
  TypeError: initial_value must be str or None, not StreamingBody
  
  pd.read_csv(io.BytesIO(file), header=14, delimiter="\t", low_memory=False)
  
  returns
  
  TypeError: 'StreamingBody' does not support the buffer interface
  </code></pre>
  
  <p>UPDATE - Using the following worked</p>
  
  <pre><code>file = response["Body"].read()
  </code></pre>
  
  <p>and </p>
  
  <pre><code>pd.read_csv(io.BytesIO(file), header=14, delimiter="\t", low_memory=False)
  </code></pre>