title: Luigi Pipeline beginning in S3
body: <p>My initial files are in <code>AWS S3</code>. Could someone point me how I need to setup this in a <code>Luigi Task</code>?</p>
  
  <p>I reviewed the documentation and found <code>luigi.S3</code> but is not clear for me what to do with that, then I searched in the web and only get links from <code>mortar-luigi</code> and implementation in top of luigi.</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>After following the example provided for @matagus (I created the <code>~/.boto</code> file as suggested too):</p>
  
  <pre><code># coding: utf-8
  
  import luigi
  
  from luigi.s3 import S3Target, S3Client
  
  class MyS3File(luigi.ExternalTask):
      def output(self):
          return S3Target('s3://my-bucket/19170205.txt')
  
  class ProcessS3File(luigi.Task):
  
      def requieres(self):
          return MyS3File()
  
      def output(self):
          return luigi.LocalTarget('/tmp/resultado.txt')
  
      def run(self):
          result = None
  
          for input in self.input():
             print("Doing something ...")
             with input.open('r') as f:
                 for line in f:
                     result = 'This is a line'
  
          if result:
              out_file = self.output().open('w')
              out_file.write(result)
  </code></pre>
  
  <p>When I execute it nothing happens</p>
  
  <pre><code>DEBUG: Checking if ProcessS3File() is complete
  INFO: Informed scheduler that task   ProcessS3File()   has status   PENDING
  INFO: Done scheduling tasks
  INFO: Running Worker with 1 processes
  DEBUG: Asking scheduler for work...
  DEBUG: Pending tasks: 1
  INFO: [pid 21171] Worker Worker(salt=226574718, workers=1, host=heliodromus, username=nanounanue, pid=21171) running   ProcessS3File()
  INFO: [pid 21171] Worker Worker(salt=226574718, workers=1, host=heliodromus, username=nanounanue, pid=21171) done      ProcessS3File()
  DEBUG: 1 running tasks, waiting for next task to finish
  INFO: Informed scheduler that task   ProcessS3File()   has status   DONE
  DEBUG: Asking scheduler for work...
  INFO: Done
  INFO: There are no more tasks to run at this time
  INFO: Worker Worker(salt=226574718, workers=1, host=heliodromus, username=nanounanue, pid=21171) was stopped. Shutting down Keep-Alive thread
  </code></pre>
  
  <p>As you can see, the message <code>Doing something...</code> never prints. What is wrong?</p>