title: Makefile with distant (AWS S3) targets
body: <p>I would like to create a makefile where the targets and dependencies would not be local files but rather live in some AWS/S3 bucket.</p>
  
  <p>Consider the following example that would simply copy a 'data_raw' files into 'obj1' and then 'obj2' (you would need to edit 'bucket' to some bucket you own and create some 'data_raw' file before running this):</p>
  
  <pre><code># local, works fine
  bucket = /tmp/test/
  cp = cp
  
  # remote, does not work
  bucket = s3://bucket/test/
  cp = s3cmd cp
  
  all : $(bucket)obj2
  
  $(bucket)obj2 : $(bucket)obj1
      $(cp) $(bucket)obj1 $(bucket)obj2
  
  $(bucket)obj1 :
      $(cp) $(bucket)raw_data $(bucket)obj1
  </code></pre>
  
  <p>The error I get with this is:</p>
  
  <pre><code>makefile:9: *** target pattern contains no `%'.  Stop.
  </code></pre>
  
  <p>which is for:</p>
  
  <pre><code>all : $(bucket)obj2
  </code></pre>
  
  <p>I suspect that make does not understand remote URIs ("s3://xxx") at all.</p>
  
  <p>All the example/documentation I could find seem to implicitly refer to local files for targets and dependencies. Extensive googling only yielded some seemingly unfinished ideas about creating ant tasks for s3 (<a href="http://code.google.com/p/awstasks/" rel="nofollow">http://code.google.com/p/awstasks/</a>).</p>
  
  <p>This is in the context of running several complex/intricate MapReduce jobs in Python.</p>
  
  <p>I would rather use GNU make but will certainly consider alternatives.</p>
  
  <p>I could always create some light local mirror of the remote targets but certainly there is a better way?</p>
  
  <p>Thanks in advance!</p>
  
  <p>Nic</p>