title: How to add connectors to presto on Amazon EMR
body: <p>I've set up a small EMR cluster with Hive/Presto installed, I want to query files on S3 and import them to Postgres on RDS.</p>
  
  <p>To run queries on S3 and save the results in a table in postgres I've done the following:</p>
  
  <ol>
  <li>Started a 3 node EMR cluster from the AWS console.</li>
  <li>Manually SSH into the Master node to create an EXTERNAL table in hive, looking at an S3 bucket.</li>
  <li><p>Manually SSH into each of the 3 nodes and add a new catalog file:</p>
  
  <pre><code>/etc/presto/conf.dist/catalog/postgres.properties 
  </code></pre>
  
  <p>with the following contents</p>
  
  <pre><code>connector.name=postgresql
  connection-url=jdbc:postgresql://ip-to-postgres:5432/database
  connection-user=&lt;user&gt;
  connection-password=&lt;pass&gt;
  </code></pre>
  
  <p>and edited this file </p>
  
  <pre><code>/etc/presto/conf.dist/config.properties
  </code></pre>
  
  <p>adding </p>
  
  <pre><code>datasources=postgresql,hive
  </code></pre></li>
  <li><p>Restart presto by running the following manually on all 3 nodes</p>
  
  <pre><code>sudo restart presto-server
  </code></pre></li>
  </ol>
  
  <p>This setup seems to work well.</p>
  
  <p>In my application, there are multiple databases created dynamically. It seems that those configuration/catalog changes need to be made for each database and the server needs to be restarted to see the new config changes.</p>
  
  <p>Is there a proper way for my application (using boto or other methods) to update configurations by</p>
  
  <ol>
  <li>Adding a new catalog file in all nodes /etc/presto/conf.dist/catalog/ for each new database</li>
  <li>Adding a new entry in all nodes in /etc/presto/conf.dist/config.properties </li>
  <li>Gracefully restarting presto across the whole cluster (ideally when it becomes idle, but that's not a major concern.</li>
  </ol>