title: Bigquery Streaming inserts, persistent or new http connection on every insert?
body: <p>I am using <a href="https://github.com/google/google-api-ruby-client" rel="nofollow"><code>google-api-ruby-client</code></a> for <a href="https://cloud.google.com/bigquery/streaming-data-into-bigquery" rel="nofollow">Streaming Data Into BigQuery</a>. so whenever there is a request. it is pushed into <a href="http://redis.io/" rel="nofollow">Redis</a> as a queue &amp; then a new <a href="http://sidekiq.org/" rel="nofollow">Sidekiq</a> worker tries to insert into bigquery. I think its involves opening a new HTTPS connection to bigquery every insert.</p>
  
  <p>the way, I have it setup is:
  Events post every 1 second or when the batch size reaches 1MB (one megabyte), whichever occurs first. This is per worker, so the Biquery API may receive tens of HTTP posts per second over multiple HTTPS connections. </p>
  
  <p>This is done using the provided API client by Google. </p>
  
  <p>Now the Question -- For Streaming inserts, what is the better approach:-</p>
  
  <ol>
  <li><strong>persistent HTTPS connection</strong>. if yes, then should it be a global connection that's shared across all requests? or something else?</li>
  <li><strong>Opening new connection</strong>. like we are doing now using <code>google-api-ruby-client</code></li>
  </ol>