title: Logstash cluster output to Elasticseach cluster without multicast
body: <p>I want to run logstash -> elasticsearch with high availability and cannot find an easy way to achieve it. Please review how I see it and correct me:</p>
  
  <p>Goal:</p>
  
  <ul>
  <li>5 machines each running elasticsearch united into a single cluster.</li>
  <li>5 machines each running logstash server and streaming data into elasticsearch cluster.</li>
  <li>N machines under monitoring each running lumberjack and streaming data into logstash servers.</li>
  </ul>
  
  <p>Constraint: </p>
  
  <ul>
  <li>It is supposed to be run on PaaS (CoreOS/Docker) so multi-casting
  discovery does not work.</li>
  </ul>
  
  <p>Solution:</p>
  
  <ul>
  <li>Lumberjack allows to specify a list of logstash servers to forward data to. Lumberjack will randomly select the target server and switch to another one if this server goes down. It works.</li>
  <li>I can use zookeeper discovery plugin to construct elasticsearch cluster. It works.</li>
  <li>With multi-casting each logstash server discovers and joins the elasticsearch cluster. Without multicasting it allows me to specify a single elasticsearch host. But it is not high availability. I want to output to the cluster, not a single host that can go down. </li>
  </ul>
  
  <p>Question:</p>
  
  <ul>
  <li>Is it realistic to add a zookeeper discovery plugin to logstash's embedded elasticsearch? How?</li>
  <li>Is there an easier (natural) solution for this problem?</li>
  </ul>
  
  <p>Thanks!</p>