anchor.diff:  
   A common concern (read optimization) for improving performance is to increase the number of shards and thus increase the number of tasks on the Hadoop side. Unless such gains are demonstrated through benchmarks, we recommend against such a measure since in most cases, an {es} shard can *easily* handle data streaming to a Hadoop task.
   
  +[[arch-writing]]
  +[float]
   === Writing to {es}
   
   Writing to {es} is driven by the number of input splits (or Hadoop tasks) available. In the current form, {eh} uses only one node on the {es} side to handle the writing which acts as a proxy within the {es} cluster: as data flows in, based on its identity and cluster setting, {es} will route it to the appropriate shard.
  
positive.code: 
  For the vast majority of cases, using only one node on the {es} side should pose no problem - if the network is not saturated, {es} can ingest the data just fine. However, for cases where this happens (again benchmark first instead of guessing), we are working on using multiple nodes based on the available index.
  
  === Data co-location
  
  Whenever possible, {eh} shares the {es} cluster information with Hadoop to facilitate data co-location. In practice, this means whenever data is read from {es}, the source nodes IPs are passed on to Hadoop to optimize task execution. If co-location is desired/possible, hosting the {es} and Hadoop clusters within the same rack will provide significant network savings.
negative.code: [[arch]]
  == Architecture
  
  At the core, {eh} integrates two _distributed_ systems: *Hadoop*, a batch-oriented computing platform and *{es}*, a real-time search and analytics engine. From a high-level view both provide a computational component: Hadoop through {mr} and {es} through its search and aggregation.
  
  {eh} goal is to 'connect' these two entities so that they can transparently benefit from each other.
  
  [[arch-shards]]
  [float]
  === {mr} and Shards
  
  A critical component for scalability is parallelism or splitting a task into multiple, smaller ones that execute at the same time, on different nodes in the cluster. The concept is present in both Hadoop through its `splits` (the number of parts in which a source or input can be divided) and {es} through {ref}/glossary.html#glossary-shard[`shards`] (the number of parts in which a index is divided into).
  