title: Compute timeseries in a streaming way using input time-series from disparates sources
body: <p>I develop and maintain a web app that displays metrics data.
  Theses metrics are mainly time series with a regular resolution coming from
  sensors, programmable logic controllers etc. Some of theses times series are
  measures (like temperature etc.) and some others are the result of a compute
  using others time-series as inputs. The measures come from distincts sources
  with differents formats,protocols and delay of availability. They are all batch oriented for
  now but stream oriented will coming soon.</p>
  
  <p>For now Importation of data is done using Apache Camel that stores data into a
  database. The compute of metrics is then done by a "homemade" batch program
  that is called by apache Camel each time a batch import is done, giving to this script the start and end date of the imported time-series and some kind of namespace to process only time-series that are concerned by the batch.</p>
  
  <p>While this solution works for now, it has drawbacks due its batch orientation. I'm starting to think about a better solution for future improvments.</p>
  
  <p>To be more precise i need a system that is able to generate output time-series
  by applying mathematical operations on inputs time-series, things as simple as </p>
  
  <pre><code>tsout = (t1+t2) / t3
  </code></pre>
  
  <p>considering inputs (t1, t2, t3) are not available at the same time because
  they come from distincts sources (some from batch, some from streams..). Some
  computed time-series are used as input of some others. All of theses create
  a dependency graph.</p>
  
  <p>Are there some design patterns, tools, frameworks or even "concepts" that
  could help me to improve the generation/computing of metrics in a more
  "streaming" way as the data is coming, abstrating disparate sources and
  considering the dependencies ?</p>
  
  <p>I took a look at frameworks like apache Kafka, Flink, Spark.. but i doubt they
  really answer my needs. They allows to do <code>stream / event processing</code> but i'm 
  not sure this refers to the same kind of processing i need to do. </p>
  
  <p>Hope my question is sufficiently clear, it is not easy to explain.</p>