title: APACHE NIFI vs APACHE AIRFLOW vs APACHE FALCON ? Which suits best in the below scenario?
body: <p>I am developing a solution in Java which communicates with a set of devices through REST APIs which belongs to different vendors. So for each vendor, there are a set of processes that I have to perform inside my solution. However, these processes will be changed according to each vendor. Following are the high-level processes that need to be performed.</p>
  
  <ul>
  <li><code>Retrieve an XML file from a folder</code></li>
  <li><code>Process the XML file</code></li>
  <li><code>Perform some image processing</code></li>
  <li><code>Schedule a job and execute it on the scheduled time</code></li>
  <li><code>Storing data on a MySQL DB and perform some REST calls to outside APIs</code></li>
  </ul>
  
  <p>So for one vendor might have all of the above processes. But for another, might not have some processes (Eg: Image processing). Following things should be able to obtain from the selected solution.</p>
  
  <ul>
  <li><code>I should be able to create custom workflows for new vendors</code></li>
  <li><code>Need to identify any failures that have been occurred within the workflow and perform retry mechanisms.</code></li>
  <li><code>Should be able to execute some functions parallelly (Eg: Image processing)</code></li>
  <li><code>Scalable</code></li>
  <li><code>Opensource</code></li>
  </ul>
  
  <p>So I was told to look into workflow managers like Nifi/Airflow/Falcon. I did some research on them but couldn't finalize the most suitable solution. </p>
  
  <p><em>NOTE: There is NO requirement to use Hadoop or any other cluster and data flow frequency is not that high</em></p>
  
  <p>Currently, I am thinking of using Nifi. But can anyone please give your opinion on that? What would be the best solution for my use case?</p>