ssage for a defined period of time, and it is the responsibility of the consumer to track messages. Kafka is also particu- larly useful for ingesting streaming data across a variety of use cases as described on the Apache Kafka website [Kafka]. 8.2.4.3 YARN (Yet Another Resource Manager) When the initial version of Hadoop was rolled out, it only supported batch processing, and a single NameNode managed the entire cluster for both job scheduling and resource management. A more TABLE 8.3 Sqoop and Flume Comparison Feature Sqoop Flume Basic difference Useful for ingesting data from relational databases Designed for ingesting streaming logs into Hadoop (e.g., JMS or spooling directory) Data flow Works well with any relational databased having Works well for streaming logfile data sources JDBC connectivity from multiple sources Type of loading Not event-driven Event-driven Destination HDFS, HBase, HIVE From multiple sources into HDFS Where to use? Parallel data transfer, helps mitigate excessive loads Flexible data ingestion tool, high throughput, to external systems, provides data interaction low latency, fault tolerant, linearly scalable programmatically Network Data and Optimization flexible approach was later introduced with YARN [YARN], which separates resource management and job scheduling to allow different applications to separately negotiate resources from a global resource manager. YARN thus allows multiple diverse user applications to run on a multitenant plat- form, SO that users are no longer confined to the I/O intensive, high latency MapReduce framework, and alternate processing frameworks supporting near real-time processing (such as that described in Section 8.2.4.6) can be utilized within the same data lake. 8.2.4.4 Storage: Nonrelational Databases, NoSQL, Not Only SQL In addition to HDFS, the Hadoop ecosystem defines several nonrelational "not only Structured Query Language (SQL)" (NoSQL) storage approaches. Table 8.4 illustrates a comparison approach of different types of NoSQL databases [Acci14]. 