why apache spark is used?
Spark is a general-purpose distributed data processing engine that is suitable for use in a wide range of circumstances. ... Tasks most frequently associated with Spark include ETL and SQL batch jobs across large data sets, processing of streaming data from sensors, IoT, or financial systems, and machine learning tasks.