1. A system for scheduling jobs, said system comprising: a plurality of interconnected compute nodes defining a cluster of compute nodes, the cluster including a NameNode and a multitude of DataNodes; the NameNode including at least one processor programmed to: learn central processing unit (CPU) capabilities and disk input/output (I/O) bandwidth capabilities of compute nodes in the cluster; determine an optimal number of containers for each application running on each DataNode of the multitude of DataNodes; and schedule execution of a plurality of tasks on compute nodes of the cluster based on: (i) the learned CPU capabilities and I/O bandwidth capabilities of compute nodes of the cluster, and (ii) at least one of the determined optimal number of containers.