Uber Engineering released its resource scheduling and management utility Peloton as open source today. According to company engineers, Peloton was developed to fill a missing niche in Uber’s internal software suite, allowing web-scale compute cluster management, resulting in improved resource utilization across its infrastructure.
Min Cai, Compute Cluster Platform senior staff engineer at Uber, and Mayak Bansal, staff engineer at Uber Big Data, outlined a number of use-case scenarios for Peloton in a blog post. This includes preempting traffic spikes, economizing resource use by running complementary workloads together, and allocating dedicated disaster recovery resources for batch jobs until data center failover.
The two wrote that prior to building Peloton, which was first introduced in November, Uber’s workloads were spread across a large number of clusters and “to their knowledge” there were no existing tools for economizing the work cross-cluster at the scale Uber needed.
Cai and Bansal wrote that support for stateless services is planned for the near future.