TorchServe (Experimental)
TorchServe is a flexible and easy to use library for serving PyTorch models in production performantly at scale. It is cloud and environment agnostic and supports features such as multi-model serving, logging, metrics, and the creation of RESTful endpoints for application integration. TorchServe was jointly developed by engineers from Facebook and AWS with feedback and engagement from the broader PyTorch community. The experimental release of TorchServe is available today. Some of the highlights include:

Support for both Python-based and TorchScript-based models
Default handlers for common use cases (e.g., image segmentation, text classification) as well as the ability to write custom handlers for other use cases
Model versioning, the ability to run multiple versions of a model at the same time, and the ability to roll back to an earlier version
