Torchserve let you scale your model serving and handle the peak traffic by supporting batch inference and multiple  workers that serve your model. Scaling can be done through management  API and settings through a configuration file. Also, metric API helps you to monitor your model serving through default and customizable metrics.
Other advanced settings such as the length of the queue for the received requests, maximum wait time for a batch of inputs and many other properties are configurable through a config file that can be passed to Torchserve when it is started.
Steps to serve your model with Torchserve