Make the ML model pass the Load Test
Assuming the server is as at least as powerful as the first local machine, you should obtain the same time or shorter duration by request. If not, there's a problem worth investigating. 

Then to improve further it could be useful to design the server side architecture so that:


the model is pre-loaded in memory, not read from file every time (huge bottleneck).
there are multiple instances of the program running in parallel so as to distribute the load.