Workflow for Iteratively Capturing and Improving Machine Learning Models
For Modelling , it's probably best to go with Partial learning models. Explore DASK. 

If you can, getting cloud support to run your models is also excellent idea. For examples Google colab.

For all models keep log of model accuracy and cross validation accuracy. It's good idea to keep these details in small table and build a line graph which shows accuracy details for each version.

For latency, use Pytest and build differential testing code so that after each iteration of testing it provide tesrit results.