How do you evaluate ML model already deployed in production?
After the models are deployed in production, I'd monitor the following:

(1) The same metric you used to evaluate the performance of your model, in some cases it is accuracy, or it could be precision, recall, RMSE. I'd plot a daily time series charting the metric and see that it is still performing above a satisfactory threshold. There might be seasonality within the calender, the model performs well around certain months and not so well in other months. I'd compare the performance against the test/validation sets of the same months to account for seasonality.

(2) Apart from looking at the performance of the model, especially if one is working with shared computing resources, I'd also recommend keeping a close tab on the data aggregation runtime, model runtime, success rates of models runs over the past period.