Why does my sagemaker model need a larger instance for real-time inference
The values attribute of classification-type SkLearn tree models is essentially a data matrix (n_leaves, n_classes) (probability distributions associated with each leaf node).
In the first experiment this matrix has 4 columns, in the second experiment it has 10 columns - a 2.5X increase in size.
I have a sagemaker endpoint that serves a Random Forest Classifier SKLearn model. This model predicts the kind of user someone is based on there interaction on a website. I have had two releases for this model.
The first model had 4 kinds of user: 'user-type-1', 'user-type-2', 'user-type-3', 'other'
The second release differed from the first in that there was more training data used and there were 10 types of user 'user-type-1', 'user-type-2', 'user-type-3', 'user-type-4', 'user-type-5', 'user-type-6', 'user-type-7', 'user-type-8', 'user-type-9','other'. To be more specific, in the first model many users got predicted as 'other' whereas in the second model many of these users were in one of the new categories.
The parameters for the random forest was the same in both models.
Question: The second model uses a lot more memory than the first and I cannot figure out why. Both models are of similar size and the same number of calls to the endpoint are being made under the second release. Any ideas why I might need a larger instance with more memory in the second model than the first?