1. A computer-implemented method for a machine learning system that mitigates catastrophic forgetting, comprising: receiving a data item; processing, by a first node that comprises a plurality of centroids, information from at least a portion of the data item to generate a first feature vector, wherein the first feature vector comprises a plurality of feature elements, each of the plurality of feature elements having a similarity value representing a similarity to one of the plurality of centroids; selecting a subset of the plurality of feature elements from the first feature vector, the subset containing one or more feature elements of the plurality of feature elements that have highest similarity values; generating a second feature vector from the first feature vector by replacing similarity values of feature elements in the first feature vector that are not in the subset with zeros; processing the second feature vector by a second node to determine an output; determining, by the first node, a novelty rating for the data item based on similarity values of the plurality of feature elements in at least one of the first feature vector or the second feature vector; determining a relevancy rating for the data item; and determining whether to update the first node based on the novelty rating and the relevancy rating.