finding similarity of a new datapoint I have built a recommendation engine using cosine similarity. When I want to find all the records similar to a given record that is already present in the dataset it works. Consider a case, a user enters a new record which does not exist in the similarity matrix, to find and recommend similar records I am adding the new data point into the dataset built the similarity matrix and then pick the similar records, Is there a better way to handle this problem?
Adding more details
data=[&quot;document1&quot;, &quot;document2&quot;, &quot;document&quot;, ... &quot;documentN&quot;]
count_matrix = count.fit_transform(data)
cosine_sim = cosine_similarity(count_matrix, count_matrix)
Instead of transforming, and calculating the whole cosine similarity matrix every time you receive a new data point, store your existent data, and whenever you get a new sample calculate the cosine similarity of a new sample against the existent data. Then append and store it.
That is, you need a two function:

First function - Calculate the distance matrix among N existent documents, then store it to the [N x N matrix], also you need to store transformed data, count_matrix.

Second function - When you get a new data point, calculate the cosine similarity of a new datapoint to all existent datapoints in the count_matrix, and append the new [1 x N+1] list to the existent cosine similarity matrix. Also, update the count_matrix. Afterward, store both.


Simply, instead of creating [N+1 x N+1] matrix, store the data and when you have new data, calculate [1 x N+1] and combine it with the existent matrix and store it.