I am newbie to data science and I do not understand the difference between fit and fit_transform methods in scikit learn.I have seen similar questions but I did not get intuition from answers. Can anybody simply explain why we might need to transform data?What does it mean fitting model on training data and transforming to test data.Does it mean for example converting categorical variables into numbers in train and transform new feature set to test data?Any help would be really appreciated. Please do not refer to scikit learn documentations. Try to explain as simple as possible on your own words :)
Difference between fit and fit_transform in scikit_learn models?
python;scikit learn
To center the data (make it have zero mean and unit standard error), you subtract the mean and then divide the result by the standard deviation.$$x' = \frac{x-\mu}{\sigma}$$You do that on the training set of data. But then you have to apply the same transformation to your testing set (e.g. in cross-validation), or to newly obtained examples before forecast. But you have to use the same two parameters $\mu$ and $\sigma$ (values) that you used for centering the training set.Hence, every sklearn's transform's fit() just calculates the parameters (e.g. $\mu$ and $\sigma$ in case of StandardScaler) and saves them as an internal objects state. Afterwards, you can call its transform() method to apply the transformation to a particular set of examples.fit_transform() joins these two steps and is used for the initial fitting of parameters on the training set $x$, but it also returns a transformed $x'$. Internally, it just calls first fit() and then transform() on the same data.