To demonstrate that the rotational invariance of MLP classifiers can lead to embeddings in end-to-end training whose features do not carry meaning individually (rotation occurs), we calculated the CKA with linear and RF kernel and evaluated the performance of RFs on embeddings generated by convolutional encoders. The same experiment shows that CKA can differentiate between learned embeddings stemming from different encoding methods, even if the performance similarity is high.