Image clustering with deep learning
Few clustering algorithms have any kind of loss. That is pretty much only k-means and a few closely related methods that fit this narrow world view of "everything is optimizing a loss".

Now if your features would minimize the same loss, then the result should be close to just doing k-means on the raw pixels - which is known to not work well. You do want features that lose a lot of irrelevant information.