Dense output from neural network I would like to create a loss function that encourages the output of the embedding of an autoencoder to be dense. I don't have an explicit condition for how density is defined, but one option would be that the average distance between a point and its k nearest neighbors should be minimized, where k is some parameter. This is balanced against the reconstruction error, so the embedded points won't all converge to the same value. Is there a way to construct a loss function of this, or something similar, in a way that allows the average density around each point to be backprogated through the autoencoder?
Very good question!
An approach towards your goal is Center loss. The loss rewards if points of a class gather around a center. You can adapt this approach for your Autoencoder.