Sparse Representation vs Dense Representation
Sparse representations are not always better than Dense, though they do have an upper hand. In terms of various inputs being handled (varying amount of information), without a sparse prior we shall be trying to learn same size fit for all representation, that may be inadequate. Here dense representations will have lesser degrees of freedom, resulting in non-linear relationship. Overall our representation mathematically needs to mimic the topology of underlying manifold. But in how many scenarios can we have sparse representation, is the bigger question/problem, where we we have sufficient uncorrupted identities. Approaches like SLR dictionary decomposition come in handy in such scenarios.