What exactly is "sparse representation"?
Take a sine-like signal $s$. In the appropriate Fourier $\mathcal{F}$ domain, it is represented by two "peaks", the other coefficients being zero. Fourier is a sparse representation for sines or close-to-sine signals. Conversely, a zero signal, except for a few values, is sparse in its original domain.

In narrow sense, a sparse representation of data is a representation in which few parameters or coefficients are not zero, and many are (strictly) zero. This can be measured by the $\ell_0$ count index, which yields the number of non-zero components. Here, $\ell_0(\mathcal{F(s)})=2$. 

However, in practice, it is very rare to have few exactly non-zero components, and a lot being exactly zero. This happens especially as floating point computations are rarely exactly zeros, albethey very small. You can see for instance the simulations for Why is $\exp(\ln(x))-x \neq 0$ in floating point arithmetic?

So, broader senses are accepted in practice, amounting to few high components, many small components.
I will thus extend the discussion on terminology on either "sparse" and "representation". 
On sparsity, or parsimony, one should mention the economy principle, law of parsimony or  Ockham's razor:


  An explanation of a thing or event is made with the fewest possible
  assumptions


Initially, this pre-scientific principle has being used in theological debates.
In science, it is  used as a heuristic guide in the selection or development  of theoretical models. One early example is the geocentric model. Starting from the belief that Earth was the center of the Universe, further observations revealed complicated star motion patterns, building a model with complex (epicycloidal) curves, requiring many parameters to explain. Until some understood that the hypothesis of a centered earth was not appropriate, and that other models could provide simpler explanations, and formulae, with the same, or a better, prediction or description power. Kepler or Newton laws were way more efficient.

Another example is polynomial representation: suppose that you have fifteen couples of $x_i,y_i$ values that you want to use for prediction. Of course a Lagrange polynomial of degree 14 will fit then perfectly. Should it be used for prediction? Probably not, as data are imprecise, and the polynomial extrapolation will be unstable; maybe, a one or two degrees fit, according to some cost function, might be more appropriate. The idea is that two or three parameters (slope/intercept/second degree term) could be more efficient for a given task.

We now have the major ingredients. Take a set of data $d_i$,   with meaningful content. The data generally has many dimensions, $i\in \mathcal{I}$ (think about the number of pixels in images). There is some expectation that there exists a combination of a small subset of features (sparsity) $c_k$ that can faithfully describe  the dataset, or predict, infer from this dataset. In other words, with $k\in \mathcal{K}$, with cardinals such that $|\mathcal{K}| \ll |\mathcal{I}|$, there is a transformation $T$ such that the quantities $d_i - T(c_k)$ are small enough.

In the case of sines, $|\mathcal{K}|=2$ is sufficient in a Fourier basis. In practice, one tries to  have sequences of $c_k$ such that most of them are very small (if not zero), and few of them recovers data correctly. There are measures to quantify the sparseness of a sequence, and to quantify how close the sparse representation is to the actual goal.

Transformations  to represent data are standard tools from harmonic analysis (Fourier, Wavelets, filter banks, Radon, bases or redundant frames), statistical analysis (least-square regression, PCA, PLS, lasso, ridge regression), clustering and segmentation tools, etc. The distance is often  some norm or quasi-norm. Even neural networks strive to provide learned sparse representations from a training set.

Sparsity indices are 0-homogeneous (Gini index, norm ratios) or akin to norms or quasi-norms for minimization purposes. Some use a milder description, i.e. the compressibility, reordering coefficients $T(c_k)$ with decreasing magnitudes $|(c_{\sigma(k)}|$ and studying the exponent $\alpha\le 0$ such that:

$$\|d_i - T(c_{\sigma(k)})\| \leq K|k|^{\alpha}$$

or "how fast   compressible representations can approximate a data or a dataset".

I consider the idea of sparse representations to be a key driver in machine learning, classification, compresison, and you can find a plenty of ressources to reuse.