What is a manifold for Unsupervised Learning? I've been watching Dr. G. Hinton lectures on Neural Networks in Machine Learning, and in one of the lectures he explains what the goals of Unsupervised Learning are. 

I am having trouble understanding the part where high-dimensional inputs such as images live on or near a low-dimensional manifold (or several such manifolds). What is a manifold exactly, and why is this the case?

Thanks!
Depends who you ask but generally mannifold is just some  structure in a (high) dimensional space having finite dimension : a line, a curve, a plane, a rock, a sphere, a ball, a cylinder, a torus, a "blob"... something like this :



If you ask a mathematician 
they would say that it is a general term describing "a curve" (dimension 1) or "surface" (dimension 2), or a 3D object (dimension 3)... for any possible finite dimension $n$. A one dimensional manifold is simply a curve (line, circle...). A two dimensional manifold is simply a surface (plane, sphere, torus, cylinder...). A three dimensional manifold is a "full object" (ball, full cube, the 3D space around us...). 

TO ANSWER YOUR QUESTION what he is referring is that information in this high-dimensional space can be compressed and perserved in some smaller space. (remember images when quantified live in some space that contains information about them)