To make a sparse autoencoder,one way is add something to the function we want to minimize,which the further do average activity of hidden units to a sparsity parameter,the bigger your added thing will be.But why will this autoencoder become sparse?Why can't most unit become not big and not small in most of time instead of little time big and most time small?Why not straightly use the sum of ln(1+x^2) with x runs over all hidden units instead?
Is sparsity parameter really useful?
machine learning;neural networks