Theoretical treatment of unlabeled samples In a typical supervised learning setting with a few positive and a few negative examples, it is clear that unlabeled data carries some information that can benefit learning and that is not captured in the labeled data. For example one can estimate mean values, bounds and some other geometrical characteristics of the data-set with much higher precision if you do not discard the (massive) unlabeled data. 

On the other hand, the most common ML algorithms from Neural Networks to SVM do not take advantage of this information (at least in their standard, most common form). My question:


Is there any theoretical framework where unlabeled data is treated in the supervised setting?


I can think of semi-supervised ways to approach this (first cluster and then label the clusters). Are there any other?
You already dropped the key word "semi-supervised" in your question. Indeed, semi-supervised learning is the answer to your question. Search for this term with your favourite search engine or library catalog to find out about algorithms for semi-supervised learning.