News clustering on unlabeled datasets
Simply clustering and then labeling all the texts in cluster will yield very noisy results (and you might learn your clustering similarity function rather that the "real" one). while there are some techniques that handle noisy labels (e.g- this NIPS paper: http://papers.nips.cc/paper/5073-learning-with-noisy-labels.pdf), there are several other options to consider:


labeling a small amount of data (be sure to have label data from each category, assuming you know the categories), training a classifier, classifying new examples, adding the most certain one to the training set, than retraining the model and so on. For example, see this article: http://link.springer.com/article/10.1023/A:1007692713085
Active Learning can be useful in this scenario. E.g: http://www.kamalnigam.com/papers/emactive-icml98.pdf
Semi-supervised learning techniques. E.g: http://mlg.eng.cam.ac.uk/zoubin/papers/zglactive.pdf


It is a common practice to combine more than one method. Also, LDA might be useful here.