Online news classification
The usual rule is to ask only one question per post.  I will answer your first question.

Build a dictionary, once, in advance.  With a bit of effort you should be able to construct a dictionary that has all the words that are likely to appear in the future. Take all news articles over the past year, or all of Wikipedia, or some other very large collection, and it's likely to contain all the words you need; the likelihood that you're missing some important word is pretty low.  Now, each time you see an article, throw away any words not in the dictionary.  That way, you won't need to vectorize the old news over and over every time; you can do it once and be done.  That should improve performance.

Another variation: if you see a new word on some day, you can add that word to the dictionary; at the end of the day retokenize everything you saw that day.