How to recognize a two part term when the space is removed? ("bigdata" and "big data")
There is a nice implementation of this in gensim: http://radimrehurek.com/gensim/models/phrases.html

Basically, it uses a data-driven approach to detect phrases, ie. common collocations. So if you feed the Phrase class a bunch of sentences, and the phrase "big data" comes up a lot, then the class will learn to combine "big data" into a single token "big_data". There is a more complete tutorial-style blog post about it here: http://www.markhneedham.com/blog/2015/02/12/pythongensim-creating-bigrams-over-how-i-met-your-mother-transcripts/