MemoryError when concatenating a large data-frame
You should use sparse matrices 

When you are categorizing a column, you are creating many columns that mostly hold zeroes, and since dense data structures keep track of all the zeros, the memory usage explodes. Sparse matrices on the other hand only keep track of non-zero values and their indexes. There are many version of sparse matrices optimized for different use cases.