Is it OK to train a binary classifier using all the extremely imbalanced data if the majority class is negative? I'm training a neural network as a binary classifier for text classification. The data is very imbalanced, where the ratio of TRUE:FALSE is approximately 100:10000

Intuitively, it feels like using all the negative samples would prevent the classifier from learning invalid patterns (that might otherwise be learned using undersampling, for example).

Am I underestimating the effect of the imbalance on classifier performance?
An imbalance of this magnitude is definitely a problem. Because machine learning algorithms penalize model complexity and try to maximize fit, a model that always predicts "False" will often outperform any other model because it predicts the right outcome for 99% of the data and is super simple!  

What one should do in this case is downsample (http://www.simafore.com/blog/handling-unbalanced-data-machine-learning-models), train the model, and then scale the resulting predictions back up to reflect the imbalance in the original data.