Can I arbitrarily eliminate 20% of my training data if doing so significantly improves model accuracy?
One flaw in your procedure is the use of SMOTE before splitting in train/test. This should be avoided as you may have synthetic examples in the test data which generation depends on training data and that will be highly close to this data in your feature space (as SMOTE uses Euclidean distance).

Moreover, if most of the minority data belongs to the not-skewed region of your specific variables, these points will be also over sampled and so this reduction in the variables space will produce an overly optimistic performance which does not reflect the real distribution of the data.