Is generalizing a model, then removing the generalization good for FFNNs?
L1 and L2 regularizations matter only during training, they are a way to update the Network's weights in (hopefully) the right way. Once you use your model for prediction that doesn't matter anymore.

Dropout is active only during training. Once it's done, the Network usese all the trained nodes to make a prediction.

In other words, no need to remove regularization techniques manually.