Hesitate to drop a feature
That isn't easy to answer, as it highly depends on your taks. The only time where it is definitely recommended to remove a feature is when two attributes correlate perfectly (e.g. price and tax of a product). 

Removing attributes that do not correlate perfectly could worsen your accuracy as information gets lost. But keeping them could affect the computation time. If you don't worry about computation time, I would highly recommend you to keep every bit of information about the data.

If you are concerned about overfitting there are way better techniques such as regularization to solve this problem.

Note: Dimensionality reduction is mostly used to visualize high dimensional data, so that a human eye can detect patterns. For example by shrinking the dimension to 3D or even 2D one could observe an exponential dependency.