feature selection techniques
Doing that is a very good idea. The problem is that doing that is very hard.
Feature selection is a NP-complete problem.
The practical meaning as that we don't know any fast algorithm that can select only the needed feature.

In the other direction, omitting features that don't have mutual information (MI) with the concept might cause you to throw the features you need most.
There are cases in which a single feature is useless but given more features it becomes important.

Consider a concept which is the XOR of some features. Given all the features, the concept is totally predictable. Given one of them, you have 0 MI.

A more real life example is of age at death. Birth date and death date give you the age. One of them will have very low correlation (due to increase in life expectancy). 

In practice, omitting features with low MI is OK. Many learning algorithms are using MI so they won't be able to use the omitted variables anyway.
As for the selection itself, there are many algorithms, usually heuristics or approximation algorithm that are quite handy.