Concept of Mutual Information I want to get mutual information in iris dataset to select best feature but i confused about mutual information. 


What is concept of mutual information for selecting feature?
Can anyone explain it in simple way?



  You do not really understand something unless you can explain it to
  your grandmother.
  
  Albert Einstein
Mutual information (MI) quantifies the amount of information needed to express one variable with the help of another variable. If two variables are independent of each other, their MI is zero, else it will be greater than zero. MRMR, a mutual information based feature selection uses MI, which considers a feature effective if it has maximum MI with its class label(maximum relevance) and minimum MI with rest of the features(minimum redundancy). This combination of maximum relevance and minimum redundancy will ensure better performance with smaller feature dimension.