Concept of Cross-Entropy Can anyone say the cross entropy Conceptually in machine learning for selecting best features, in simple way?
I confused about that.

You do not really understand something unless you can explain it to
your grandmother.
Albert Einstein
Entropy can be calculated using formula:-

$Entropy = -p\log_2 p − q\log_2 q$

Here $p$ and $q$ are probability of success and failure respectively in that node.
Entropy is also used with categorical target variable. It chooses the split which has
lowest entropy compared to parent node and other splits. The lesser the entropy,
the better it is.

Steps to calculate entropy for a split:


Calculate entropy of parent node
Calculate entropy of each individual node of split and calculate weighted
average of all sub-nodes available in split.