What are Entropy, Information Gain, and Gini Index used for in Decision Trees?

Question

MLQuiz · Accepted Answer

Entropy measures impurity; Information Gain = Entropy(parent) - weighted average Entropy(children). Gini Index is an alternative impurity measure. Algorithms like ID3 use Information Gain; CART uses Gini.