Entropy and Information Gain

What is Entropy:

Entropy is the measure of impurity, disorder and uncertainty

In machine Learning: Average number of yes/no questions we need to guess the target class

 

How does entropy play?

Entropy controls how a Decision Tree decides to split the data. It actually effects how a decision tree draws its boundaries.

 

What is Information Gain:

Information Gain(IG) measures how much "information" a feature gives us about the class

 

Decision Tree Algorithm tries to split by attributes with highest Information Gain

 

https://www.youtube.com/watch?v=9r7FIXEAGvs

참고: https://medium.com/coinmonks/what-is-entropy-and-why-information-gain-is-matter-4e85d46d2f01