Neural Mastery
← Practice
Decision Tree Information Gain via Entropyeasy

Decision Tree Information Gain via Entropy

easy Classical ML from Scratch 10–15 min

NumPy allowed · plain Python earns 10 bonus points

Measure Shannon entropy reduction achieved by partitioning a node in a decision tree.

Related Skills

Task

Implement entropy_info_gain(parent_labels, left_labels, right_labels): Shannon entropy is H(S) = -sum(p_k * log2(p_k)) (0.0 for empty set). Information Gain is H(parent) - (|left|/|parent|)*H(left) - (|right|/|parent|)*H(right). Round float to 4 decimal places.

Function Signature

entropy_info_gain(parent_labels: list[any], left_labels: list[any], right_labels: list[any]) -> float

Examples

Example 1: Perfect split gives 1.0 bit gain
Input: parent_labels = [0, 0, 1, 1] left_labels = [0, 0] right_labels = [1, 1]
Output: 1
Example 2: Useless split zero gain
Input: parent_labels = [0, 1, 0, 1] left_labels = [0, 1] right_labels = [0, 1]
Output: 0

Constraints

  • len(left_labels) + len(right_labels) == len(parent_labels)
Python3Saved ✓
import math

def entropy_info_gain(parent_labels: list[any], left_labels: list[any], right_labels: list[any]) -> float:
"""Compute Information Gain (entropy reduction) for split.
Round float to 4 decimal places.
"""
pass

Case 1: Perfect split gives 1.0 bit gain
Input: parent_labels = [0, 0, 1, 1] left_labels = [0, 0] right_labels = [1, 1]
Expected: 1
Case 2: Useless split zero gain
Input: parent_labels = [0, 1, 0, 1] left_labels = [0, 1] right_labels = [0, 1]
Expected: 0