How does the class_weight parameter mathematically address class imbalance in scikit-learn classifiers?

When classes are imbalanced (e.g. 95% negative, 5% positive), a model trained with the standard loss function will naturally lean toward predicting the majority class, since doing so minimises average loss across the imbalanced training set even while completely ignoring the minority class. The class_weight='balanced' option modifies the loss function to multiply each sample's contribution by a weight inversely proportional to its class frequency: weight_c = n_samples / (n_classes × n_samples_c).

This re-weighting effectively makes errors on minority-class examples count more in the total loss, forcing the optimisation to pay attention to them despite their rarity. For logistic regression, this directly modifies the cross-entropy loss term per sample; for SVMs, it modifies the penalty for margin violations on each class; for trees, it modifies the impurity calculation to weight minority-class samples more heavily when computing splits.

from sklearn.linear_model import LogisticRegression
from sklearn.utils.class_weight import compute_class_weight
import numpy as np

y = np.array([0]*950 + [1]*50)  # 95%/5% imbalance

weights = compute_class_weight('balanced', classes=np.unique(y), y=y)
print(weights)  # [0.526, 10.0] — minority class weighted 19x more

# weight_0 = 1000 / (2 * 950) = 0.526
# weight_1 = 1000 / (2 * 50)  = 10.0

# Apply during training
model = LogisticRegression(class_weight='balanced')
model.fit(X_train, y_train)

# Custom weights also supported
model_custom = LogisticRegression(class_weight={0: 1, 1: 15})

# Tree-based models support the same parameter
from sklearn.ensemble import RandomForestClassifier
rf = RandomForestClassifier(class_weight='balanced')

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.