How does maximum likelihood estimation connect to the logistic regression cost function?

Logistic regression models the probability of a binary outcome using the sigmoid function: p(y=1|x) = σ(xᵀβ) = 1/(1+e^(-xᵀβ)). To fit β, we use maximum likelihood estimation (MLE) — finding the parameters that make the observed labels most probable under the model.

For a single example, the likelihood is p^y · (1-p)^(1-y) (this equals p when y=1 and 1-p when y=0 — a compact way to write both cases in one expression). Taking the log of the product of likelihoods across all n examples gives the log-likelihood, and negating it (since we minimise rather than maximise) yields the binary cross-entropy loss: L = -1/n Σ[yᵢ log(pᵢ) + (1-yᵢ) log(1-pᵢ)]. This is exactly the loss function scikit-learn's LogisticRegression minimises.

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import log_loss

model = LogisticRegression().fit(X_train, y_train)
probs = model.predict_proba(X_test)[:, 1]

# log_loss computes exactly the negative log-likelihood (cross-entropy)
loss = log_loss(y_test, probs)
print(f'Cross-entropy loss: {loss:.4f}')

# Manual implementation of the cross-entropy formula
def manual_log_loss(y_true, p_pred, eps=1e-15):
    p = np.clip(p_pred, eps, 1 - eps)  # avoid log(0)
    return -np.mean(y_true * np.log(p) + (1 - y_true) * np.log(1 - p))

print(manual_log_loss(y_test, probs))  # matches log_loss above

Why log instead of raw likelihood: multiplying many small probabilities together causes numerical underflow; taking the log converts the product into a sum, which is numerically stable and also turns the optimisation into a convex problem that gradient-based methods can solve reliably.