Python / PyTorch Fundamentals Interview Questions
What are the most important loss functions in PyTorch and when do you use each?
Choosing the right loss function is critical — it defines what the model is optimising for. PyTorch provides loss functions in torch.nn as modules and in torch.nn.functional as functions.
| Loss | Use case | Input / Target |
|---|---|---|
| nn.MSELoss | Regression — minimise squared error | pred: float, target: float |
| nn.MAELoss / L1Loss | Regression — robust to outliers | pred: float, target: float |
| nn.CrossEntropyLoss | Multi-class classification | pred: (N,C) logits, target: (N,) long |
| nn.BCEWithLogitsLoss | Binary classification (numerically stable) | pred: (N,) logits, target: (N,) float 0/1 |
| nn.NLLLoss | Used with log-softmax output | pred: (N,C) log-probs, target: (N,) long |
| nn.KLDivLoss | Distribution divergence (VAE, distillation) | pred: log-probs, target: probs |
| nn.HuberLoss | Regression robust to outliers | pred: float, target: float |
import torch, torch.nn as nn
batch = 8
# ── Regression
pred = torch.randn(batch, 1)
target = torch.randn(batch, 1)
mse = nn.MSELoss()(pred, target)
mae = nn.L1Loss()(pred, target)
print(mse, mae)
# ── Multi-class classification
logits = torch.randn(batch, 10) # raw scores, NOT softmax
labels = torch.randint(0, 10, (batch,)) # class indices, dtype=long
ce_loss = nn.CrossEntropyLoss()(logits, labels)
print(ce_loss)
# Class-weighted cross entropy (handle imbalance)
weights = torch.tensor([1.0]*9 + [5.0]) # upweight class 9
ce_weighted = nn.CrossEntropyLoss(weight=weights)(logits, labels)
# ── Binary classification (single output neuron)
bin_logits = torch.randn(batch) # single score
bin_labels = torch.randint(0, 2, (batch,)).float() # 0 or 1, float!
bce_loss = nn.BCEWithLogitsLoss()(bin_logits, bin_labels)
# BCEWithLogitsLoss = sigmoid + BCE in one numerically stable op
# ── Label smoothing (reduces overconfidence)
ce_smooth = nn.CrossEntropyLoss(label_smoothing=0.1)(logits, labels)
# reduction parameter
nn.MSELoss(reduction="mean") # default: mean over batch
nn.MSELoss(reduction="sum") # sum over batch
nn.MSELoss(reduction="none") # per-sample loss (no reduction)
Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!
Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.
Invest now!!! Get Free equity stock (US, UK only)!
Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.
The Robinhood app makes it easy to trade stocks, crypto and more.
Webull! Receive free stock by signing up using the link: Webull signup.
More Related questions...
