Python / PyTorch Fundamentals Interview Questions
What loss functions does PyTorch provide and when do you use each?
Loss functions (criteria) measure the difference between predictions and targets. PyTorch provides them in torch.nn. Choosing the right one for your task is critical — using the wrong loss gives poor training signal even if the architecture is correct.
| Loss | Class | Task | Target dtype |
|---|---|---|---|
| Cross-entropy | nn.CrossEntropyLoss | Multi-class classification | Long (class indices) |
| Binary cross-entropy + logits | nn.BCEWithLogitsLoss | Binary / multi-label | Float |
| MSE | nn.MSELoss | Regression | Float |
| MAE / L1 | nn.L1Loss | Robust regression | Float |
| Huber / Smooth L1 | nn.HuberLoss / nn.SmoothL1Loss | Robust regression | Float |
| NLL Loss | nn.NLLLoss | After log-softmax | Long |
| KL Divergence | nn.KLDivLoss | Distribution matching | Float |
| Triplet Margin | nn.TripletMarginLoss | Metric learning | Float |
import torch, torch.nn as nn # Multi-class classification: CrossEntropyLoss # Input: (N, C) logits â raw, before softmax # Target: (N,) class indices â dtype=long ce = nn.CrossEntropyLoss() logits = torch.randn(4, 10) # 4 samples, 10 classes targets = torch.tensor([2, 5, 0, 9]) # true class indices loss = ce(logits, targets) # Binary classification: BCEWithLogitsLoss # Numerically stable (fuses sigmoid + BCE) bce = nn.BCEWithLogitsLoss() preds = torch.randn(4) # logits, NOT sigmoid output true = torch.tensor([1.,0.,1.,0.]) loss_b = bce(preds, true) # Class weighting for imbalanced datasets weights = torch.tensor([1.0]*9 + [10.0]) # class 9 is rare ce_w = nn.CrossEntropyLoss(weight=weights) # Label smoothing (reduces overconfidence) ce_ls = nn.CrossEntropyLoss(label_smoothing=0.1) # Regression: MSE vs Huber mseLoss = nn.MSELoss() huberLoss = nn.HuberLoss(delta=1.0) # L2 near 0, L1 for large errors pred_r = torch.randn(4) true_r = torch.randn(4) print(mseLoss(pred_r, true_r)) print(huberLoss(pred_r, true_r))
Critical gotcha: nn.CrossEntropyLoss expects raw logits (before softmax), not probabilities. It internally applies log-softmax, so applying softmax first leads to double-softmax and incorrect training.
What dtype must the target tensor be for nn.CrossEntropyLoss?
Why is nn.BCEWithLogitsLoss preferred over applying sigmoid then nn.BCELoss?
More Related questions...
