Prev Next

Python / PyTorch Fundamentals Interview Questions

What are learning rate schedulers in PyTorch and how do you use them?

A learning rate scheduler adjusts the learning rate during training — typically starting high for fast initial progress and decaying for fine-grained convergence. Schedulers wrap an optimizer and must be stepped after each epoch (or each batch for some schedulers).

Common LR schedulers
SchedulerBehaviourStep
StepLRMultiply lr by gamma every step_size epochsPer epoch
MultiStepLRDecay at specified milestone epochsPer epoch
ExponentialLRlr *= gamma every epochPer epoch
CosineAnnealingLRCosine decay from lr to eta_minPer epoch
OneCycleLRWarmup then cosine decay (superconvergence)Per batch
ReduceLROnPlateauReduce lr when metric stops improvingPer epoch (with metric
CosineAnnealingWarmRestartsCosine with periodic restartsPer epoch
;
import torch, torch.optim as optim
import torch.nn as nn

model = nn.Linear(128, 10)
optimizer = optim.SGD(model.parameters(), lr=0.1)

# ── StepLR: multiply lr by 0.1 every 30 epochs
scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=30, gamma=0.1)

# ── CosineAnnealingLR: smooth cosine decay
scheduler = optim.lr_scheduler.CosineAnnealingLR(
    optimizer, T_max=100, eta_min=1e-6
)

# ── OneCycleLR: requires total_steps at init
scheduler = optim.lr_scheduler.OneCycleLR(
    optimizer,
    max_lr=0.1,
    total_steps=100 * len([1]*1000),  # epochs * batches_per_epoch
)

# ── ReduceLROnPlateau: triggered by validation loss
scheduler = optim.lr_scheduler.ReduceLROnPlateau(
    optimizer, mode="min", factor=0.5, patience=5, verbose=True
)

# ── Training loop integration
for epoch in range(100):
    train_loss = 0.0  # ... train ...
    val_loss   = 0.0  # ... validate ...

    # Most schedulers: step after epoch
    scheduler.step()              # for StepLR, CosineAnnealingLR etc.
    # scheduler.step(val_loss)    # for ReduceLROnPlateau (needs metric)

    # Check current lr
    current_lr = optimizer.param_groups[0]["lr"]
    print(f"Epoch {epoch}: lr={current_lr:.6f}")

When should you call scheduler.step() for most epoch-based learning rate schedulers?
Which PyTorch scheduler is most suitable when you want to reduce learning rate only if validation loss stops improving?

Invest now in Acorns!!! 🚀 Join Acorns and get your $5 bonus!
Acorns Logo

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Earn passively and while sleeping

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Robinhood Logo

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull Logo

Webull! Receive free stock by signing up using the link: Webull signup.

More Related questions...

What is PyTorch and what are its key advantages over other deep learning frameworks? What is a PyTorch tensor and how does it differ from a NumPy array? What are the most important tensor operations in PyTorch? What are tensor data types (dtypes) in PyTorch and why do they matter? How does broadcasting work in PyTorch and what are the rules? What is autograd in PyTorch and how does it compute gradients? What is the computation graph in PyTorch and how does the dynamic graph differ from a static graph? How do torch.no_grad() and tensor.detach() differ, and when do you use each? What is nn.Module and how do you build a custom neural network in PyTorch? What are nn.Sequential and other container modules in PyTorch? What built-in layers does PyTorch's nn module provide and how do you use the most common ones? What are activation functions in PyTorch and how do you apply them? What are the most important loss functions in PyTorch and when do you use each? What optimizers does PyTorch provide and how do you configure them? What are learning rate schedulers in PyTorch and how do you use them? What are the most common built-in layers in torch.nn and what do they do? How do you initialise weights in a PyTorch model? What loss functions does PyTorch provide and when do you use each? What optimizers does PyTorch provide and how do you choose between them? What are learning rate schedulers in PyTorch and how do you use them? What activation functions are commonly used in PyTorch and how do you choose between them? What loss functions does PyTorch provide and how do you choose the right one? What optimizers does PyTorch provide and what is the difference between SGD, Adam, and AdamW? What is the standard PyTorch training loop and what does each step do? What are Dataset and DataLoader in PyTorch and how do they work together? How do you move tensors and models between CPU and GPU in PyTorch? What is the difference between model.parameters() and model.state_dict() in PyTorch? How do you save and load PyTorch models correctly, including full training checkpoints? What is overfitting and what regularization techniques does PyTorch support to address it? What is the vanishing/exploding gradient problem and how do you detect and fix it in PyTorch? What is weight initialization in PyTorch and why does it matter? What is the difference between nn.Parameter and a regular tensor attribute in nn.Module? How do you implement and use learning rate schedulers in PyTorch? How do you debug a PyTorch training loop where the loss is not decreasing or is NaN? What is the difference between torch.tensor() and torch.Tensor() (capital T) for creating tensors? How does gradient accumulation work in PyTorch and when would you use it? What is mixed precision training in PyTorch and how do you implement it with torch.cuda.amp? What is torch.compile() and how does it speed up PyTorch model execution? What is the difference between batch size, epoch, and iteration in PyTorch training? How do you compute and track evaluation metrics like accuracy during PyTorch training? What is the purpose of torch.manual_seed() and how do you ensure reproducibility in PyTorch? How does PyTorch handle multi-dimensional indexing and slicing of tensors? What is the difference between.view(),.reshape(), and.contiguous() in PyTorch, and why does it matter? How do you freeze layers and perform transfer learning / fine-tuning in PyTorch? What is the purpose of torch.utils.data.random_split() and how do you create train/validation/test splits in PyTorch? What is Batch Normalization in PyTorch and how does it differ from Layer Normalization? How do you implement and use a custom loss function in PyTorch? What is torch.compile() vs TorchScript and how do you export a PyTorch model for production deployment?
Show more question and Answers...

Uvicorn Fundamentals Interview Questions

Comments & Discussions

Interviews Questions

About Javapedia.net

Javapedia.net is for Java and J2EE developers, technologist and college students who prepare of interview. Also this site includes many practical examples.

This site is developed using J2EE technologies by Steve Antony, a senior Developer/lead at one of the logistics based company.

contact: javatutorials2016[at]gmail[dot]com

Kindly consider donating for maintaining this website. Thanks.

Copyright © 2026, javapedia.net, all rights reserved. privacy policy.

DMCA.com Protection Status