What is the difference between batch size, epoch, and iteration in PyTorch training?

These three terms are fundamental to understanding any training loop, and confusing them is a common source of bugs when computing metrics or setting up learning rate schedules.

import torch
from torch.utils.data import DataLoader, TensorDataset

# Example: 1000 training samples, batch size 32
X = torch.randn(1000, 20)
y = torch.randint(0, 5, (1000,))
dataset = TensorDataset(X, y)
loader  = DataLoader(dataset, batch_size=32, shuffle=True)

iterations_per_epoch = len(loader)   # = ceil(1000 / 32) = 32
print(f"Iterations per epoch: {iterations_per_epoch}")

n_epochs = 10
total_iterations = n_epochs * iterations_per_epoch
print(f"Total training iterations: {total_iterations}")  # 320

global_step = 0
for epoch in range(n_epochs):
    for batch_idx, (X_batch, y_batch) in enumerate(loader):
        # This inner loop body executes once PER ITERATION
        # X_batch.shape[0] == batch_size (32, except possibly the last batch)
        global_step += 1
        if global_step % 10 == 0:
            print(f"Epoch {epoch}, iteration {batch_idx}, global step {global_step}")

    print(f"--- Completed epoch {epoch} ---")  # runs once PER EPOCH

# Common pitfall: confusing scheduler.step() granularity
# Some schedulers (StepLR) expect ONE call per epoch
# Others (OneCycleLR) expect ONE call per iteration/step
# Mixing these up silently breaks the intended learning rate schedule

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.