Python / PyTorch Fundamentals Interview Questions
What are the most common built-in layers in torch.nn and what do they do?
PyTorch's torch.nn module provides all the standard building blocks for neural networks. Understanding what each layer does mathematically and when to use it is fundamental to building effective models.
| Layer | Formula / behaviour | Typical use |
|---|---|---|
| nn.Linear(in, out) | y = xW^T + b | Fully connected / dense layer |
| nn.Conv2d(in, out, k) | 2D convolution with kernel k×k | Image feature extraction |
| nn.BatchNorm1d/2d | Normalise per feature/channel over batch | After linear/conv, before activation |
| nn.LayerNorm | Normalise over feature dim per sample | Transformers, NLP |
| nn.Dropout(p) | Zeros random fraction p during train | Regularisation |
| nn.Embedding(V,d) | Lookup table V vocab × d dim | Word/token embeddings |
| nn.ReLU/GELU/Tanh | Element-wise activations | After linear/conv layers |
| nn.Softmax(dim) | exp(x)/Σexp(x) along dim | Output probabilities (use LogSoftmax+NLLLoss or CrossEntropyLoss directly) |
| nn.MaxPool2d | Takes max over kernel window | Spatial downsampling in CNNs |
| nn.LSTM/GRU | Gated recurrent cells | Sequence modelling |
import torch, torch.nn as nn
# Linear layer internals
fc = nn.Linear(4, 8)
print(fc.weight.shape) # (8, 4) — note: output × input
print(fc.bias.shape) # (8,)
# Embedding
emb = nn.Embedding(num_embeddings=10000, embedding_dim=128,
padding_idx=0) # index 0 gets a zero vector
tokens = torch.tensor([1, 42, 7]) # shape (3,)
out = emb(tokens) # shape (3, 128)
# BatchNorm vs LayerNorm
bn = nn.BatchNorm1d(64) # input (N, 64) — normalises across N
ln = nn.LayerNorm(64) # input (N, 64) — normalises across 64 features
x = torch.randn(16, 64)
print(bn(x).shape) # (16, 64)
print(ln(x).shape) # (16, 64)
# Dropout only active during training
drop = nn.Dropout(p=0.5)
model = nn.Sequential(nn.Linear(32,32), drop, nn.ReLU())
model.train(); x_tr = model(torch.randn(4,32)) # 50% zeros
model.eval(); x_ev = model(torch.randn(4,32)) # all active
Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!
Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.
Invest now!!! Get Free equity stock (US, UK only)!
Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.
The Robinhood app makes it easy to trade stocks, crypto and more.
Webull! Receive free stock by signing up using the link: Webull signup.
More Related questions...
