Python / Python Deep Learning and Neural Networks Interview Questions

How would you implement and train a simple feedforward neural network in PyTorch from scratch, without using nn.Sequential?

This question tests whether you understand the full PyTorch workflow: defining a custom nn.Module, implementing forward, and running the standard train loop. It is a common practical screen in ML engineering interviews.

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

# âââ 1. Define the model ââââââââââââââââââââââââââââââââââââââââââââ
class FeedForwardNet(nn.Module):
    def __init__(self, in_dim: int, hidden_dim: int, out_dim: int,
                 dropout: float = 0.1):
        super().__init__()
        self.fc1     = nn.Linear(in_dim, hidden_dim)
        self.bn1     = nn.BatchNorm1d(hidden_dim)
        self.relu    = nn.ReLU()
        self.drop    = nn.Dropout(dropout)
        self.fc2     = nn.Linear(hidden_dim, out_dim)
        self._init_weights()

    def _init_weights(self):
        nn.init.kaiming_uniform_(self.fc1.weight, nonlinearity='relu')
        nn.init.zeros_(self.fc1.bias)
        nn.init.xavier_uniform_(self.fc2.weight)
        nn.init.zeros_(self.fc2.bias)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        x = self.relu(self.bn1(self.fc1(x)))
        x = self.drop(x)
        return self.fc2(x)

# âââ 2. Create data ââââââââââââââââââââââââââââââââââââââââââââââââââ
torch.manual_seed(42)
X = torch.randn(1000, 20)
y = (X[:, 0] + X[:, 1] > 0).long()  # binary label
ds     = TensorDataset(X, y)
loader = DataLoader(ds, batch_size=64, shuffle=True)

# âââ 3. Instantiate model, loss, optimizer âââââââââââââââââââââââââââ
device    = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model     = FeedForwardNet(20, 64, 2, dropout=0.1).to(device)
criterion = nn.CrossEntropyLoss()
optimizer = optim.AdamW(model.parameters(), lr=1e-3, weight_decay=1e-4)

# âââ 4. Training loop âââââââââââââââââââââââââââââââââââââââââââââââââ
for epoch in range(30):
    model.train()
    epoch_loss = 0.0
    for X_b, y_b in loader:
        X_b, y_b = X_b.to(device), y_b.to(device)
        optimizer.zero_grad(set_to_none=True)
        loss = criterion(model(X_b), y_b)
        loss.backward()
        optimizer.step()
        epoch_loss += loss.item()
    if epoch % 5 == 0:
        print(f'Epoch {epoch:3d}: loss={epoch_loss / len(loader):.4f}')

Key interview checkpoints: (1) subclass nn.Module and call super().__init__(); (2) define all layers as attributes in __init__; (3) implement forward; (4) follow the zero-grad → forward → loss → backward → step order; (5) call model.train() before training and model.eval() before evaluation.

Take quiz

What is the required order of operations in a standard PyTorch training step?forward → backward → zero_grad → step

✗ Try again.

zero_grad → forward → loss → backward → step

✓ Correct! Well done.

step → zero_grad → forward → backward

✗ Try again.

backward → forward → zero_grad → loss → step

✗ Try again.

Why must all learnable layers be defined as attributes in nn.Module's __init__ rather than created inside forward()?forward() is called once; __init__ is called on every batch

✗ Try again.

Attributes defined in __init__ are registered as nn.Parameters or sub-modules, allowing .parameters(), .state_dict(), .to(device), and .train()/.eval() to find and manage them — layers created inside forward() are not registered and would be invisible to these methods

✓ Correct! Well done.

Python garbage collection removes layers created inside forward()

✗ Try again.

PyTorch requires all tensors to be created before the training loop begins

✗ Try again.

More Related questions...

What is a neural network and how does forward propagation work mathematically? Explain backpropagation mathematically. How does the chain rule enable computing gradients through many layers? What are the most common activation functions and why did ReLU replace sigmoid/tanh as the default? What are vanishing and exploding gradients, and what techniques are used to address them? Why does weight initialization matter in neural networks, and what is the difference between Xavier and He initialization? How does Batch Normalization work mathematically and why does it stabilize training? Compare SGD, SGD with momentum, RMSProp, and Adam optimizers. When do you choose each? How does Dropout work mathematically, and why does it act as regularization? Explain how convolutional layers work and why they are well-suited to image data. How do RNNs work and why did LSTMs solve the long-range dependency problem? What is the self-attention mechanism in Transformers and why did it replace RNNs for sequence modeling? What loss functions does PyTorch provide for classification and regression, and which to use when? What is transfer learning and how do you fine-tune a pretrained model in PyTorch? How does PyTorch's Dataset and DataLoader pipeline work, and what are the key performance considerations? Why is learning rate scheduling important and what are the most common strategies? What are the most effective regularization strategies for deep learning and how do they differ from classical ML regularization? What are embedding layers in deep learning and how are they different from one-hot encoding? How do you save and load PyTorch models correctly, and what is included in a proper checkpoint? What is mixed precision training and how does it speed up deep learning with torch.cuda.amp? What is the difference between model.eval(), torch.no_grad(), and torch.inference_mode()? When do you use each? How do you use GPUs in PyTorch and what are the key patterns for writing device-agnostic code? What are the differences between Batch Norm, Layer Norm, Group Norm, and Instance Norm? What is an autoencoder and what can a well-trained latent space be used for? How do you diagnose a neural network that is not training correctly from its loss curves? What is the mathematical setup of a Generative Adversarial Network (GAN) and what training challenges do they have? What is torch.compile and how does it speed up PyTorch model execution? Why do Transformers need positional encodings and how does sinusoidal encoding work? What are the most impactful hyperparameters to tune in deep learning and what is the recommended search order? What is an encoder-decoder architecture and how is it used for sequence-to-sequence tasks? What is model quantization in deep learning and how does PyTorch support it? What does a production-quality PyTorch training loop look like, incorporating all best practices? How does batch size affect deep learning training mathematically and practically? How do you choose the right layer type (Linear, Conv, Attention) for a given input modality? What evaluation metrics are most commonly used in deep learning tasks and how do you implement them in PyTorch? How do you export a PyTorch model for production deployment using TorchScript or ONNX? What is knowledge distillation and how does it compress large neural networks into smaller ones? What is self-supervised learning and how do contrastive methods like SimCLR learn representations? How would you implement and train a simple feedforward neural network in PyTorch from scratch, without using nn.Sequential?

Show more question and Answers...

Python Modern Generative AI and Agents Interview Questions

	Interviews Questions Java Spring Hibernate DevOps Testing API BigData Web Erlang AI Database Integration Cloud Python Scala Tools Golang SAP	About Javapedia.net Javapedia.net is for Java and J2EE developers, technologist and college students who prepare of interview. Also this site includes many practical examples. This site is developed using J2EE technologies by Steve Antony, a senior Developer/lead at one of the logistics based company. Follow Javapedia updates
	contact: javatutorials2016[at]gmail[dot]com
Kindly consider donating for maintaining this website. Thanks.
	Copyright © 2026, javapedia.net, all rights reserved. Privacy Policy · Terms of Service.

Python / Python Deep Learning and Neural Networks Interview Questions

How would you implement and train a simple feedforward neural network in PyTorch from scratch, without using nn.Sequential?

Comments & Discussions

Recently added questions