What is an autoencoder and what can a well-trained latent space be used for?

An autoencoder is a neural network trained to reconstruct its input through a bottleneck. The encoder f: X → Z maps inputs to a lower-dimensional latent space Z, and the decoder g: Z → X̂ reconstructs the input. Training minimises the reconstruction loss (e.g. MSE for continuous inputs, binary cross-entropy for binary) without any labels — it is an unsupervised learning technique.

The bottleneck forces the encoder to learn a compressed, information-dense representation. A well-trained latent space can be used for: (1) dimensionality reduction and visualisation (better than PCA for non-linear data); (2) anomaly detection (normal samples reconstruct well; anomalies have high reconstruction error); (3) de-noising (train with noisy input, clean target — denoising autoencoders); (4) generative modelling (Variational Autoencoders / VAEs impose a probabilistic structure on Z that enables generation).

import torch
import torch.nn as nn

class Autoencoder(nn.Module):
    def __init__(self, input_dim=784, latent_dim=32):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Linear(input_dim, 256), nn.ReLU(),
            nn.Linear(256, latent_dim)
        )
        self.decoder = nn.Sequential(
            nn.Linear(latent_dim, 256), nn.ReLU(),
            nn.Linear(256, input_dim), nn.Sigmoid()  # pixel values in [0,1]
        )

    def forward(self, x):
        z = self.encoder(x)
        return self.decoder(z)

ae = Autoencoder()
optimizer = torch.optim.Adam(ae.parameters(), lr=1e-3)
criterion = nn.MSELoss()

for X_batch, _ in loader:   # labels not used!
    X_flat = X_batch.view(X_batch.size(0), -1)  # flatten images
    X_hat  = ae(X_flat)
    loss   = criterion(X_hat, X_flat)
    optimizer.zero_grad(); loss.backward(); optimizer.step()

# Anomaly detection at inference:
ae.eval()
with torch.no_grad():
    X_hat = ae(test_samples)
    recon_error = ((test_samples - X_hat) ** 2).mean(dim=1)
# High recon_error => anomalous sample