What are the most important tensor operations in PyTorch?

PyTorch provides a rich set of tensor operations covering arithmetic, shape manipulation, reduction, and linear algebra. Most have both a functional form (torch.add) and a method form (tensor.add), plus in-place variants with a trailing underscore (tensor.add_).

import torch

a = torch.tensor([[1.,2.,3.],[4.,5.,6.]])
b = torch.tensor([[7.,8.,9.],[10.,11.,12.]])

# ── Arithmetic
print(a + b)          # element-wise add
print(a * b)          # element-wise multiply (Hadamard)
print(torch.matmul(a, b.T))  # matrix multiply  (2×3) @ (3×2) → (2×2)
print(a @ b.T)        # same with @ operator

# ── Shape manipulation
print(a.shape)                    # torch.Size([2, 3])
print(a.reshape(3, 2))            # (3, 2) — new view if possible
print(a.view(6))                  # (6,)   — must be contiguous
print(a.unsqueeze(0).shape)       # (1, 2, 3) — add dim
print(a.squeeze(0).shape)         # removes dim of size 1
print(torch.cat([a, b], dim=0))   # (4, 3) — concatenate rows
print(torch.stack([a, b], dim=0)) # (2, 2, 3) — new dim
print(a.permute(1, 0))            # (3, 2) — transpose

# ── Reduction
print(a.sum())           # scalar sum
print(a.sum(dim=1))      # sum along rows → (2,)
print(a.mean(dim=0))     # mean along columns → (3,)
print(a.max(), a.min())
print(a.argmax())        # index of max (flattened)

# ── In-place (modifies tensor, avoids memory allocation)
a.add_(1)   # a += 1
a.mul_(2)   # a *= 2
# Warning: in-place ops on tensors requiring grad can cause issues!

Key distinction: reshape returns a view when possible (no copy) and falls back to a copy if the tensor is not contiguous. view always requires a contiguous tensor and always returns a view. Use contiguous().view() or just reshape() to be safe.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.