Tensors: The Language of GPUs

You already know scalars, vectors, and matrices. But deep learning does not stop at 2D. A color image needs three layers (Red, Green, Blue) — that is a 3D tensor. A batch of images adds another dimension — 4D. A batch of videos? 5D.

In this article, you will climb the dimension ladder, understand why EVERY AI input gets converted to a tensor, and learn why GPUs demand batches of data rather than one example at a time.

The Dimension Ladder

A tiny 4x4 RGB image shows the dimension ladder in action. Grayscale: a matrix with shape (4, 4) = 16 values. Color: a 3D tensor with shape (3, 4, 4) = 48 values. Three layers stacked: Red, Green, Blue. Pixel at position (2, 3): Red=220, Green=45, Blue=180 — purple.

import numpy as np

# Scalar (0D)
scalar = np.array(42)          # Shape: ()

# Vector (1D)
vector = np.array([1.0, 2.0, 3.0])  # Shape: (3,)

# Matrix (2D)
matrix = np.zeros((4, 4))      # Shape: (4, 4)

# 3D Tensor (RGB image)
rgb_image = np.random.randint(0, 256, (3, 4, 4))  # Shape: (3, 4, 4)

print(rgb_image.shape)  # (3, 4, 4)
print(rgb_image[0, 2, 3])  # Red value at position (2,3)

A 0D scalar holds a single value. But with each new dimension, information density grows: a 768-dimensional vector encodes the meaning of an entire word, a matrix transforms that meaning — and a 4D tensor can push 32 images through a neural network simultaneously.

Interactive: Three Views of Tensors

Tensors are abstract — but different analogies make them tangible. Switch between three perspectives to understand what a tensor is: a data cube for stacking, a mathematical transformation, or a numbered storage system.

Why ML Thinks in Tensors

GPUs and TPUs process dense, fixed-size numerical arrays with vectorized instructions. Therefore every input — image, text, audio, tabular data — must be "translated" into a tensor with a defined shape. The GPU speaks only one language: tensor.

The shape tells you what each dimension means. Wrong shape = wrong interpretation = bugs or nonsensical results. The shape of a tensor is the single most important piece of metadata in deep learning.

RuntimeError: mat1 and mat2 shapes cannot be multiplied
  (32x768 and 512x256)

# Expected: (32, 512) @ (512, 256) -> (32, 256)
# Got: (32, 768) @ (512, 256) -> Error!

Why fixed size? GPUs need to know the memory layout in advance. Variable-length inputs get padded to a fixed maximum length or truncated.

# PyTorch: Channels First (B, C, H, W)
pytorch_image = torch.randn(1, 3, 256, 256)

# TensorFlow: Channels Last (B, H, W, C)
tf_image = tf.random.normal([1, 256, 256, 3])

# Convert PyTorch -> TensorFlow
tf_image = pytorch_image.permute(0, 2, 3, 1)
# (1, 3, 256, 256) -> (1, 256, 256, 3)

import numpy as np
import torch

# NumPy -> Tensor (on GPU)
np_array = np.array([[1.0, 2.0], [3.0, 4.0]])
tensor = torch.from_numpy(np_array).cuda()

# Tensor -> NumPy (back to CPU)
np_back = tensor.cpu().numpy()

# Tensor has gradient tracking
w = torch.tensor([2.0], requires_grad=True)
loss = (w * 3 - 6) ** 2
loss.backward()  # Gradient computed automatically
print(w.grad)    # tensor([0.])

Batching: Feeding the GPU

GPUs have thousands of cores all executing the same operation on different data simultaneously (SIMD — Same Instruction, Multiple Data). Processing one image at a time wastes most cores. That is why the batch dimension is prepended.

Three Reasons for Batching

GPU utilization: All cores busy simultaneously. A single image uses only a fraction of the available compute power.

Statistical stability: The gradient averaged over a batch is less noisy than the gradient of a single example. Training converges more smoothly.

Trade-off: Larger batch = better GPU utilization, but more memory usage. Batch size is a critical hyperparameter.

Shape Progression

import torch

# Single image: shape (3, 256, 256)
img = torch.randn(3, 256, 256)

# Batch of 32 images: shape (32, 3, 256, 256)
batch = torch.stack([torch.randn(3, 256, 256) for _ in range(32)])
print(batch.shape)  # torch.Size([32, 3, 256, 256])

# Memory: 32 * 3 * 256 * 256 * 4 bytes = ~25 MB