Back to Learning Area

Data Tables & Transformations (Matrices)

The 2D grid of numbers where every ML library primarily thinks.

  • Fundamentals
  • 12 min
  • Intermediate

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Matrices: tables of numbers

What is a matrix?

A matrix is a grid of numbers. Imagine a table. Only with numbers. Each row is a data point. Each column is a property. For example: 1,000 houses with 5 properties each result in a 1,000-row table.

Matrices in neural networks

Neural networks use matrices as weights. Weights are numbers in the network. They determine the transformation of inputs. A weight matrix transforms inputs into other numbers. This is how the network learns step by step.

Matrix multiplication

Matrices can be multiplied with each other. Matrix multiplication means: combining two matrices. The result is a new matrix. This operation is important in AI. Every network layer uses this calculation operation.

You already know vectors — ordered lists of numbers that represent data points. But what happens when you stack many vectors together? You get a matrix — a rectangular grid of numbers. Every table of numbers, every dataset, every neural network layer is built on matrices.

In this article, you will learn what matrices are, how to multiply them (and why the definition is not arbitrary), and why this single operation is the computational heart of modern AI.

What Is a Matrix?

Definition

Matrix

Analogy

Think of a spreadsheet with only numbers: rows are entries (houses), columns are properties (area, rooms, year built). That IS a matrix. Each row is a vector, and all vectors together form the grid.

Example

Real spreadsheets can have text, formulas, and empty cells. A mathematical matrix contains only numbers, has a fixed size, and supports algebraic operations like multiplication.
Matrix as Data Table

Rows = data points, columns = features. A 1000×5 matrix stores 1000 houses with 5 properties each.

Matrix as Transformation

The matrix defines HOW an input vector is turned into an output vector. Weight matrices in neural networks are transformations.

Three houses with the features area, rooms, and year built form a 3×3 matrix. Each row is a house, each column is a feature.

import numpy as np

# Three houses as a 3x3 matrix
houses = np.array([
    [120, 4, 1990],
    [85,  3, 2005],
    [200, 6, 1975]
])
print(houses.shape)     # (3, 3)
print(houses[1, 2])     # 2005 — year built of house 2

Caution

Misconception: Matrices Are Just Tables

Matrices are far more than data storage. Their real power lies in representing transformations. A weight matrix in a neural network does not store data — it defines HOW inputs are transformed into outputs.

Every tabular dataset in machine learning is a matrix: rows are examples, columns are features. And every fully connected network layer is a matrix transformation.

Matrix Multiplication — Why This Definition?

Matrix multiplication looks complicated at first — but it has a deep reason: It exactly describes the composition of two transformations. Apply matrix B first, then matrix A, and the result is the matrix AB.

  1. 1. Take row i of matrix A.
  2. 2. Take column j of matrix B.
  3. 3. Compute the dot product (multiply element by element and add up).
  4. 4. The result goes at position (i, j) of the result matrix C.
c_ij = a_i1·b_1j + a_i2·b_2j + … + a_in·b_nj
Matrix Multiplication

Each element c_ij of the result is the dot product of row i of A with column j of B. The connection to the vectors article: matrix multiplication consists of many dot products.

(m × n) · (n × p) = (m × p)
Dimension Rule

The inner dimensions must match. An (m×n) matrix times an (n×p) matrix produces an (m×p) matrix. If the inner dimensions do not match, multiplication is undefined.

import numpy as np

# Weight matrix W (2x3) times input vector x (shape (3,))
W = np.array([[0.5, -0.3, 0.8],
              [0.1,  0.7, -0.2]])
x = np.array([1, 2, 3])

print(W @ x)          # [2.3, 0.9]
print(W.shape, '@', x.shape, '→', (W @ x).shape)
# (2, 3) @ (3,) → (2,)

Caution

Misconception: A × B = B × A

Matrix multiplication is NOT commutative. A × B generally gives a different result from B × A. Often, only one of the two products is even defined because the dimensions do not match. Order matters!

In NumPy, * is the elementwise product (Hadamard) and @ is matrix multiplication. They give completely different results! The most common beginner mistake: using * when @ is intended.

import numpy as np
A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])

print(A * B)   # [[ 5, 12], [21, 32]] — elementwise
print(A @ B)   # [[19, 22], [43, 50]] — matrix multiplication

Interactive: Row Times Column

The starting values are A and B from the deep dive on the Hadamard product: A @ B = [[19, 22], [43, 50]]. Change a number in A or B (whole numbers from −10 to 10), and the product updates instantly. Pick a cell of the result: the box marks the row of A and the column of B it comes from. The chart shows one of the matrices as a transformation: it reshapes the unit square.

Matrices in Neural Networks — Why GPUs Calculate

Every fully connected layer of a neural network computes exactly one matrix multiplication: weight matrix times input plus bias. After that, the layer usually applies an activation function to the result, number by number; the heavy lifting, though, is still the matrix multiplication. Training means adjusting the entries of the weight matrix to reduce the error.

output = W × input + bias
Neural Network Layer

Shape Chain Through a Mini-Network

Input (5 data points) 5 × 3
Weights W1 3 × 4
Hidden Layer 5 × 4
Weights W2 4 × 2
Output 5 × 2

Instead of processing one vector at a time, many inputs are stacked into a matrix. This way, an entire batch (e.g., 32 images) is processed in ONE matrix multiplication. This is dramatically faster. The inputs then sit as rows on the left: X @ W instead of W × x, as in the code below; the weight matrix has the shape (features in × features out).

A100 GPU (FP16, Tensor Cores)

312 TFLOPS

CPU (order of magnitude)

~1–5 TFLOPS

Speedup (peak performance)

~60–300 ×

Why are GPUs so fast at this? Because each dot product in a matrix multiplication can be computed completely independently of the others. A CPU has a few very fast cores (e.g., 16). A GPU has thousands of slower cores (the A100, for example, has 6,912, plus 432 Tensor Cores that each multiply a small matrix block per clock cycle). For matrices — where thousands of simple calculations happen simultaneously — the GPU is dramatically superior. NVIDIA's A100 datasheet gives the 312 TFLOPS for 16-bit numbers (FP16/BF16) on the Tensor Cores; the CPU figure is only an order of magnitude. The A100 came out in 2020; the newer NVIDIA B200 from the Blackwell generation reaches about 2,250 TFLOPS in the same number format.

import numpy as np

# Mini-network (without bias and activation): 5 data points, 3 features
X = np.random.randn(5, 3)       # Input: (5, 3)
W1 = np.random.randn(3, 4)      # Layer 1: (3, 4)
W2 = np.random.randn(4, 2)      # Layer 2: (4, 2)

hidden = X @ W1                  # (5, 3) @ (3, 4) = (5, 4)
output = hidden @ W2             # (5, 4) @ (4, 2) = (5, 2)
print(f'Input: {X.shape} → Hidden: {hidden.shape} → Output: {output.shape}')

The inner dimension disappears at each multiplication — like canceling fractions. (5×3) @ (3×4) = (5×4): the 3 cancels out. Then (5×4) @ (4×2) = (5×2): the 4 cancels out. Data flows through the network layer by layer, and dimensions change in a controlled way.

# Shape chain visualized:
# (batch × features_in) @ (features_in × features_out) = (batch × features_out)
#   5 × 3               @          3 × 4              =       5 × 4
#   5 × 4               @          4 × 2              =       5 × 2
# The inner dimension must always match!

Note

Key Takeaways

  • A matrix is a rectangular grid of numbers — it can represent a dataset (rows = examples) or a transformation (input → output).
  • Matrix multiplication consists of dot products between rows and columns. It is defined this way because it describes the composition of linear transformations — not an arbitrary convention.
  • Every fully connected neural network layer computes output = W × input + bias — one matrix multiplication. GPUs excel at this because every output element can be computed independently.

Check

Quiz: Matrices

Not completed

A dataset has 500 images, each described by 3 features. What shape does the corresponding matrix have?

Select one answer

Answer Key: 1) B · 2) B · 3) B · 4) B

Check

Checkpoint: Matrices

Can you explain this in your own words? Tick what you've got.

0 of 3 done