Spaces and Directions (Vectors)

What do a photo, a sentence, and a song have in common? Before any AI model can process them, they all become the same thing: a list of numbers. That list is called a vector. Vectors are the universal data format of artificial intelligence.

In this article, you will learn what vectors are, how to work with them (add, scale, measure), and why the dot product — a simple sum of multiplied components — is the operation that makes search engines find relevant results and neural networks recognize patterns.

Scalar vs. Vector — Single Value vs. List

Every data point becomes a vector: A house with Area=120, Rooms=4, Year=1990, Price=350,000 becomes [120, 4, 1990, 350000] — a 4D feature vector. 1,000 houses = 1,000 vectors = an ML dataset.

import numpy as np

# A house as a 4D feature vector
house = np.array([120, 4, 1990, 350000])
print(house.shape)  # (4,) — 4 dimensions

EVERY input to a neural network — text, image, audio — is first converted into a numerical vector. No vectors, no AI.

Vector Operations — Add, Scale, Measure

Three fundamental operations make vectors the workhorse of AI: addition, scalar multiplication, and the norm (length).

Vector Addition

Componentwise: Each position is added separately. Geometrically, this means chaining vectors end-to-end (parallelogram rule).

Scalar Multiplication

Each component is multiplied by the same scalar. This stretches (c > 1), shrinks (0 < c < 1), or reverses (c < 0) the vector.

Norm (Length)

The norm generalizes the Pythagorean theorem to any number of dimensions. For [3, 4]: sqrt(9 + 16) = sqrt(25) = 5.

import numpy as np
v1 = np.array([3, 4])
v2 = np.array([1, 2])

print(v1 + v2)              # [4, 6]  — Addition
print(2 * v1)               # [6, 8]  — Scalar multiplication
print(np.linalg.norm(v1))   # 5.0    — Norm
print(v1 / np.linalg.norm(v1))  # [0.6, 0.8] — Normalization

The Dot Product — Measuring Similarity

The sum of componentwise products. The result is a scalar — a single number.

Same direction → cos(0°) = 1 → maximum dot product. Perpendicular → cos(90°) = 0. Opposite → cos(180°) = -1.

Cosine Similarity

The normalized dot product yields a value between -1 and +1, independent of vector length. In word embeddings, cosine similarity is the standard metric for semantic relatedness.

import numpy as np

king  = np.array([0.9, 0.1, 0.8])
queen = np.array([0.9, 0.8, 0.1])
apple = np.array([0.1, 0.3, 0.05])

def cosine_sim(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

print(cosine_sim(king, queen))  # ~0.66
print(cosine_sim(king, apple))  # ~0.41

Interactive: How Operations Scale with Dimension

Move the slider to change the number of dimensions d — from a simple 2D coordinate to a 1000-dimensional embedding. Observe how differently the costs of various operations grow: vector addition and dot product scale linearly with d, but the number of data points needed for adequate coverage explodes exponentially.