Spaces and Directions (Vectors)

What do a photo, a sentence, and a song have in common? Before any AI model can process them, they all become the same thing: a list of numbers. That list is called a vector. Vectors are the universal data format of artificial intelligence.

In this article, you will learn what vectors are, how to work with them (add, scale, measure), and why the dot product — a simple sum of multiplied components — is the operation that makes search engines find relevant results and neural networks recognize patterns.

Scalar vs. Vector — Single Value vs. List

Every data point becomes a vector: A house with Area=120, Rooms=4, Year=1990, Price=350,000 becomes [120, 4, 1990, 350000] — a 4D feature vector. 1,000 houses = 1,000 vectors = an ML dataset.

import numpy as np

# A house as a 4D feature vector
house = np.array([120, 4, 1990, 350000])
print(house.shape)  # (4,) — 4 dimensions

EVERY input to a neural network — text, image, audio — is first converted into a numerical vector. No vectors, no AI.

Vector Operations — Add, Scale, Measure

Three fundamental operations make vectors the workhorse of AI: addition, scalar multiplication, and the norm (length).

Vector Addition

Componentwise: Each position is added separately. Geometrically, this means chaining vectors end-to-end (parallelogram rule).

Scalar Multiplication

Each component is multiplied by the same scalar. This stretches (c > 1), shrinks (0 < c < 1), or reverses (c < 0) the vector.

Norm (Length)

The norm generalizes the Pythagorean theorem to any number of dimensions. For [3, 4]: √(9 + 16) = √25 = 5.

import numpy as np
v1 = np.array([3, 4])
v2 = np.array([1, 2])

print(v1 + v2)              # [4 6]  — Addition
print(2 * v1)               # [6 8]  — Scalar multiplication
print(np.linalg.norm(v1))   # 5.0    — Norm
print(v1 / np.linalg.norm(v1))  # [0.6 0.8] — Normalization

The Dot Product — Measuring Similarity

The sum of componentwise products. The result is a scalar — a single number.

Same direction → cos(0°) = 1 → the largest dot product for given lengths. Perpendicular → cos(90°) = 0. Opposite → cos(180°) = −1.

Cosine Similarity

The normalized dot product yields a value between -1 and +1, independent of vector length. In word embeddings, cosine similarity is the standard metric for semantic relatedness. An embedding from the BERT-base language model (Devlin et al. 2018), for example, has 768 dimensions.

import numpy as np

king  = np.array([0.9, 0.1, 0.8])
queen = np.array([0.9, 0.8, 0.1])
apple = np.array([0.1, 0.3, 0.05])

def cosine_sim(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

print(cosine_sim(king, queen))  # ~0.66
print(cosine_sim(king, apple))  # ~0.41

Interactive: Add, Scale and Compare Vectors

The two arrows start at v1 and v2 from the code example above, called a and b here. Set their components with the sliders and pick an operation: addition chains b onto a, the scalar c stretches, shrinks or reverses a, and the dot product uses the angle θ to show how similar the two directions are.