Back to Learning Area

Spaces and Directions (Vectors)

Why AI constantly works with arrows in n-dimensional space.

  • Fundamentals
  • 11 min
  • Beginner

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Vectors: lists of numbers

What is a vector?

A vector is an ordered list of numbers. Every AI input becomes a vector. For example, a house becomes numbers: area, rooms, year of construction. This is how the computer can calculate with it.

Calculating with vectors

You can calculate with vectors. You can add them. You can stretch or compress them. The length of a vector is called the norm. The norm measures the distance from the origin.

Measuring similarity

Vectors can be compared to each other. This is called a dot product. Dot product means: multiplying two lists and adding the results. This is how a search finds similar texts.

What do a photo, a sentence, and a song have in common? Before any AI model can process them, they all become the same thing: a list of numbers. That list is called a vector. Vectors are the universal data format of artificial intelligence.

In this article, you will learn what vectors are, how to work with them (add, scale, measure), and why the dot product — a simple sum of multiplied components — is the operation that makes search engines find relevant results and neural networks recognize patterns.

Scalar vs. Vector — Single Value vs. List

Definition

Scalar and Vector

Analogy

GPS coordinates: A single latitude value (52.52) is a scalar — it only pins you to an east-west line (the parallel of latitude). Add longitude [52.52, 13.41] and you have a 2D vector that locates Berlin. Add altitude [52.52, 13.41, 34] — a 3D vector. AI embeddings work identically, just with hundreds or thousands of dimensions.

Example

GPS has 2-3 dimensions with clear physical meaning. AI embedding dimensions are emergent — learned by optimization — and not directly interpretable.
  1. 1. Latitude: 52.52 — a scalar. Pins only a line.
  2. 2. + Longitude: [52.52, 13.41] — a 2D vector. Locates Berlin.
  3. 3. + Altitude: [52.52, 13.41, 34] — a 3D vector.
  4. 4. Embedding: 768 dimensions — the math stays identical.

Every data point becomes a vector: A house with Area=120, Rooms=4, Year=1990, Price=350,000 becomes [120, 4, 1990, 350000] — a 4D vector (if a model is to predict the price, the first three numbers are the features and the price is the target). 1,000 houses = 1,000 vectors = an ML dataset.

import numpy as np

# A house as a 4D vector
house = np.array([120, 4, 1990, 350000])
print(house.shape)  # (4,) — 4 components: a 4D vector, to NumPy a 1D array

Caution

Misconception: A Vector Is an Arrow

In 2D/3D you can draw a vector as an arrow — useful for getting started. But in higher dimensions (e.g. 768D for a sentence embedding), you can no longer draw arrows. A vector is an abstract ordered list of numbers. The arrow intuition is a starting point, not the full picture.

EVERY input to a neural network — text, image, audio — is first converted into a numerical vector. No vectors, no AI.

Vector Operations — Add, Scale, Measure

Three fundamental operations make vectors the workhorse of AI: addition, scalar multiplication, and the norm (length).

Vector Addition

[a₁, a₂] + [b₁, b₂] = [a₁+b₁, a₂+b₂]
Vector Addition

Componentwise: Each position is added separately. Geometrically, this means chaining vectors end-to-end (parallelogram rule).

Scalar Multiplication

c · [a₁, a₂] = [c·a₁, c·a₂]
Scalar Multiplication

Each component is multiplied by the same scalar. This stretches (c > 1), shrinks (0 < c < 1), or reverses (c < 0) the vector.

Norm (Length)

‖v‖ = √(v₁² + v₂² + … + vₙ²)
Euclidean Norm

The norm generalizes the Pythagorean theorem to any number of dimensions. For [3, 4]: √(9 + 16) = √25 = 5.

import numpy as np
v1 = np.array([3, 4])
v2 = np.array([1, 2])

print(v1 + v2)              # [4 6]  — Addition
print(2 * v1)               # [6 8]  — Scalar multiplication
print(np.linalg.norm(v1))   # 5.0    — Norm
print(v1 / np.linalg.norm(v1))  # [0.6 0.8] — Normalization

Note

Normalization: Only Direction Matters

Dividing a vector by its norm produces a unit vector (length 1). Direction is preserved, scale is removed. In embedding pipelines this is essential: Two documents about the same topic should be similar regardless of their length. Not to be confused with the glossary entry: there, normalization means bringing feature scales into line (e.g. min-max); here it is about the length of a single vector.

The Dot Product — Measuring Similarity

a · b = a₁b₁ + a₂b₂ + … + aₙbₙ
Dot Product

The sum of componentwise products. The result is a scalar — a single number.

a · b = ‖a‖ · ‖b‖ · cos(θ)
Geometric Interpretation

Same direction → cos(0°) = 1 → the largest dot product for given lengths. Perpendicular → cos(90°) = 0. Opposite → cos(180°) = −1.

Dot Product

Measures directional similarity, but is also influenced by vector length.

Euclidean Distance

Measures how far apart two points are in space (distance).

Cosine Similarity

cos(θ) = (a · b) / (‖a‖ · ‖b‖)
Cosine Similarity

The normalized dot product yields a value between -1 and +1, independent of vector length. In word embeddings, cosine similarity is the standard metric for semantic relatedness. An embedding from the BERT-base language model (Devlin et al. 2018), for example, has 768 dimensions.

King / Queen

0.66

King / Apple

0.41

Dimensions (BERT-base)

768

import numpy as np

king  = np.array([0.9, 0.1, 0.8])
queen = np.array([0.9, 0.8, 0.1])
apple = np.array([0.1, 0.3, 0.05])

def cosine_sim(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

print(cosine_sim(king, queen))  # ~0.66
print(cosine_sim(king, apple))  # ~0.41

Caution

Misconception: The Dot Product Measures Distance

The raw dot product is influenced by vector length — double a vector, and its dot product with any other vector doubles too. It measures a combination of direction and magnitude. Cosine similarity removes the length factor and compares pure direction. Distance is measured by Euclidean distance ||a - b||.

z = w · x + b
Neuron Computation

Every neuron in a neural network computes exactly one dot product: weight vector w times input vector x plus bias b. The perceptron demo on this portal shows this operation in action.

Interactive: Add, Scale and Compare Vectors

The two arrows start at v1 and v2 from the code example above, called a and b here. Set their components with the sliders and pick an operation: addition chains b onto a, the scalar c stretches, shrinks or reverses a, and the dot product uses the angle θ to show how similar the two directions are.

"More dimensions = better" is a common misconception. In very high dimensions, for many data distributions (such as randomly scattered points), the pairwise distances even out — everything becomes almost equally far from everything else.

Distance-based algorithms (k-NN, clustering) then lose their effectiveness. This is why dimensionality reduction techniques like PCA and, mainly for visualization, t-SNE exist: They project high-dimensional data into a lower-dimensional space that preserves the most important structures. That does not hold for all data, though: real embeddings are not randomly scattered but clustered by meaning. In such data the distances stay meaningful, which is why similarity search with 768 dimensions still works.

Note

Key Takeaways

  • A scalar is one number; a vector is an ordered list of numbers. Every AI input — text, image, audio — gets converted into a vector.
  • Vectors can be added, scaled, and measured. The norm gives length; normalization preserves only direction.
  • The dot product measures how aligned two vectors are, weighted by their length. Cosine similarity normalizes this to [-1, +1] and is the standard metric for embeddings. Every neuron computes a dot product.

Check

Quiz: Vectors

Not completed

What is the difference between a scalar and a vector?

Select one answer

Answer Key: 1) B · 2) C · 3) B · 4) B

Check

Learning Goals

Can you explain this in your own words? Tick what you've got.

0 of 3 done