Spaces and Directions (Vectors)
Why AI constantly works with arrows in n-dimensional space.
What do a photo, a sentence, and a song have in common? Before any AI model can process them, they all become the same thing: a list of numbers. That list is called a vector. Vectors are the universal data format of artificial intelligence.
In this article, you will learn what vectors are, how to work with them (add, scale, measure), and why the dot product — a simple sum of multiplied components — is the operation that makes search engines find relevant results and neural networks recognize patterns.
Scalar vs. Vector — Single Value vs. List
Definition
Scalar and Vector
Analogy
Example
Definition
- 1. Latitude: 52.52 — a scalar. Pins only a line.
- 2. + Longitude: [52.52, 13.41] — a 2D vector. Locates Berlin.
- 3. + Altitude: [52.52, 13.41, 34] — a 3D vector.
- 4. Embedding: 768 dimensions — the math stays identical.
Every data point becomes a vector: A house with Area=120, Rooms=4, Year=1990, Price=350,000 becomes [120, 4, 1990, 350000] — a 4D vector (if a model is to predict the price, the first three numbers are the features and the price is the target). 1,000 houses = 1,000 vectors = an ML dataset.
import numpy as np
# A house as a 4D vector
house = np.array([120, 4, 1990, 350000])
print(house.shape) # (4,) — 4 components: a 4D vector, to NumPy a 1D arrayCaution
Misconception: A Vector Is an Arrow
In 2D/3D you can draw a vector as an arrow — useful for getting started. But in higher dimensions (e.g. 768D for a sentence embedding), you can no longer draw arrows. A vector is an abstract ordered list of numbers. The arrow intuition is a starting point, not the full picture.
EVERY input to a neural network — text, image, audio — is first converted into a numerical vector. No vectors, no AI.
Vector Operations — Add, Scale, Measure
Three fundamental operations make vectors the workhorse of AI: addition, scalar multiplication, and the norm (length).
Vector Addition
Componentwise: Each position is added separately. Geometrically, this means chaining vectors end-to-end (parallelogram rule).
Scalar Multiplication
Each component is multiplied by the same scalar. This stretches (c > 1), shrinks (0 < c < 1), or reverses (c < 0) the vector.
Norm (Length)
The norm generalizes the Pythagorean theorem to any number of dimensions. For [3, 4]: √(9 + 16) = √25 = 5.
import numpy as np
v1 = np.array([3, 4])
v2 = np.array([1, 2])
print(v1 + v2) # [4 6] — Addition
print(2 * v1) # [6 8] — Scalar multiplication
print(np.linalg.norm(v1)) # 5.0 — Norm
print(v1 / np.linalg.norm(v1)) # [0.6 0.8] — NormalizationNote
Normalization: Only Direction Matters
Dividing a vector by its norm produces a unit vector (length 1). Direction is preserved, scale is removed. In embedding pipelines this is essential: Two documents about the same topic should be similar regardless of their length. Not to be confused with the glossary entry: there, normalization means bringing feature scales into line (e.g. min-max); here it is about the length of a single vector.
The Dot Product — Measuring Similarity
The sum of componentwise products. The result is a scalar — a single number.
Same direction → cos(0°) = 1 → the largest dot product for given lengths. Perpendicular → cos(90°) = 0. Opposite → cos(180°) = −1.
Measures directional similarity, but is also influenced by vector length.
Measures how far apart two points are in space (distance).
Cosine Similarity
The normalized dot product yields a value between -1 and +1, independent of vector length. In word embeddings, cosine similarity is the standard metric for semantic relatedness. An embedding from the BERT-base language model (Devlin et al. 2018), for example, has 768 dimensions.
King / Queen
0.66
King / Apple
0.41
Dimensions (BERT-base)
768
import numpy as np
king = np.array([0.9, 0.1, 0.8])
queen = np.array([0.9, 0.8, 0.1])
apple = np.array([0.1, 0.3, 0.05])
def cosine_sim(a, b):
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
print(cosine_sim(king, queen)) # ~0.66
print(cosine_sim(king, apple)) # ~0.41Caution
Misconception: The Dot Product Measures Distance
The raw dot product is influenced by vector length — double a vector, and its dot product with any other vector doubles too. It measures a combination of direction and magnitude. Cosine similarity removes the length factor and compares pure direction. Distance is measured by Euclidean distance ||a - b||.
Every neuron in a neural network computes exactly one dot product: weight vector w times input vector x plus bias b. The perceptron demo on this portal shows this operation in action.
Interactive: Add, Scale and Compare Vectors
The two arrows start at v1 and v2 from the code example above, called a and b here. Set their components with the sliders and pick an operation: addition chains b onto a, the scalar c stretches, shrinks or reverses a, and the dot product uses the angle θ to show how similar the two directions are.
"More dimensions = better" is a common misconception. In very high dimensions, for many data distributions (such as randomly scattered points), the pairwise distances even out — everything becomes almost equally far from everything else.
Distance-based algorithms (k-NN, clustering) then lose their effectiveness. This is why dimensionality reduction techniques like PCA and, mainly for visualization, t-SNE exist: They project high-dimensional data into a lower-dimensional space that preserves the most important structures. That does not hold for all data, though: real embeddings are not randomly scattered but clustered by meaning. In such data the distances stay meaningful, which is why similarity search with 768 dimensions still works.
Note
Key Takeaways
- A scalar is one number; a vector is an ordered list of numbers. Every AI input — text, image, audio — gets converted into a vector.
- Vectors can be added, scaled, and measured. The norm gives length; normalization preserves only direction.
- The dot product measures how aligned two vectors are, weighted by their length. Cosine similarity normalizes this to [-1, +1] and is the standard metric for embeddings. Every neuron computes a dot product.
Check
Quiz: Vectors
What is the difference between a scalar and a vector?
Select one answer
1. What is the difference between a scalar and a vector?
2. v = [3, 4]. What is the length (norm) of this vector?
3. Two vectors have a cosine similarity of 0.95. What does this mean?
4. Why do we normalize vectors before comparing them with the dot product?
Check
Learning Goals
Can you explain this in your own words? Tick what you've got.
0 of 3 done