Dimensionality is just how many numbers

01 · The object itself

A point described by two numbers

One number places a point on a line. Two numbers place it on a plane; three, in the space around you. Drag the slider past three and geometry gives up — we can't draw a 7-dimensional room — but the math never notices. A point in 768-dimensional space is simply a list of 768 numbers, and distances between such points are computed the same way as in 2-D.

02 · Why size matters

Meaning needs room

Models place inputs into this space so that similar things sit near each other. That only works if there's room to keep every kind of similarity at once. Below, seventeen words are arranged in two dimensions — animals near animals, fruit near fruit. Now squash them onto a single dimension and watch the neighborhoods collapse.

Hover any word to see its three nearest neighbors in the current space.

This is the whole argument in miniature: with one number per word, “cat” ends up next to “truck” because there's nowhere else to put it. Real models use hundreds or thousands of dimensions so that a word can be close to other words along many independent directions — animal-ness, size, sentiment, formality — all at the same time.

03 · Text

Text expands: a few characters become hundreds of numbers

Raw text is tiny but meaningless to arithmetic — the bytes c·a·t say nothing about whiskers. So the model trades size for meaning: each token is looked up in an embedding table and becomes a long vector. Dimensionality goes up — from a handful of characters to, say, 768 learned numbers.

04 · Images

Images compress: millions of pixels become a short summary

An image has the opposite problem. It is already all numbers — far too many, and most of them redundant (the sky is blue in every one of its thousands of sky pixels). An image encoder squeezes the picture down to a compact latent vector that keeps only what matters: dimensionality goes down, from millions to hundreds.

Notice the latent stays 512 numbers whether the photo is 32 × 32 or 1024 × 1024. The encoder isn't storing pixels; it's storing what the image is about — and that doesn't grow with resolution.

05 · The punchline

Both roads lead to the same neighborhood

Put both quotes on one axis and the pattern appears: text is pushed up into more dimensions, images are pulled down into fewer — and both land in the same band of hundreds to thousands of numbers. That's the size at which a vector is rich enough to hold meaning, yet small enough to do arithmetic on billions of times per second. Hidden states live here.