03 · homemade
Tokenization: How AI Reads Text
AI doesn't read words. It reads bricks of text called tokens.
8 slices · ~5m
Every tile we've ever rolled out: zero to hero tutorials.
AI doesn't read words. It reads bricks of text called tokens.
Words as coordinates. The trick behind every AI that feels smart.
From keyword matching to finding meaning. The database that understands context.
Digital neurons, layered deep. The engine behind every AI breakthrough.
The architectural secret behind GPT, Claude, and Gemini.
70 billion dials. All tuned to perfection.
Two phases. One brain. Here's how a model goes from clueless to expert.
Why brilliant AI can still go terribly wrong. And how we fix it.