Fundamentals|Kartik Bansal, Founder & CEO|May 17, 2025|5 min read

Understanding Tokens in AI: A Beginner’s Guide

Learn how tokens work in AI models. This guide explains tokenization in text, image, and audio processing, and why tokens are key to how AI understands data.

Tokens in AI

Language models don’t read words — they read tokens. A token is the small unit a model actually processes, and almost everything practical about working with AI (cost, context limits, speed) is measured in them. Understanding tokens is the fastest way to stop being surprised by an AI system.

What a token is

Tokenization splits text into pieces — sometimes a whole word, often a fragment. “Tokenization” might become token + ization. A rough rule of thumb for English: one token is about four characters, and 100 tokens is roughly 75 words.

text
"Knowledge agents act."
-> ["Know", "ledge", " agents", " act", "."]
-> 5 tokens

Tokens beyond text

  • Images — split into patches, each turned into a token a vision model can read.
  • Audio — sampled into discrete units representing slices of sound.
  • Code — tokenized much like text, with structure-aware splitting.

Why tokens matter in practice

  • Context window — a model’s memory is a token budget; overflow it and the oldest content falls out.
  • Cost — you pay per token, in and out, so prompt design is cost design.
  • Latency — more tokens to generate means more time to wait.

“You can’t reason about an AI system’s cost, speed or limits until you think in tokens. It’s the unit the whole field is priced in.”

KnackLabs
Kartik Bansal
About the author
Kartik Bansal
Founder & CEO, KnackLabs

Kartik Bansal is the Founder and CEO of KnackLabs, an AI transformation firm that embeds senior engineers inside businesses to build and run production AI. Before KnackLabs he was a senior engineer at Microsoft and co-founder and CTO of Convosight.

←All field notes
ShareLinkedInX
Stay in the loop

New field notes, when we ship something worth writing about.

No cadence, no filler. Just the engineering and case studies as they go live.