Seventeen thousand tokens a second.
This chip, the Taalas HC1, writes about 17,000 tokens a second for a single user. That's roughly a novel in under ten seconds. The best graphics processors, serving one person, manage a few hundred.
To see how, we start with a problem we met in episode 1.
The math is cheap. The trip is not.
In episode 1 we followed one number from memory to math. Reading 32 bits from DRAM costs about 640 pJ; multiplying them costs about 3.7 pJ. Fetching is well over a hundred times dearer than thinking.
Sixteen gigabytes per word.
A small model like Llama 3.1 8B is 8 billion numbers. At 16 bits each, that's 16 GB. Writing one token means multiplying through all of them, so all 16 GB have to stream past the math units, again, for every token.
The limit is the pipe, not the math.
For one conversation, speed ≈ memory bandwidth ÷ bytes per token. An H100 moves 3.35 TB/s, so its ceiling is about 210 tokens a second on this model. A B200 at 8 TB/s: about 500. Real services land below that.
One read, a hundred conversations.
Data centres hide the wall with batching: read the weights once, use them for many users at the same moment. Total throughput soars and cost per token falls. But each person still gets their words one trickle at a time.
Hundreds of chips, one model.
Groq keeps weights in fast on-chip SRAM instead of HBM. But its original chip held only about 230 MB, so a single model is spread across many chips wired together. Fast, at about 600 tokens a second on this model, but it takes a room.
A chip the size of a dinner plate.
Cerebras builds one chip from an entire silicon wafer, with vast on-chip memory, and reaches about 2,000 tokens a second on Llama 3.1 8B. Astonishing engineering. But the numbers still live in memory and still have to be fetched.
What if there were
no memory at all?
The numbers are the circuit.
Toronto's Taalas, founded by Ljubisa Bajic (a co-founder of Tenstorrent), took one model, Meta's Llama 3.1 8B, and built its weights into the chip's wiring as read-only circuitry, right beside the math. Nothing is fetched from outside.
Same box. A new cylinder for every tune.
In a music box the comb, spring and case never change. The tune lives on the pins of a turning cylinder. Swap the cylinder and you have a new song, without building a new box.
Only two layers hold the model.
A chip is built in layers: transistors at the bottom, a dozen-plus layers of metal wiring above. Taalas says only two metal layers change from one model to the next. Those two masks are the cylinder.
Times five is a shift and an add.
When a weight never changes, multiplying by it needs no general-purpose multiplier and no lookup. x × 5 = (x shifted left by 2) + x: two wires and an adder, laid down once. Bajic says one 4-bit parameter is stored and multiplied on a single transistor; the exact circuit isn't public.
Eight possible values.
To fit 8 billion weights on one die, most are stored in a custom 3-bit format (just eight values), with some at 6 bits. Taalas acknowledges "some quality loss". The chip has 53 billion transistors, with SRAM for the conversation's working memory and small fine-tuning add-ons.
About 50× a B200, for one user.
Tokens per second for one user on Llama 3.1 8B, as compared by Taalas (GPU figures from Nvidia and Taalas; others from Artificial Analysis). EE Times measured 15,000+ on the public demo.
Company claims also include 10× less power and 20× cheaper to build. How many users one chip serves at once isn't disclosed, and part of the gap comes from the 3-bit compression.
No towers. No plumbing.
Remember the chip from episode 1? An H100 draws up to 700 W through a stack of layers: package substrate, silicon interposer, towers of HBM. Its successors run hotter (a B200 up to about 1,000 W), and racks like Nvidia's GB200 NVL72, at roughly 120 kW, are liquid cooled by design.
HC1 has no HBM and no advanced packaging. Taalas quotes about 200–250 W per card and about 2.5 kW for a ten-card server, cooled by ordinary air. Fetch less, and there's less heat to carry away.
Frozen forever.
The base model can never change. Small LoRA adapters can tune it, but a new model means a new chip. Llama 3.1 was already 18 months old when HC1 launched. Compression costs quality, no benchmark scores have been published, and room for long conversations is tight.
Two months from weights to chips.
Because only two masks change, Taalas says a finished model becomes working cards in about two months, with the model-to-circuit step taking about a week. The bet: when a model is good enough, freeze it, and print the next one when it's better.
Freeze what's settled.
Bitcoin mining moved from GPUs to custom chips from 2013, once its algorithm stopped changing. 1980s gate arrays were customised only in their top metal layers. The pattern: flexible chips for what's new, fixed silicon for what's settled.
One chip reads. Another writes.
Nvidia licensed Groq's technology in a deal reported at about $20B and now pairs Groq LPUs with its GPU racks. On 6 August 2026 AMD agreed to buy Taalas, about $2.9B including contingent payments, per AMD's filing. The deal hadn't closed at the time of writing.
Stop treating it as software.
Taalas has promised a frontier-class model on a second chip, HC2, spread over a few dozen chips. As of November 2026 it hasn't been shown publicly. If it works, the fastest way to run a settled AI may be to carve it into silicon.