CPU, GPU, TPU, NPU, DPU, QPU: six chips, one question

Hello, I'm Maneshwar, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product.

Open your laptop's spec sheet and you will find a CPU, a GPU, and if it is recent enough, an NPU.

Open a cloud instance page and you get offered GPUs, TPUs, and a networking card that turns out to be a whole computer wearing a trench coat.

Somewhere in a lab, a fridge colder than deep space is running a QPU.

Six acronyms, all ending in PU, all doing "computation."

The obvious question, and the one I kept dodging for years: why not just build one really good processor and stop?

The short answer is that "computation" is not one thing. The longer answer is the fun part.

The CPU is the friend who says yes to everything

CPU stands for central processing unit, and it is the generalist.

It runs your OS, your API handler, your regex, your if statements, your database, your build tool, and the Slack client that eats four gigs of RAM for no reason.

A CPU is built to be good at unpredictable work. Branchy code where the next instruction depends on the last result.

Pointer chasing. Long dependency chains where nothing can be done in parallel because step 5 genuinely needs the output of step 4.

To pull that off, a modern CPU core is stuffed with machinery that has nothing to do with math: branch predictors guessing which way your if goes, out-of-order execution reordering your instructions behind your back, and a cache hierarchy doing everything it can to hide the fact that RAM is embarrassingly far away.

All that cleverness costs transistors and power. So you get a handful of very smart cores, not thousands of them.

Which is great, right up until the workload changes shape.

Then someone hands you a million identical sums

Say you need to do the same operation across a few million values. Same instruction, different data, no branches, nobody waiting on anybody.

A CPU can do this. It will even do it in parallel across its cores and its SIMD units.

It just will not do it well, because you are paying for all that branch prediction hardware to predict a branch that does not exist.

Enter the GPU, or graphics processing unit.

GPUs came from rendering, and rendering is the purest form of this problem.

A single 4K frame is roughly 8.3 million pixels, and at 60fps you are shading all of them 60 times a second, with the same lighting math applied to each one.

So GPU designers made the opposite trade from CPU designers. Drop the fancy per-core intelligence, spend the transistors on arithmetic units instead, and run thousands of threads in lockstep.

The model has a name, SIMT, single instruction multiple threads.

The punchline nobody planned: neural networks turned out to be the same shape as graphics.

A forward pass through a transformer is mostly enormous matrix multiplications, which is millions of independent multiply-and-add operations. That is a GPU's native language.

Graphics cards accidentally became the substrate for the entire AI industry, which is possibly the most profitable accident in hardware history.

Here is the same silicon budget, spent two completely different ways. On the left, four cores that are mostly not arithmetic.

On the right, a field of arithmetic units that cannot think for themselves at all.

If the math is always the same, build a chip that only does that math

Once matrix multiplication became the dominant cost of an industry, the next move was inevitable.

That is the TPU, or tensor processing unit, Google's machine learning accelerator.

A GPU is still a general parallel computer. It has schedulers, register files, and a memory model built to handle whatever you throw at it.

A TPU narrows the target: build the silicon around tensor and matrix operations specifically, and design the data movement around how those operations actually flow.

Google's first TPU paper is worth a read if you like this stuff, because it is refreshingly blunt about the trade.

The chip is not clever.

It is a big systolic array of multiply-accumulate units with the control logic stripped out, and it wins on performance per watt precisely because it refuses to be flexible.

This is the whole thesis of specialised hardware in one line: the narrower the assumption, the bigger the payoff when the assumption holds.

And the bigger the faceplant when it does not.

Point a TPU at your JSON parser and watch nothing good happen.

That trade is worth drawing out, because it is the single idea that explains the whole zoo.

There is one rope, flexibility at one end and throughput per watt at the other, and every classical chip is standing somewhere along it:

But not all AI lives in a data center

Your phone unlocks with your face, blurs your background on a call, transcribes a voice note, and cleans up a photo you took in bad light.

None of that should require a round trip to a GPU cluster. It would be slow, it would burn battery on the radio, and it would mean shipping your face to somebody's server for the privilege.

That is the NPU, or neural processing unit: a small AI accelerator sitting on the device itself, next to the CPU and GPU on the same chip.

Apple calls theirs the Neural Engine and exposes it through Core ML, Qualcomm and Intel ship their own, and every "AI PC" sticker on a laptop is really a sticker about an NPU.

TPU and NPU are both AI accelerators, so the difference is worth stating plainly. It is not really about the math, it is about the environment they are built for:

• TPU: data center. Plugged into a wall. Trains and serves enormous models. Optimised for throughput per rack.
• NPU: your pocket. Running on a battery. Executes small models continuously. Optimised for operations per watt, and for not setting your jeans on fire.

Same equations. Wildly different constraints. That is enough to justify two different chips.

The chip whose job is not computing at all

Here is the part that surprised me most when I first dug into it.

Picture a cloud host running a few dozen VMs. Before a single line of your application code runs, the machine has to: terminate network packets, run a virtual switch, enforce security groups, encrypt traffic, present virtual disks that are actually remote storage, and handle every I/O interrupt in the building.

Traditionally the host CPU does all of that. Which means you buy a 64 core server and a meaningful slice of it never touches customer workloads.

The industry politely calls this the "datacenter tax."

The DPU, or data processing unit, moves that tax onto its own chip.

It is a card with its own CPU cores, its own NICs, and hardware engines for networking, storage, and crypto, and it takes over the infrastructure work so the host CPU can go back to running applications.

AWS Nitro is the famous example, and it is why AWS can hand you a bare metal instance with nearly all the host's cores intact.

NVIDIA BlueField is the merchant version of the same idea.

So a DPU is not further along the generality ladder.

It is standing on a completely different axis, because its workload is moving data rather than crunching it.

And then there is the one that is not even playing the same game

Every processor so far is classical. CPU, GPU, TPU, NPU, DPU, all of them push bits around, and a bit is a 0 or a 1.

A QPU, or quantum processing unit, uses qubits.

A qubit can sit in superposition, qubits can be entangled with each other, and quantum algorithms exploit those properties to explore a problem space in a way classical algorithms structurally cannot.

Two things people constantly get wrong here, so let me be annoying about them:

A QPU is not a fast CPU. It is a different computational model.
There is no world where you port your web server to it and get a speedup.
Most problems get no quantum advantage at all.

"Exponentially faster at everything" is marketing.
The real list of promising areas is narrow and specific: simulating quantum systems (chemistry, materials), certain optimisation problems, and cryptography, where Shor's algorithm is the reason people care about post-quantum crypto today rather than in 2040.
If you want to poke at real hardware instead of vibes, IBM Quantum will let you run a circuit for free.

Today's machines are noisy and small, and error correction is the wall everyone is working on. It is genuinely exciting research. It is not a data center upgrade.

So how do you actually pick one?

Do not start from the acronym. Start from the shape of the work.

Read it as a set of questions rather than a hierarchy:

• Is the work branchy and varied? CPU. This is most software, and that is fine.
• Is it the same operation across a mountain of data? GPU.
• Is that operation specifically tensor math, at data center scale? TPU.
• Is it tensor math on a battery? NPU.
• Is the bottleneck moving data, not computing on it? DPU.
• Is it a problem where a quantum algorithm has a known advantage? QPU, eventually, probably not today.

And here is the thing worth internalising: this is not a contest. These chips are colleagues, not competitors.

A single request from your phone to an AI feature can touch four of them. The phone's NPU decides whether it can handle the request locally.

If not, the CPU builds the request, a DPU in the data center terminates the connection and does the storage and crypto work, a server CPU runs your application logic, and a GPU or TPU does the actual inference.

The part that bites you in real code

Knowing the taxonomy is nice.

The practical trap is different: an accelerator only helps the part of the work it is actually doing.

Amdahl's law is unromantic about this. If 20% of your runtime is not accelerated, making the other 80% infinitely fast still caps you at 5x.

In practice the unaccelerated 20% is data loading, preprocessing, and copying tensors across a PCIe bus.

This is why so many "we bought GPUs and got a 1.3x speedup" stories exist. The GPU was fine. The GPU was starving.

You can watch it happen with about ten lines:

import time, torch

x = torch.randn(8192, 8192)

t = time.perf_counter()
x @ x
print(f"cpu:            {time.perf_counter() - t:.3f}s")

g = x.cuda()
torch.cuda.synchronize()

t = time.perf_counter()
g @ g
torch.cuda.synchronize()          # kernels are async, so measure honestly
print(f"gpu (resident): {time.perf_counter() - t:.3f}s")

t = time.perf_counter()
(x.cuda() @ x.cuda()).cpu()       # the version people accidentally write
torch.cuda.synchronize()
print(f"gpu (+copies):  {time.perf_counter() - t:.3f}s")

The middle number is the one in the marketing material.

The bottom number is the one you get if you move data across the bus every call.

On a big matmul the transfer can cost more than the math, and the fix is always the same: keep the data resident on the device and batch your work so the trip is worth it.

Same lesson shows up on an NPU, where the fastest path is the one where a tensor never leaves the shared memory the accelerator can see, and on a DPU, where the entire point is that packets stop bouncing through host memory.

If you want to know what your own machine is carrying:

lscpu | grep -E 'Model name|^CPU\(s\)|Flags' | cut -c1-120   # cores + SIMD support
nvidia-smi --query-gpu=name,memory.total --format=csv        # discrete GPU
ls /dev/accel* /dev/dri/render* 2>/dev/null                  # accelerators + render nodes

On a modern laptop that last line is quietly interesting, because there is usually more silicon in there than you expected.

The mental model, in one block

If you keep one thing from all of this, keep the question rather than the acronyms.

Here is the version I would paste into my own notes:

flowchart TD
    A[What shape is this work?] --> B{Branchy, varied, sequential?}
    B -->|Yes| CPU[CPU]
    B -->|No| C{Same op over huge data?}
    C -->|No| D{Bottleneck is moving data?}
    D -->|Yes| DPU[DPU]
    D -->|No| Q{Known quantum advantage?}
    Q -->|Yes| QPU[QPU, eventually]
    Q -->|No| CPU
    C -->|Yes| E{Neural network math?}
    E -->|No| GPU[GPU]
    E -->|Yes| F{Data center or device?}
    F -->|Data center| TPU[TPU]
    F -->|Device on battery| NPU[NPU]

We did not end up with six processors because computing got needlessly complicated.

We ended up with six because "make it general" and "make it fast" pull in opposite directions, and different problems sit at different points on that rope.

A CPU says yes to everything and is therefore mediocre at all of it.

A TPU says yes to almost nothing and is spectacular at the little it accepts.

Everything else is somewhere in between, doing its own particular job.

Your team's attention is limited, and the deluge of AI-generated code is making it harder to keep production code safe without slowing you down.