Here's the question that keeps coming up in r/LocalLLaMA: can a 27B model feel fast on a single consumer GPU? Two fresh benchmark drops answer it with actual numbers instead of vibes. One covers a single AMD Radeon R9700 running Unsloth's Qwen3.8 27B NVFP4 build through a tuned vLLM fork. The other is the first public M5 Ultra data from omlx.ai, same model in q4.

Both reports separate the three numbers that matter: prefill (time to first token), decode (streaming speed), and concurrency (what happens when more than one request hits the card). Most benchmark posts only give you one of those. That single number is almost always misleading.

The short version: the R9700 post claims decode performance doubled after kernel optimization, hitting 153 tok/s peak on structured generation, roughly 3,600 tok/s prefill, and about 470 tok/s aggregate with eight concurrent requests. The M5 Ultra lands at 50 tok/s decode and 1,800 tok/s prefill at 8K context, with multi-token prediction disabled. Both are usable. They're usable in different ways.

Two very different single-GPU stories

Metric	AMD R9700	Apple M5 Ultra
Setup	1x Radeon R9700, Linux, tuned vLLM fork	Mac, MLX/omlx stack
Quantization	Unsloth NVFP4 (4-bit MXFP4 format)	q4
Peak decode	153.1 tok/s (json workload)	~50 tok/s
Prefill, 8K context	~3,536 tok/s	~1,800 tok/s
Multi-request decode	~471 tok/s at 8 concurrent	not published
MTP	not reported	disabled for this run
Tooling	vllm-mxfp4 "radiance" branch, BetterBench	omlx / MLX

The NVFP4 format is the premise for all of this. It squeezes the 27B model down to roughly 14 GB of weights, which is the only reason a single consumer card can attempt it at all. On the AMD side, that quantized model goes through a custom vLLM fork (vllm-mxfp4, radiance branch) that Bruce Kuyper and collaborators tuned specifically for RDNA4-class cards. The author says the fork roughly doubled performance versus stock builds. The M5 Ultra path is the opposite: no fork, no kernel flags, just the omlx/MLX stack doing its thing. Keep that difference in effort in mind when you compare the two.

Decode speeds that actually tell you something

Workload	Decode (tok/s)	Median update interval (ms)
chat	67.1	42.3
code	120.5	42.5
file_edit	138.0	42.5
json	153.1	42.4
math	140.0	42.5
prose	69.2	42.3
reasoning	123.9	34.3
summarization	141.7	34.2

The spread between categories is normal and you shouldn't treat it as variance. Chat and prose run at 67-69 tok/s; structured workloads (json, file_edit, summarization, math) run at 138-153 tok/s. The reason is mostly output composition. Long free-form generation exercises the sampler and the KV cache differently than short structured outputs, so the median decode rate lands lower.

What a 42 ms median update interval means in practice: your client receives a fresh chunk of text roughly 24 times per second. That's below the threshold a human perceives as lag. And 153 tok/s means a 1,000-token JSON payload lands in about 6.5 seconds. Even the "slow" chat number, 67 tok/s, is roughly 3,000 words per minute of output. The card isn't the bottleneck for any interactive use case.

So when you read "153 tok/s on an R9700," read it as "best-case structured generation." For chat, expect half that. Both are fast enough that the difference only matters for benchmark bragging rights.

First impressions live in prefill, not decode

Prefill is where the user actually waits. An 8,000-token prompt on the R9700 produces its first token in about 2.3 seconds. The same prompt on the M5 Ultra takes about 4.4 seconds. At the extreme, a 64K prompt on the R9700 still costs roughly 20 seconds to first token. Everything feels instant until you load a big context, then prefill is the whole experience.

The prefill numbers stay remarkably flat on the R9700: 3,552 tok/s at 2K depth, 3,619 at 16K, 3,437 at 32K, and 3,192 at 64K. The card only starts to fade when the context gets truly large. Prefill is parallel and memory-bandwidth-friendly; decode is serial and bandwidth-bound. That's why prefill runs about 20x faster than decode, and it's why a benchmark post that only shows decode is hiding the part users actually wait on.

Quick Take: If a local model is going to feel slow, it's the prefill that makes it feel slow; both of these setups clear the "first token in under 5 seconds" bar at 8K context, and that's what makes them usable day to day.

Concurrency: one card, many users

Single-stream decode on this benchmark suite measured 120 tok/s. Add a second request and it jumps to 215 tok/s because decode is bandwidth-bound and the second stream fills lanes that would otherwise sit idle. At 8 concurrent requests you get 471 tok/s aggregate, roughly 59 tok/s per user. That's enough for a small team to share one card for code completion, chat, and summarization without anyone noticing a slowdown.

Don't extrapolate this linearly. The curve is already flattening by 8 requests, and the card runs out of memory bandwidth long before it runs out of queue slots. Plan a shared-card deployment around 50-60 tok/s per seat, not 470 divided by however many users you hope to support.

Key numbers from the two posts: 153.1 tok/s peak decode on json, 3,619 tok/s prefill at 16K depth, 471 tok/s aggregate at 8 concurrent requests, 42 ms median streaming interval, 50 tok/s decode on the M5 Ultra, 1,800 tok/s prefill on the M5 Ultra at 8K context.

NVFP4 and the kernels behind the numbers

I kept seeing people ask about single R9700 cards in the comments and on Discord. The usual answer was "usable, but underwhelming." Then the radiance branch of vllm-mxfp4 came out, and performance at least doubled across the board. The post that reported these numbers also updated both repos because users asked for GitHub copies of the code, and built BetterBench to make the results visual instead of a wall of text.

Comments on the post split the way you'd expect. AMD owners finally had evidence their card wasn't a mistake. NVIDIA owners argued the comparison wasn't apples-to-apples because the result leans on the MXFP4 kernel path. A few people ran the same weights through upstream vLLM and got much lower numbers, which tells you exactly how much of this result is the fork and not the hardware. When I tested the stock path against the fork, the gap was the difference between "fine for chat" and "fast enough to be a default daily driver." On this hardware, the software stack is the performance.

Apple Silicon M5 Ultra: the other contender

The M5 Ultra numbers come from omlx.ai, and the poster flags them as early and unverified. Treat them as directional. The reported figures: 50 tok/s decode throughput, 1,800 tok/s prefill at 8K context, with multi-token prediction disabled.

Break that down. 50 tok/s streams at roughly 2,250 words per minute, slower than the R9700 but faster than any human can read. An 8,000-token prompt delivers its first token in about 4.4 seconds. For interactive chat, agent loops, and daily code assistance, that's comfortable. For batch processing or high-volume structured generation, it's not in the R9700's league.

The MTP flag is the detail to watch. These chips support multi-token prediction; the number was measured with it off. If the MLX stack enables MTP, decode figures move up meaningfully because the model emits several tokens per step. The R9700 post doesn't mention MTP at all, so the comparison sits on different footing until both sides state their settings.

The other angle is the one that doesn't show up in a table. The M5 Ultra is a quiet, low-maintenance box. No custom fork, no kernel tuning, no fan curve. If your time is worth more than a 3x decode advantage, 50 tok/s is a perfectly good place to be.

Common Pitfalls

A few mistakes show up in every thread about these numbers.

• Quoting peak decode as the card's speed. 153 tok/s is the json category. Chat runs at 67. If you size a product decision around the peak, the real experience will feel half as fast as advertised.
• Running the same weights through upstream vLLM and expecting these results. The 2x gain came from the radiance fork and its MXFP4 kernels. Unsloth's NVFP4 quant is only half the story.
• Comparing prefill numbers at different context depths. 3,619 tok/s at 16K becomes 3,192 at 64K. A 64K prompt still costs about 20 seconds to first token. Benchmark at the depth you actually plan to use.
• Adding users linearly. 8 concurrent requests yields 471 tok/s, or about 59 per seat. Doubling the requests won't double the throughput; the card saturates on memory bandwidth first.
• Ignoring the MTP setting. The M5 figures were captured with multi-token prediction off. Comparing them against a run with MTP enabled inflates the gap, in either direction.

One thing to remember

Qwen3.8 27B is daily-driver class on a single consumer card right now, but only because of two pieces of software. NVFP4 quantization brings the weights down to roughly 14 GB, and the radiance fork doubles decode throughput. Continuous batching then turns one card into a four-to-eight seat server. Without those, the same hardware tells a much more boring story, and any purchase decision you make from this benchmark should include the cost of running that stack.

The Bottom Line

1. If you're buying a single GPU for local coding work and want the fastest Qwen3.8 27B experience today, go with an R9700 and plan on maintaining the radiance vLLM fork with NVFP4 weights; that's where the 153 tok/s decode and 470 tok/s concurrency numbers actually come from.
2. If you're on a Mac or want a quiet, low-maintenance box, skip the kernel tinkering and use the M5 Ultra; 50 tok/s decode and 1,800 tok/s prefill are enough for interactive chat and agent work, and you'll spend your hours on the product instead of the stack.
3. Watch the MXFP4 kernel path over the next few months; the tuning that doubled the R9700's decode is model-agnostic, so expect it to land for other 27B and 30B models and possibly in upstream vLLM, which would change which card is the obvious buy.