DeepSeek V4 Flash on AMD Strix Halo: What Actually Makes It Fast

A 284B-parameter DeepSeek V4 Flash runs locally on a Ryzen AI MAX+ 395 at 303 tok/s prefill and 42 tok/s aggregate decode across four concurrent slots. Getting there took a seven-arm benchmark matrix, and the results were not what the flag documentation suggested. The build mattered roughly three times more than any flag. One ubatch setting was worth +36%. And two heavily-advertised optimisations turned out to be unavailable on this stack entirely.

Everything below is measured on one machine, with one variable changed at a time. Where a result contradicts a published claim, I have said so and shown the number.

The test rig

The single biggest win is the build, not the flags

Same model, same box, same flags. Only the llama.cpp binary differs.

The mechanism is v0.7.0’s Lightning Indexer rework, which targets this model’s DSA indexer.* tensors: 3.1x on the small-batch verify path, 1.5x on the prefill indexer op, and 12–18% on the batch-1 decode indexer that every DeepSeek V4 token uses. Because the indexer’s share of decode grows with depth, deep agent contexts benefit most.

If you take one thing from this post: before tuning a single flag, check whether someone has built a fork for your architecture. Three hours of flag sweeps bought less than swapping the binary.

The flag matrix

-ub 2048 is the one flag that matters: +36% prefill over the 512 default. Note this contradicts the fork’s own recommendation of -ub 1024 for this model — 2048 measured faster here, and it cost about 1 GiB of per-batch memory. Decode is unaffected either way, because single-token graphs do not care about batch size.

Prompt caching dwarfs every flag

A repeated 20k prompt restores in 0.12–0.14 seconds against 63–90 seconds cold. That is a ~500x saving, and it was invariant across every arm tested (20,012 of 20,016 tokens cached).

For agent workloads that re-send a near-identical system prompt every turn, this is the whole ballgame. No flag in the matrix moves the needle remotely as much. It also means the metric you should watch in production is not tokens per second — it is your cache hit rate.

Depth costs prefill time, not decode rate

Prefill likewise holds 252–314 t/s from 20k to 60k. Decode degrades only 12% across a 12,000x increase in context. The real cost of deep context is the cold prefill: a 60k prompt costs 163–231 seconds before the first token appears.

Four negative results, all measured

These are the findings I would have wanted before starting, and none of them are in the documentation.

1. Expert top-k is not a runtime knob. Both --override-kv deepseek4.expert_used_count=int:4 and :8 fail at load:

check_tensor_dims: tensor 'blk.0.ffn_gate_tid2eid.weight' has wrong shape;
expected 4, 129280, got 6, 129280

The hash-routing table is baked at 6 experts. Any published “top-k 4” figures for this model must come from a differently-built quant or a different engine.

2. Both DeepSeek V4 drafters are unusable here. The DSpark file reports unknown model architecture: 'deepseek4-dspark'; the MTP file’s arch is deepseek4_mtp_support, also absent. The binary does contain DSV4 DSpark code, but it expects dflash.block_size metadata while the published files use dspark.block_size — they were built for a different server. So there is no speculative decoding for DS4 on this stack, and 9.8 GB of drafter downloads sit idle. I predicted this would work; it did not.

3. q4_0 KV is nearly free and nearly pointless. At ub2048 it matches q8_0 on prefill (303.2 vs 303.6) and wins marginally on decode (17.32 vs 16.98 at 20k) for 1 GiB less GTT. The fork’s headline 48% decode win at 64k does not transfer: DS4’s MLA cache (head_count_kv: 1, kv length 512) is about 3.5 GiB against ~80 GB of weights streamed per token, so KV bandwidth is a rounding error. That 48% figure came from a head-dim-128 GQA model.

4. f16 KV is not a penalty. At 314.5 t/s it was the fastest prefill arm, confirming the fork’s “f16 catches up” claim, at a cost of 3–4 GiB more GTT than q4_0.

And one non-result worth stating: concurrency throughput is invariant to every flag tested — 10.61–10.81 t/s per slot and 41.9–42.4 t/s aggregate across all five loading arms. If you are tuning for a multi-agent workload, none of these knobs will help you.

Quality, and why my quality suite was a weak instrument

Ten mechanically-checked tasks at temperature 0: arithmetic with a fractional intermediate, a sorted-slice sum, exact-format compliance, JSON validity, a constraint-satisfaction puzzle, code generation, an exact word count, multi-step time arithmetic, a negative instruction, and factual recall.

Ten items cannot separate configurations that differ subtly, and the 9-vs-10 spread on identical config shows where the variance comes from. On constrained-format questions this thinking model can spend 1,000–6,000 reasoning tokens, and if the budget runs out it returns empty content with finish_reason: length rather than a truncated answer. Budget your completions generously or you will get silent blanks and read them as failures.

Worth admitting: my answer keys were wrong three times during this work — an expected sum that was off, a code-output answer that was off by two, and a logic puzzle that turned out to be unsatisfiable. Each time the model was right and my checker was wrong. If you build one of these suites, verify the keys computationally before trusting a single failure.

The configuration I settled on

llama-server --alias DeepSeek-V4-Flash-IQ2XXS \
  -m DeepSeek-V4-Flash-IQ2XXS-...-imatrix-0731.gguf \
  -ngl 999 -c 524288 -np 4 -cb -fa on \
  -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 \
  --ctx-checkpoints 4 --cache-ram 4096

-ub 2048 is the one flag that matters. KV type is a genuine toss-up — pick on memory, not speed. And context is nearly free on this architecture: doubling -c from 262144 to 524288 cost 1.3 GiB and zero throughput, because the MLA cache is so small. If you are running agent profiles with large system prompts, take the wide context.

What this generalises to

Three things transfer beyond this specific model and box. Check for an architecture-specific fork before you tune anything. Measure your ubatch rather than trusting a default or a README. And instrument your cache hit rate ahead of your token throughput, because on agent workloads a cache restore is worth roughly five hundred prefills.

The rest is model-specific, and I would not carry any of the KV or top-k conclusions to a different architecture. DS4’s compressed MLA cache is what makes KV quantisation irrelevant here; on a GQA model with a fat KV, the same test would very likely come out the other way.