Qwen3.8-Flash-Next on a Mac Studio M5 Ultra: 112 GB of weights, one million tokens

Alibaba released on 26 August 2026 a model of 180 billion parameters of which only six work on each token, and whose thirty-six of forty-eight layers accumulate no cache at all. Here is what it asks of a 512 GB Mac Studio M5 Ultra, and the throughput you can expect from it.

By QDNA · Published on 30 August 2026 · 9 min read

What this model actually is

The official model card states 125 billion parameters of which 6 billion are active per token, to which 51 billion n-gram embeddings are added, bringing the file to roughly 180 billion parameters in total. Those embeddings are not a compute layer but a lookup table of twenty million bigrams and trigrams placed at the second layer, which Qwen presents as a scaling axis demanding less computation than a mixture of experts would.

The repository's config.json gives the exact structure, and that structure drives the entire sizing exercise: forty-eight layers, of which thirty-six use Gated DeltaNet linear attention and only twelve use full attention, with two key-value heads of dimension 256. Routing activates ten experts out of five hundred and twelve on every token.

What it actually weighs under MLX

No conversion comes from the mlx-community organisation, which usually serves as the reference: the available versions are the work of individual contributors, which makes checking provenance a prerequisite before any production use. The sizes below are read from the published safetensors files rather than estimated.

The ratio between these sizes and the parameter count gives 0.62 byte per parameter at 4-bit, appreciably more than the 0.5 byte of a pure quantisation, the gap coming from quantisation scales and from layers kept at higher precision. It is this measured figure, and not the naive arithmetic, that must be carried into any throughput estimate.

Why context costs almost nothing

Only the twelve full-attention layers accumulate a key-value cache, the other thirty-six maintaining a recurrent state of fixed size. The calculation follows directly from two tensors, twelve layers, two key-value heads and a dimension of 256, which comes to 24 KiB per token in FP16 and half that, 12 KiB, in FP8.

The state of the linear layers occupies roughly 57 MB in total, and that value does not move with context length, which is precisely the point of linear attention. The table below draws the consequences for the two windows that matter in practice.

With 400 GB free after the 4-bit weights, the available memory would cover more than sixteen million tokens of FP16 cache. In other words the machine is not what bounds the context, the model is, since its native window stops at 262,144 tokens and only reaches a million through a YaRN extension.

The throughput you can expect

Generation rereads the active weights on every token, so speed depends first on memory bandwidth, which reaches 1.2 TB/s on the M5 Ultra. With 6 billion active parameters at 0.62 byte, each token requires reading about 3.7 GB, which places the arithmetic ceiling at around 320 tokens per second.

That number is a calculation rather than a measurement, and it should be read as such. A bandwidth ceiling is never reached in practice, real utilisation sitting closer to sixty or eighty per cent depending on engine and batch size. Prefill obeys a different logic anyway, bound by compute rather than by memory, as our article on prefill, decode and the KV cache sets out.

Three caveats to know before committing

The 512 GB configuration of the Mac Studio M5 Ultra is announced for the end of October 2026, which rules out any measurement today and explains why this article publishes calculations only. The entry machine stops at 96 GB, which is too little for this model whatever the quantisation retained.

What happens to the 51 billion n-gram embeddings is not documented on the MLX side. Qwen describes them as amenable to offloading, which suggests they are consulted by point lookup rather than traversed on every token, but no source confirms this for that software stack. Were MLX to keep them fully resident and reread them, the throughput ceiling would fall sharply.

Finally, the licence is qwen-community-1.0 rather than Apache or MIT, which makes reading its terms a prerequisite for any commercial use. Our comparison of open-weight models returns to this often-overlooked point.

References

• Qwen/Qwen3.8-Flash-Next, official model card and config.json
• QwenLM/Qwen3.8-Flash-Next, reference repository
• MLX 4-bit conversion, safetensors size as read