Qwen3.8-27B on-premises: 262,144 tokens of context on a single machine
Released on 13 August 2026 under the Apache 2.0 licence, Qwen3.8-27B brings a native 262,144-token context. Long context is no longer reserved for GPU racks. Here is the real sizing, card by card.
What changes with Qwen3.8-27B
Qwen3.8-27B is a dense 27-billion-parameter model (all parameters active, without any mixture of experts), released by Alibaba on 13 August 2026 under the Apache 2.0 licence together with a native vision encoder covering images and hour-scale videos. On benchmarks it establishes itself ahead of considerably larger models on agentic software engineering: 61.7 on SWE-bench Pro (versus 53.4 for Opus 4.6 Max under the identical evaluation protocol), 90.3 on LiveCodeBench v6, and 84.3 on OSWorld-Verified for computer use, with reasoning depth adjustable per request through reasoning_effort and enabled by default.
For an on-premises deployment, however, the most structural consideration lies elsewhere: the native context reaches 262,144 tokens, extensible to 1 million via YaRN, and above all the memory cost of maintaining that context has collapsed, which is precisely what determines the hardware bill.
Why the KV cache becomes (almost) affordable
In a conventional attention model, every token in the context permanently occupies memory in the form of the KV cache, whose mechanics we previously documented in our article on prefill, decode and KV cache. For a full-attention dense model of this magnitude, that cache rapidly becomes the dominant memory item as soon as the context exceeds a few tens of thousands of tokens.
Qwen3.8-27B sidesteps a substantial part of that limitation through its architecture: its 64 layers follow a hybrid arrangement of 16 blocks, each composed of three Gated DeltaNet layers (linear attention) followed by a single full-attention layer, so that only 16 layers out of 64, one quarter, feed a growing KV cache. The 48 linear layers keep a constant recurrent state of about 75 MB in total, regardless of context length.
The arithmetic gives 64 KiB per token in FP16 (2 × 4 KV heads × 256 dimensions × 16 layers × 2 bytes), i.e. 32 KiB in FP8: roughly four times less than an equivalent fully attentive dense model.
| Context | KV cache FP16 | KV cache FP8 |
|---|---|---|
| 32,768 tokens | 2 GB | 1 GB |
| 131,072 tokens | 8 GB | 4 GB |
| 262,144 tokens (native) | 16 GB | 8 GB |
| 524,288 tokens (YaRN ×2) | 32 GB | 16 GB |
| 1,048,576 tokens (YaRN ×4) | 64 GB | 32 GB |
Note equally that the KV cache is paid per concurrent request: serving four simultaneous users at 262,144 tokens in FP8 requires 32 GB of cache on top of the weights, so platform sizing must reason in terms of weights plus N times the cache, never in terms of weights alone.
Weights + cache: the real sizing
The weights take up about 62.1 GB in FP16 (vision included), 29 GB in FP8 and 15.5 GB in NVFP4. Crossed with the hardware catalogue, four profiles emerge:
- 24 GB card (RTX 3090/4090): NVFP4 or Q4, short-to-medium context. Community measurements on an RTX 3090 report approximately 43 tokens per second in Q4_K_XL with MTP speculative decoding, occupying 19 GB of VRAM, a configuration appropriate for a single-user workstation.
- DGX Spark, 128 GB unified: full FP8 + native 262,144-token context, with headroom for a few concurrent requests. The first hardware tier in which the native context window becomes genuinely exploitable. Details: Qwen 3.8 27B on DGX Spark.
- Mac Studio Ultra, 512 GB: full FP16 at maximum context, or FP8 with heavy concurrency. Details: Qwen 3.8 27B on Mac Studio Ultra.
- RTX PRO 6000 (768 GB) or SXM server: FP8 + YaRN ×4 to approach one million tokens while keeping comfortable batches. Details: RTX PRO 6000 and H200 SXM.
Serving it: engines and settings
The officially recommended engines are vLLM, SGLang and TokenSpeed, each with a dedicated recipe. Beyond 262,144 tokens, extension goes through YaRN, where a scaling factor of 2 suffices for 524,288 tokens and preserves model performance considerably better than the factor-of-4 configuration required for the million-token regime. Operators should nevertheless remember that open-source engines implement static YaRN, which, left permanently enabled, can measurably degrade shorter contexts, and should therefore only be configured when the workloads genuinely require it.
Two configuration settings deserve attention from the outset: preserve_thinking, enabled by default, retains reasoning from previous turns and measurably improves cache reuse across the successive turns of an agentic session, while the recommended output budget is deliberately generous (up to 262,144 reasoning tokens, 131,072 answer tokens), a margin that must be planned inside the context window.
The llama.cpp / GGUF caveat
GGUF conversions already exist (unsloth distributes Q4_K_XL and Q6_K, accompanied by MTP drafts for speculative decoding), but operators should beware of outdated builds, in which a defect in the CUDA kernels handling the DeltaNet layers produced completely corrupted output without any visible error or measurable slowdown. Operators who nevertheless go through llama.cpp should use a recent build and verify that the loaded libggml-cuda libraries are up to date, whereas a production platform should rely on vLLM or SGLang, where performance on this hybrid architecture is markedly higher.
Key takeaways
Qwen3.8-27B makes long context compatible with desktop-class hardware: 262,144 tokens cost 16 GB of FP16 KV cache, and a 128 GB unified machine covers an FP8 model plus its native context. For long agentic workloads (whole-repository code review, document analysis, automation sessions), it is the first open model of this size where the context window no longer dictates the server architecture. Precise per-machine sizing is detailed in our dimensioning sheets.
Sources: the Qwen3.8-27B model card on Hugging Face, from which the parameter count, native context length and attention configuration are taken, and the llama.cpp documentation for GGUF formats.
Frequently asked questions
What hardware do you need to run Qwen3.8-27B locally?
In 4-bit quantisation (Q4_K_XL or NVFP4) the model fits in roughly 15.5 to 19 GB: a 24 GB card such as an RTX 3090 or 4090 handles short contexts. To use the full native 262,144 tokens, budget for the weights plus 16 GB of FP16 KV cache (8 GB in FP8): a 128 GB machine like a DGX Spark covers the whole stack in FP8.
What is Qwen3.8-27B's KV cache size?
64 KiB per token in FP16 and 32 KiB in FP8, i.e. 16 GB for 262,144 tokens in FP16. This low figure comes from the hybrid architecture: only 16 of the 64 layers use full attention; the other 48 (Gated DeltaNet) keep a constant state of about 75 MB, independent of context length.
Does Qwen3.8-27B work with llama.cpp or Ollama?
GGUF conversions exist (unsloth) and llama.cpp supports it, but insist on a recent build: older CUDA kernels for the DeltaNet layers produced silently corrupted output. For serious use, the engines Qwen recommends are vLLM, SGLang and TokenSpeed, markedly faster on this architecture.