Local LLM: which open-source AI model to choose? 2026 comparison

GLM, DeepSeek, Kimi, Mistral, Qwen: open-weight models now rival the best APIs and run on your own hardware, and here is how to choose based on your use case, your hardware and your constraints.

By QDNA · Published on 18 July 2026 · Updated on 18 October 2026 · 9 min read

Updated on 18 October 2026: The open-weight line has moved since this review, as Meta published Muse Spark Open on 15 October 2026, NVIDIA its Nemotron 3.5 Super VL the same day, and DeepSeek its V4.1 Flash 1017 on the 17th. Xiaomi, Ling and StepFun have also released open models recently.

The ranking set out here remains that of the stated date, and the entrants mentioned above were not measured in it. The site's fact base now carries their prices and sizes, verified on 18 October 2026, while the provider pages detail where the data goes for anyone arbitrating today.

Updated on 16 October 2026: Artificial Analysis index v4.3.2 re-read on 16 October 2026. Four open models join the comparison: StepFun's Step 5 Preview (44, level with Kimi K3), InclusionAI's Ling 3.1 Flash (41), Meta's Muse Spark Open (39, released on 15 October 2026) and NVIDIA's Nemotron 3.5 Super VL (26). MiMo-V2.6-Pro (46) and GLM 5.3 Flash (42) get their own row in the table, and Nemotron 3 Ultra is no longer listed with a one-million-token context, since its published configuration declares 262,144. Three captures of the memory calculator show which GPUs hold a compact model and a large one.

Updated on 8 October 2026: Artificial Analysis index v4.3.2 re-read on 8 October 2026 (GLM 5.3 at 44.8, Kimi K3 at 43.6, DeepSeek V4.1 Flash at 39.5, DeepSeek V4 Pro 0813 at 36.0).

Updated on 26 September 2026: Artificial Analysis ranking re-read in version 4.3.2 on 26 September 2026 (recalibrated scale: GLM 5.3 at 45, Kimi K3 at 44, MiMo-V2.6-Pro first at 46, Claude Opus 5.5 at 58), Qwen 3.8 as the single-card model.

A shelf with five abstract model cores of different sizes and colours, a caliper in front, and a small server ready to receive one.

Short answer. For maximum quality, Xiaomi's MiMo-V2.6-Pro (46) and GLM 5.3 (45) lead open models on the Artificial Analysis index, ahead of StepFun's Step 5 Preview and Kimi K3 (44 each; version 4.3.2 read on 16 October 2026). For the best price-performance ratio, DeepSeek V4 or GLM 5.3 Flash. For agentic work and code, Kimi K2.7. For European sovereignty, Mistral. To run on a single card, Qwen 3.8 or Gemma 4. All of them install locally via vLLM or llama.cpp, on NVIDIA servers certified by HPE, Dell, Supermicro or Lenovo.

Why choose an open-source LLM locally?

An open-weight model runs on your own hardware, without sending your data to a third party, quality having caught up with the best closed interfaces, and the cost per token collapses once the installation is paid off, while the permissive licence allows commercial use and sovereign hosting.

The general framework is set out in our definition of the sovereign AI platform.

Comparison of 2026 open-source LLMs

Sizes indicate the total parameter count, but mixture-of-experts architectures only activate a fraction of them per token, which keeps inference compute cost down.

Model	Size	Strengths	For whom
MiMo-V2.6-Pro	1020B	First open model on index v4.3.2 (46), text, image, video and audio, 1M context, MIT licence	Demanding organisations
GLM 5.3, 5.3 Flash and 5.2	744B and 320B	Maximum quality (GLM 5.3 at 45 on index v4.3.2, second open model; 5.3 Flash at 42 with 18B active), reasoning, code	Demanding organisations
Step 5 Preview	600B	44 on index v4.3.2, 27B active, text, image and video, 1M context	Multimodal agents
Kimi K3	2800B	Frontier level (44), 1M context, vision	Large-scale agents
Ling 3.1 Flash	560B	41 on index v4.3.2, 25B active, hybrid reasoning, MIT licence	Long-running coding agents
DeepSeek V4	1600B	Price-performance ratio, Pro and Flash (V4.1 Flash at 39)	Versatile foundation
Muse Spark Open	816B	39 on index v4.3.2, 42B active, Muse licence, access approved by Meta	General-purpose use
Kimi K2.7 Code	1000B	Agentic, tool calls, MCP	Development agents
Nemotron 3 Ultra and 3.5 Super VL	550B and 120B	NVFP4, NVIDIA ecosystem, image and video for 3.5 Super VL	NVIDIA-aligned
MiniMax M3	428B	Multimodal, long context, cost-effective	Documents and image
Mistral	Open weights	Sovereign FR and EU, code and reasoning	European sovereignty
Qwen 3.8 and Gemma 4	4 to 31B	Compact, single card	SMEs and edge

Which open-source LLM by use case?

For reasoning and code generation, MiMo-V2.6-Pro (46), GLM 5.3 (45), Step 5 Preview and Kimi K3 (44) lead the Artificial Analysis index v4.3.2. GLM 5.3 Flash (42), Ling 3.1 Flash (41), DeepSeek V4.1 Flash and Muse Spark Open (39) follow, then DeepSeek V4 Pro (36), while for autonomous agents chaining tool calls, Kimi K2.7 Code brings stability.

For long context and large documents, Kimi K3, MiMo-V2.6-Pro, Step 5 Preview and MiniMax M3 offer a million tokens, and Kimi K3 measures up against closed APIs since the Artificial Analysis index v4.3.2 gives it 44 (index v4.3.2 of 8 October 2026: 43.6, behind GLM 5.3 at 44.8), against 58 for Claude Opus 5.5 and 53 for both Claude Fable 5.1 and GPT-6 Astra. Its weights are published under the Kimi K3 licence, but its 2,800 billion parameters take on the order of 1.4 TB at four bits per parameter: this model belongs in a rack, not on a workstation. Muse Spark Open, released by Meta on 15 October 2026, belongs there too: its 816 billion parameters, 42 billion of them active, take about 410 GB at four bits per parameter, and its weights can only be downloaded once Meta has approved the access request. For European sovereignty, Mistral remains the reference option, and for an SME or edge use case, Qwen 3.8 and Gemma 4 fit on a single card.

Choosing by hardware and partners

Large mixture-of-experts models require a GPU server, for example an RTX PRO 6000 server or an H200 rack, while compact models run on a DGX Spark workstation or a Mac Studio.

To check that a model fits a given machine, our VRAM memory calculator adds up the weights, the key-value cache and the state of recurrent layers, then checks, machine by machine, the memory of each GPU, the speed per user and the time to first token. The captures below come from it: they are calculations, not measurements, and each caption links to the scenario in the calculator.

QDNA memory calculator card for an RTX 5090 workstation: Qwen 3.8 27B in NVFP4 uses 22.3 GB out of 28.9 usable, 52 tokens per second per user, status qualified. — Qwen 3.8 27B in NVFP4 on a single RTX 5090: 22.3 GB used out of 28.9 usable and 52 tokens per second, qualified for one user at 15 tokens per second with an 8,192-token context (calculated on 16 October 2026). Open this scenario in the calculator.

QDNA memory calculator card for an eight-GPU RTX PRO 6000 server: GLM 5.3 Flash in NVFP4 uses 26.6 GB out of 92.1 per GPU, 45 tokens per second per user, status qualified. — GLM 5.3 Flash in NVFP4 for 16 users at 30 tokens per second: the eight-GPU RTX PRO 6000 server is qualified at 45 tokens per second per user, the four-GPU one tops out at 29 (calculated on 16 October 2026). Open this scenario in the calculator.

QDNA memory calculator card for an eight-GPU DGX H200: GLM 5.3 Flash in NVFP4 uses 26.6 GB out of 137 per GPU, 74 tokens per second per user, status qualified. — Same scenario on an eight-GPU DGX H200: qualified, with 26.6 GB used out of 137 per GPU and 74 tokens per second per user (calculated on 16 October 2026). Open this scenario in the calculator.

Compute relies on NVIDIA GPUs, and servers are certified by HPE, Dell, Supermicro, Lenovo, ASUS and Gigabyte, which guarantees manufacturer support and warranty. The tier breakdown, from the workstation to the rack, is in our hardware pages.

Sourcing models: Hugging Face, NVFP4, Unsloth

Weights are downloaded from Hugging Face, the distribution hub for open models, where each vendor publishes under its official organisation: zai-org for GLM, deepseek-ai, moonshotai for Kimi, mistralai, Qwen, stepfun-ai, inclusionAI for Ling, XiaomiMiMo, or meta-models for Muse Spark Open, where access goes through a form approved by Meta. The model card specifies the licence, the weight format, generally safetensors, and the serving instructions, while the huggingface-cli tool pulls the files directly onto the inference server.

For Blackwell GPUs, NVIDIA publishes versions already quantised to the NVFP4 format, a 4-bit format whose gain is read from repository sizes: nvidia/Qwen3.6-35B-A3B-NVFP4 weighs 23.4 GB against 71.9 GB for the original BF16 weights, a factor of 3.1, whereas nvidia/DeepSeek-V4-Flash-NVFP4 weighs 168 GB against 167 GB for the official checkpoint, already published in FP8 and MXFP4 (safetensors sizes read on 2 September 2026). These repositories, like nvidia/DeepSeek-V4-Pro-NVFP4, serve as-is with vLLM or TensorRT-LLM, with no calibration step on your side. An RTX PRO 6000 server or a B200 rack runs this FP4 compute natively.

Unsloth complements the pipeline on two fronts: its dynamic-quantisation GGUF conversions tune the compression layer by layer, which preserves the quality of models served by llama.cpp on a workstation. The project also publishes its own NVFP4 versions, such as unsloth/Qwen3.6-27B-NVFP4, 23.4 GB of weights against 55.6 GB in BF16, which leaves room on a 24 GB card for a short context only. The same library handles QLoRA fine-tuning on a single card: the model specialises on your data without it ever leaving your walls. Details are in our Unsloth page.

Diagram of an open model's pipeline: downloading safetensors weights from the official organisations on the Hugging Face hub, choosing a quantised format, 4-bit NVFP4 for a Blackwell server or layer-compressed Unsloth GGUF for a workstation, then serving via vLLM and TensorRT-LLM or via llama.cpp and Ollama — An open model's pipeline: from the Hugging Face hub to the server, in NVFP4 for Blackwell GPUs or in Unsloth GGUF for a workstation.

Licence and sovereignty

Open weights does not always mean open source in the strict sense. Check the licence, often MIT, Apache or permissive, before commercial use, knowing that running locally guarantees that data never leaves the company, which eases GDPR compliance and HDS hosting of health data. Terms vary from one model to the next: Ling 3.1 Flash and MiMo-V2.6-Pro are MIT-licensed, while Muse Spark Open requires the notice "Built with Muse Spark Open" and a separate agreement with Meta for a model-as-a-service business above 20 million dollars of revenue over twelve months, a threshold also found in the Kimi K3 licence.

Interchangeable through a gateway

These models are all unified behind a single interface with LiteLLM. Switching models takes a single configuration line with no application code to rewrite, and the practical follow-up is in our guide to installing a local LLM in the enterprise.

Frequently asked questions

What is the best open-source LLM in 2026?

MiMo-V2.6-Pro (46) and GLM 5.3 (45) lead open models on the Artificial Analysis index (version 4.3.2 read on 16 October 2026), ahead of Step 5 Preview and Kimi K3 (44), against 58 for Claude Opus 5.5; Kimi K3's weights have been published on Hugging Face since 16 July 2026. For the best cost-performance ratio, DeepSeek V4 delivers frontier-level quality at reduced cost.

Can an open-source AI model run locally?

Yes. Served by vLLM on a GPU server or by quantised llama.cpp on a workstation, an open-weight model runs entirely on your own infrastructure, with no external dependency.

Which open-source model for local coding?

Kimi K2.7 Code for agentic stability and tool calls, GLM 5.3 or DeepSeek V4 for generation quality. Mistral's Codestral is the European sovereign option.

What hardware should run an open-source LLM?

A DGX Spark or Mac Studio workstation for compact models, an RTX PRO 6000 server or an H200 rack for large models. Servers are certified by HPE, Dell, Supermicro and Lenovo on NVIDIA GPU bases.

Where can you download an open-source LLM?

On Hugging Face, from the vendor's official organisation. NVFP4 releases published by NVIDIA and Unsloth's GGUF builds are ready to serve: the former for a Blackwell server under vLLM, the latter for a workstation under llama.cpp.

Is an open-source LLM worth as much as a closed API like ChatGPT?

On quality, the best open models now match closed APIs in reasoning and code. The difference lies in control: on-premises, your data stays with you and the cost per token collapses once the hardware is paid off, whereas an API bills every call and processes your data externally. The detailed calculation is in our comparison of on-premises versus API.