Running AI locally on your PC (2026)

AI can run on your own machine, without sending your data to the cloud. From your current PC to the DGX Spark mini-supercomputer, here are the options in 2026.

By QDNA · Published 18 July 2026 · 9 min read

Why run AI locally

Running a model on your own machine keeps your data at home, works offline, and removes any subscription. It is also the best way to learn. In 2026 the stack is mature: open-weight models rival the APIs, and hardware has taken a leap forward. For an enterprise deployment, the next step is in our guide installing a local LLM.

The free option: your current PC

On an existing machine, free software is enough to get started. Ollama and LM Studio offer a simple setup on Windows, Mac, and Linux; llama.cpp remains the lightweight reference, combining CPU and GPU with quantisation from Q4 to Q8.

The limiting factor is memory. A compact model such as Qwen 3.6 or Gemma 4, from four to thirty billion parameters, runs on a recent computer with a decent graphics card, or on a Mac with unified memory. Model details are covered in our open-weight LLM comparison.

The desktop mini-supercomputer: the GB10 chip

For larger models, a new category has emerged: the desktop AI supercomputer, built on the NVIDIA GB10 Grace Blackwell chip. It combines a twenty-core Arm processor and a Blackwell GPU on a single package, linked by NVLink, with a coherent pool of 128 GB of LPDDR5X memory. It reaches about one thousand teraFLOPS in FP4, or one petaFLOP with sparsity, in a 150-millimetre-square enclosure.

This 128 GB of unified memory allows a single machine to serve models up to two hundred billion parameters, and up to four hundred and five billion by linking two units over the ConnectX-7 port. NVIDIA's DGX Spark is the reference machine in this family.

The GB10 machines, brand by brand

All these machines share the same GB10 chip, the same 128 GB of memory, the same ports, and the same DGX OS operating system, a variant of Ubuntu. The differences lie in chassis, cooling, storage, remote management, warranty, and price. The ConnectX-7 network port accounts for a notable share of the price.

Prices have risen with the memory shortage. Configurations range from about 2,800 to 5,000 euros depending on storage and brand, with the reference DGX Spark at the top of the range. Breakdown by tier is available in our AI server price guide.

Stacking DGX Spark units for larger models

Each GB10 machine carries two ConnectX-7 network ports on the back, at 200 Gb/s, with RDMA over Converged Ethernet (RoCE): the machines exchange compute data without going through the CPU. NVIDIA documents this setup under the name Spark Stacking. Two DGX Spark units connect via a simple direct QSFP cable, with no switch. The pair aggregates 256 GB of memory and serves Llama 405B in FP4, with weights split across the two machines by tensor parallelism.

Three machines form a switchless ring, using both ports on each unit. Beyond that, a RoCE-capable 200 GbE Ethernet switch federates four units and more: the pool exceeds 512 GB and loads mixture-of-experts models that no single machine could hold. The setup remains a desktop cluster, not a rack: past a few units, the next step is a GPU server.

The right expectation is capacity, not speed. At each compute pass, the machines exchange activations over the network link, and per-token throughput drops compared with a model that fits on a single machine. Stacking also helps with long contexts: splitting a hundred-and-twenty-billion-parameter model across two machines frees up memory for the conversation cache, allowing longer windows and more simultaneous sessions.

The AMD alternative, on Windows

Outside the CUDA ecosystem, the AMD Ryzen AI Max+ 395 chip, known as Strix Halo, also offers 128 GB of unified memory, at a lower price and on Windows. The Framework Desktop and the Corsair station use it, priced around one to two thousand euros. These machines draw less power at idle, but the CUDA software ecosystem remains richer on the NVIDIA side.

Which model for which machine

• Existing PC with a graphics card: a compact quantised model, Qwen 3.6 or Gemma 4, via Ollama or llama.cpp.
• GB10 or Strix Halo machine, 128 GB: a medium to large model, up to two hundred billion quantised parameters.
• Two linked GB10 machines: up to four hundred and five billion parameters.

The precise choice of model is covered in our comparison, and scaling up to a server in the enterprise installation guide.

References

• NVIDIA DGX Spark, official page
• NVIDIA DGX Spark GB10 forum
• NVIDIA DGX Spark review, Tom's Hardware
• ASUS Ascent GX10 review, ServeTheHome
• Spark Stacking, official NVIDIA guide