Unsloth

Inference Runtime

By QDNA · Published on 5 July 2026 · Updated on 2 September 2026

Unsloth speeds up fine-tuning by a factor of two, according to its publisher, while reducing the memory required, using the LoRA and QLoRA methods. Specializing an open-weight model on your business data happens on-premises, with no transfer to a third party.

Key points: Fine-tuning · LoRA and QLoRA · Claimed factor of two · Reduced memory · Sovereign.

Role

Unsloth speeds up the fine-tuning of open-weight models, to specialize them on business data.

How it works

It applies the LoRA and QLoRA methods. The publisher claims training twice as fast with 70% less VRAM on its reference notebooks (Llama 3.1 8B, gpt-oss 20B), figures published in the Unsloth repository and re-read on 2 September 2026; QDNA has not measured them.

Use case

Specializing an open-weight model on your data happens on-premises, with no transfer to a third party.

Integration

The fine-tuned model is then served via vLLM or llama.cpp, in keeping with the sovereign stack.

License and source code

Unsloth is published under the Apache-2.0 licence and implemented in Python. The source is public, which means the runtime can be audited, patched and pinned to a known commit rather than trusted on a vendor's word.

Repository: unslothai/unsloth. Licence and implementation language are re-read from this source by the QDNA fact base. The version number is deliberately not published here: it would go stale between two upstream releases without anything catching it.