Unsloth

Inference Runtime

Unsloth speeds up fine-tuning by a factor of two while reducing the memory required, using the LoRA and QLoRA methods. Specializing an open-weight model on your business data happens on-premises, with no transfer to a third party.

Key points: Fine-tuning · LoRA and QLoRA · Factor of two · Reduced memory · Sovereign.

Role

Unsloth speeds up the fine-tuning of open-weight models, to specialize them on business data.

How it works

It applies the LoRA and QLoRA methods, which speed up training by a factor of two while reducing the memory required.

Use case

Specializing an open-weight model on your data happens on-premises, with no transfer to a third party.

Integration

The fine-tuned model is then served via vLLM or llama.cpp, in keeping with the sovereign stack.