gensai.aiDistillation is a technique in which a large model, the teacher, is used to train a smaller model, the student, to reproduce its answers. The student keeps most of the teacher's abilities at a much lower running cost. In 2025, DeepSeek released distilled versions of its R1 model built on Qwen and Llama.
In practice at Gensai
Distilled models are among the options Gensai considers for On Premise deployments, where available computing power is limited.
Same theme
LLM (Large Language Model)SLM (Small Language Model)DLLM (Diffusion Large Language Model)Hallucination (AI)Prompt chainingPre-trained modelFoundation modelGenerative AITransformerTokenContext windowPromptPrompt engineeringContext (AI)Memory (AI)Fine-tuningOpen-weightQuantizationLoRA (Low-Rank Adaptation)Mixture of Experts (MoE)Reasoning modelTemperature (AI)