gensai.aiDistillation is a technique in which a large model, the teacher, is used to train a smaller model, the student, to reproduce its answers. The student keeps most of the teacher's abilities at a much lower running cost. In 2025, DeepSeek released distilled versions of its R1 model built on Qwen and Llama.
In practice at Gensai
Distilled models are among the options Gensai considers for On Premise deployments, where available computing power is limited.
Same theme
LLM (Large Language Model)SLM (Small Language Model)DLLM (Diffusion Large Language Model)Hallucination (AI)Prompt chainingPre-trained modelFoundation modelGenerative AITransformerTokenContext windowPromptPrompt engineeringContext (AI)Memory (AI)Fine-tuningOpen-weightQuantizationLoRA (Low-Rank Adaptation)Mixture of Experts (MoE)Reasoning modelTemperature (AI)
Definition written with AI assistance, checked by Sébastien Lecocq, co-founder of Gensai.
Going beyond the definition?
From concept to project: Gensai builds custom, sovereign, GDPR-compliant AI solutions.
Related services: Chatbots and answer engines, Voice AI
55 boulevard de Strasbourg, 75010 Paris