gensai.aiA Mixture of Experts model is split into many specialised sub-networks, the experts. For each token, a router activates only a small number of them. The model therefore has a very large total number of parameters but uses only a fraction of them for each computation: DeepSeek-V3 has 671 billion, of which 37 billion are active.
In practice at Gensai
Gensai takes the MoE architecture of models such as Mixtral, DeepSeek and Kimi into account when choosing an open-weight model, since it changes memory and speed requirements.
Same theme
LLM (Large Language Model)SLM (Small Language Model)DLLM (Diffusion Large Language Model)Hallucination (AI)Prompt chainingPre-trained modelFoundation modelGenerative AITransformerTokenContext windowPromptPrompt engineeringContext (AI)Memory (AI)Fine-tuningOpen-weightQuantizationDistillationLoRA (Low-Rank Adaptation)Reasoning modelTemperature (AI)
Definition written with AI assistance, checked by Sébastien Lecocq, co-founder of Gensai.
Going beyond the definition?
From concept to project: Gensai builds custom, sovereign, GDPR-compliant AI solutions.
Related services: Chatbots and answer engines, Voice AI
55 boulevard de Strasbourg, 75010 Paris