Fine-tuning

Language models

Fine-tuning partially retrains a pre-trained model on specific data to adapt it to a domain or a style: legal vocabulary, a brand's tone, a precise output format. Heavier than RAG, it is justified when the model's very behaviour must change, not just its knowledge.

In practice at Gensai

Gensai favours RAG to bring in knowledge, and reserves fine-tuning for cases where the style or output format must be deeply adapted.

Same theme

LLM (Large Language Model)SLM (Small Language Model)DLLM (Diffusion Large Language Model)Hallucination (AI)Prompt chaining Pre-trained model Foundation model Generative AI Transformer Token Context window Prompt Prompt engineering Context (AI)Memory (AI)