Transformer

Language models

The Transformer is the neural network architecture behind today's LLMs, introduced in 2017. Its attention mechanism processes all the words of a text in parallel while weighing their relationships, where previous architectures read sequentially. The “T” in GPT is precisely that.

In practice at Gensai

All the models Gensai integrates (Mistral, Claude, GPT, Gemini) are built on variants of this architecture.

Same theme

LLM (Large Language Model)SLM (Small Language Model)DLLM (Diffusion Large Language Model)Hallucination (AI)Prompt chaining Pre-trained model Foundation model Generative AI Token Context window Prompt Prompt engineering Context (AI)Memory (AI)Fine-tuning

Transformer

In practice at Gensai

Go deeper

Same theme