Quantization

Language models

Quantization compresses an AI model by reducing the precision of the numbers that make up its weights, for example from 16 bits down to 4. The model loses a touch of finesse but becomes three to four times lighter and faster, letting it run on standard hardware rather than a specialised GPU server.

In practice at Gensai

Quantization is what allowed Gensai to run the “Monsters of the Oceans” installation fully locally on a simple Mac Mini.

Same theme

LLM (Large Language Model)SLM (Small Language Model)DLLM (Diffusion Large Language Model)Hallucination (AI)Prompt chaining Pre-trained model Foundation model Generative AI Transformer Token Context window Prompt Prompt engineering Context (AI)Memory (AI)Fine-tuning Open-weight

Quantization

In practice at Gensai

Go deeper

In application

Same theme