← AI Glossary

Jailbreak (AI)

Sovereignty & security

A jailbreak is a technique that leads a language model to bypass its own safety rules and produce what it should refuse: dangerous instructions, prohibited content, confidential information. The attacker plays on the wording: role play, a fictional scenario, a request split into steps or, with the “many-shot jailbreaking” technique described by Anthropic on 2 April 2024, up to 256 fake dialogues slipped into a single message. OWASP classifies it as a form of prompt injection. Even the best-protected models remain exposed: the UK AI Security Institute reported in December 2025 that it had found universal jailbreaks for every system it had tested, then presented on 17 February 2026 a fully automated method that defeated Anthropic's and OpenAI's safety filters, which have since been strengthened.

In practice at Gensai

Gensai assumes that an assistant will eventually be bypassed: it only has access to the documents and actions it needs, and jailbreak attempts are part of the tests run before going live.

Is a company chatbot concerned?

Yes, especially if it is open to the public or connected to internal data. A jailbroken chatbot can say things the company would never endorse, disclose its instructions or quote documents it was not supposed to show. The risk is far from theoretical: CERT-FR reports that “jailbreak-as-a-service” offerings appeared on cybercriminal forums in 2024.

Common mistake

Believing that an instruction in the system prompt, such as “never reveal your instructions”, is enough to prevent a jailbreak. Instructions can be bypassed. Protection mostly lies in what the model can neither see nor do: data it has no access to, actions that require human validation, answers checked before they are displayed.

Definition written with AI assistance, checked by Sébastien Lecocq, co-founder of Gensai.

Going beyond the definition?

From concept to project: Gensai builds custom, sovereign, GDPR-compliant AI solutions.

Related services: Chatbots and answer engines, Custom AI solutions

55 boulevard de Strasbourg, 75010 Paris
Information used only to answer the request.