Loading…
Glossary term
A prompt designed to get a model to ignore its safety rules.
A jailbreak is a prompt crafted to get a model to ignore its safety training, for example by role-play framing or disguising the request. Developers patch known ones, but new variants keep appearing. Testing for them is part of red teaming.
Jailbreaks show the limits of safety training and why systems use several layers of defense.