Guardrails
Where it sits in the kitchen
Automatic checks around the chef, like the smoke alarm and the lock on the till, that stop or correct unsafe, off-topic or malformed orders and dishes.
Definition
Checks around a model or agent that block or correct unsafe, off-topic or malformed inputs and outputs. They can be rules, classifiers or other models, and work best as one layer among several, not as the only defence.
Detailed technical description
Guardrails can sit at several points: on the input before the model sees it, on the output before the user does, and on tool calls before they run. Methods include pattern rules, allowlists, schema validation, specialised classifiers and a second model acting as judge. Rules are fast and predictable but rigid; classifiers and models are flexible but can be wrong and can add delay and cost.
No guardrail is complete, and attackers probe for gaps, so treat each as one layer. Rules enforced in code, such as least-privilege access and a sandbox, hold even when the model is fooled, while a prompt that says be careful does not. Tune for both kinds of error: too lax lets harm through, too strict blocks legitimate work and teaches people to work around the check.