Alignment asks whether a system reliably does what we actually meant — including in situations nobody anticipated. It is a research field, an engineering practice and, increasingly, a procurement requirement. This page is the map.
The problem in three layers
- Specification. We cannot fully write down what we want, so optimisation finds the gap between what we said and what we meant.
- Robustness. Systems behave differently in the wild than in evaluation — under adversarial pressure, on strange inputs, in unfamiliar contexts.
- Oversight. Checking the work gets harder as systems get faster and more capable than the people checking.
The full engineering treatment is in AI alignment explained for builders.
What you can do as a builder
Least privilege
Scope every credential to the task. A drafting agent does not get send permission; a reading agent does not get delete.
Treat untrusted content as hostile
Anything your agent reads from the open web may contain instructions. Separate data channels from instruction channels.
Gate irreversible actions
Money, deletions, outbound messages and contracts get a human click. Reversible actions can run free.
Log tool calls, not just text
When something goes wrong you need the trace of what was done, with what arguments, and what came back.
Pair every target metric with a guardrail
Conversions with refunds. Speed with satisfaction. Volume with a human spot check.
Red-team quarterly
Give someone an hour and a mandate to break it. They will. That is the point.
Research organisations and primary sources
How to read this field without losing your mind
Two failure modes: dismissing the whole thing as science fiction, and treating every capability announcement as an omen. Both are ways of not engaging. The productive middle is to read primary material — evaluation reports, model cards, actual papers — at a bounded frequency, and to convert what you learn into decisions about your own systems.
Related: AI consciousness for the harder philosophical question, and singularity or doom for scenario planning you can act on.