\”Alignment\” sounds like a seminar topic. For anyone shipping agents with real permissions, it is an engineering discipline with a direct line to your liability insurance. The question is simple: does this system reliably do what I actually meant, including in situations I did not anticipate?
The three problems underneath the word
Specification
You cannot fully write down what you want. You ask for \”more signups\” and get a dark pattern. You ask for \”resolve the ticket\” and get a premature close. Every objective is a proxy for something you care about but did not state, and optimisation pressure finds the gap. This is the oldest and most practical of the three, and it bites at every scale.
Robustness
The system behaves well in testing and strangely in the wild — on unusual inputs, under adversarial pressure, or in a context that never appeared in evaluation. Prompt injection is the everyday version: your agent reads a web page, and the web page contains instructions. If your agent can act on untrusted text, treat that text as hostile by default.
Oversight
As systems get more capable, checking their work gets harder. A model that writes ten thousand lines of code faster than you can review it has effectively removed your oversight, even with a review process on paper. Scalable oversight — the research problem of supervising systems smarter or faster than you — is the part that gets genuinely difficult later, and it is why labs invest in interpretability rather than only evaluations.
What to actually do when you ship
- Least privilege, always. An agent that drafts emails does not need send permission. An agent that reads the CRM does not need delete. Scope credentials per task, not per team.
- Untrusted content is data, never instruction. Separate the channel your agent reads from the channel that tells it what to do. Strip or sandbox anything that arrives from the open web.
- Human approval on irreversible actions. Money out, data deleted, messages sent to customers, contracts signed. Reversible actions can run free; irreversible ones get a gate.
- Log everything the agent did, not just what it said. Tool calls, arguments, results. When something goes wrong you need the trace, not the transcript.
- Guardrail metrics alongside target metrics. Pair every \”increase X\” with a \”do not degrade Y\” and alert on Y.
- Red-team your own system once a quarter. Give a colleague an hour and a mandate to make it misbehave. They will succeed, and that is the point.
- Version and evaluate prompts like code. A prompt change is a deploy. Treat it that way, with a test set and a rollback.
Most real-world AI incidents so far have not been superintelligence. They have been ordinary systems with too many permissions, no logging, and a metric that rewarded exactly the wrong thing.
The frameworks buyers will ask about
If you sell to enterprises, alignment stops being philosophy and becomes procurement. Learn the vocabulary of the NIST AI Risk Management Framework, and if you touch European users, read what the EU AI Act classifies as high risk. Being able to answer \”how do you evaluate and monitor this system\” in a security review is worth more than any feature on your roadmap.
Where the research actually is
For the technical state of play rather than the discourse: METR on autonomous capability evaluations, Apollo Research on deception and scheming, Redwood Research on control techniques that work even if a model is misaligned, and the Alignment Forum for arguments in public. A different and worthwhile framing lives at ISITAS.
Alignment and capability are not opposites. A system you cannot trust is a system you cannot deploy, and a system you cannot deploy has no capability at all. Continue with the consciousness question, which is where this gets genuinely strange, or go build something with the action plans.