An AI assistant can finish a task perfectly and still leave people worse off. Imagine a support system rewarded for closing tickets. It gets faster by making it difficult to complain. The dashboard improves. The customer experience deteriorates.
That gap between the measured objective and the intended outcome is a useful starting point for AI alignment. We want systems that reliably support human purposes, including when instructions are incomplete, circumstances change, or incentives encourage shortcuts.
Alignment has a human side all the way through
Three questions belong together: What does the model do? What does the organization reward? Who gets to decide whether the outcome is acceptable? Technical work addresses behavior and reliability. Institutional design addresses authority, incentives, and accountability. Public participation helps reveal whose needs the process missed.
This does not mean every disagreement can be resolved by a vote. People have different interests, and majorities can be wrong. Some protections should survive any poll result. The aim is a legitimate process for making and revising decisions under disagreement.
From abstract values to observable behavior
“Respect people” is a starting principle. In a support system, it could mean explaining an automated decision, offering human escalation, supporting accessibility, and tracking unresolved complaints. Each commitment can become a test. If the system cannot meet it, the organization has a concrete problem to fix.
For a more capable agent, the same logic extends to permissions and oversight. Can it spend money? Change records? Contact others? Its authority should match both the task and the strength of the controls around it.
Where participation enters
Our DEO proposal puts affected people inside the feedback process. Participants surface failures, propose changes, and review whether those changes helped. Organizers must respond, and substantive work can be compensated when funding exists. This creates a route from voice to action.
The test is practical: did participation improve the system, distribute influence more fairly, and reveal problems that would otherwise have been missed? A growing audience alone does not answer those questions.
Try this: choose one AI tool you use and write its apparent goal, the outcome you actually want, and one way the two could diverge.
Further reading: NIST’s risk framework. Next: The DEO model.
