Why will you trust a stranger to fly you 35,000 feet above the ground, but hesitate to let an AI send a two-line email? That question has been bothering me. The obvious answer is that flying a plane is different from sending an email. The pilot is trained, accountable, visible, and operating within a system we understand. But there’s something deeper here. We are perfectly comfortable delegating enormous amounts of control when we understand the relationship between ourselves, the agent, and the outcome. And we become uncomfortable when that relationship becomes ambiguous. That distinction may become one of the defining UX problems of agentic AI We need to separate agency, control, and autonomy These words often get thrown around as if they mean the same thing. They don’t. I think there are three useful distinctions: Sense of agency: “Did I cause this outcome?” Control: “Can I influence, steer, or stop what happens next?” Autonomy: “Does what is happening actually represent what I want?” They overlap, but they’re not interchangeable. You can have control without agency. You can have agency without much control. And you can have a highly autonomous system that executes your instructions while completely misunderstanding your actual intent. This matters because AI agents are beginning to move all three. Humans didn’t evolve simply to dominate It’s tempting to explain our obsession with control by saying: Humans evolved to dominate their environment. There’s some truth in that, but it’s incomplete. A more useful evolutionary distinction is between dominance and prestige. Dominance is influence obtained through coercion: threat, intimidation, or control over resources. Prestige is different. Prestige comes from perceived competence. Someone knows how to hunt, build, heal, navigate, negotiate, or solve a problem better than you do. You voluntarily defer to them because learning from them or following them benefits you. Researchers studying human social status have repeatedly distinguished these two pathways. And humans are unusually dependent on social learning. We don’t need to rediscover everything ourselves. We watch someone who knows more. We copy them. We trust them. We delegate. That means the human story isn’t simply: “I need to control everything.” It’s also: “I know when someone else is better equipped to handle this.” That distinction becomes extremely important with AI. AI could become the ultimate prestige system We don’t have to imagine AI as some oppressive machine taking control from humanity. A more realistic scenario is much stranger. We voluntarily give it control because it knows more than we do. Think about what happens when you ask an AI to debug code, summarize a legal document, analyze a dataset, plan a trip, or write a proposal. You’re not necessarily thinking: “This machine is forcing me to obey.” You’re thinking: “It probably knows how to do this better than I do.” That’s prestige. But there is a catch. Once we defer to a system because of its perceived competence, we risk epistemic deference: accepting its judgment because we assume it knows better. This is where automation bias becomes relevant. Research on human-automation interaction describes a spectrum between rejecting automated advice and becoming overly reliant on it. The ideal isn’t blind trust or blind distrust, but appropriate vigilance. So the danger isn’t only that AI takes control. It may be that we willingly hand it over. The causal chain is changing For decades, software mostly followed a simple model: Human → Tool → Outcome You click a button. The system responds. You choose the next action. The causal chain is visible. Agentic systems introduce something fundamentally different: Human → AI → Decision → Action → Outcome The AI sits in the middle. It interprets your intent. It decides what to do. It executes. And only then do you see the result. This is not simply a better interface. It is a different relationship between human intention and machine action. The paradox of agentic UX Here’s the paradox: Humans want outcome control without execution friction. We want to say: “Get this done.” We don’t want to perform every step ourselves. But if the system removes all of the execution, something else can disappear with it: the connection between intention and action. Call this the intent-action rupture. You intended something. The AI interpreted it. The AI made a series of decisions you never saw. The system produced an outcome. And suddenly you are looking at something you technically asked for but don’t quite feel that you authored. The system succeeded. But the experience feels strangely unfamiliar. Think about Google Maps Google Maps is a beautiful example of delegated intelligence. You say: “Take me there.” The system calculates the route. You don’t manually compare every road. You don’t calculate traffic probabilities. You don’t optimize the route yourself. You’ve delegated the cognitive work. But you’re still holding the steering wheel. You have continuous feedback. You can turn left. You can ignore the recommendation. You can reroute. You can stop. The system has autonomy over route planning while you retain meaningful control over movement. Now compare that with an autonomous financial agent. You say: “Manage my finances better.” Three hours later: “I moved ₹25,000 from your savings account, cancelled two subscriptions, paid your credit card, and invested the remainder.” Maybe it made objectively good decisions. But the experience can still feel unsettling. Why? Because the causal chain has become opaque. You didn’t merely delegate execution. You delegated decision-making itself. The psychology of agency gives us a clue Patrick Haggard’s research on the sense of agency gives us an important piece of this puzzle. The sense of agency is the feeling that we control our actions and, through them, events in the external world. One experimental measure is intentional binding: people tend to perceive their voluntary action and its outcome as temporally closer together. This provides an implicit window into how the brain links action with consequence. The important point isn’t that intentional binding gives us a ready-made formula for designing AI agents. It doesn’t. The useful insight is more basic: Our experience of authorship depends partly on how our actions connect to their consequences. And modern human-machine interaction research explicitly examines how the sense of agency operates when machines participate in action. This gives designers an interesting question: What happens when the machine increasingly occupies the space between intention and consequence? We don’t have to assume the answer is “people will lose agency.” But we should stop assuming that removing effort is psychologically free. Automation has never been binary This isn’t a new problem. In 2000, Parasuraman, Sheridan and Wickens proposed a model of automation that described different levels of human-machine interaction. The machine could: offer alternatives narrow the choices recommend one option execute after approval execute automatically while allowing a veto execute and then inform the human or operate almost completely autonomously In other words: Automation is a spectrum. This is incredibly relevant to agentic UX. Because “AI agent” shouldn’t be treated as one interaction pattern. The better question is: At which parts of the task should the machine be autonomous, and where should the human retain authority? Different tasks can have different answers. Why control matters beyond trust There’s another reason I think designers should care about this. Control isn’t only about whether users trust a system. It’s also connected to whether they learn how to operate in their environment. Research on learned helplessness has shown how prolonged experiences of uncontrollability can produce passivity. More recent work emphasizes the importance of learning that one’s actions can actually influence outcomes. We should be careful here. An AI assistant isn’t going to automatically create “learned helplessness” just because it performs tasks for us. That would be an unjustified leap. But the underlying principle is relevant: People learn through action → consequence relationships. If software constantly changes the environment for us while hiding the causal chain, we may become less capable of understanding how the system works. The product becomes convenient. The user becomes dependent. Those aren’t necessarily the same thing. So what should agentic UX actually do? I don’t think the answer is putting an approval dialog in front of every AI action. That would just recreate the old UI with an AI assistant sitting behind it. Instead, I think we need to design for meaningful agency. 1. Confirm intent, not every action Don’t ask: “Should I click this button?” Ask: “You want me to reduce this month’s spending by ₹20,000 without touching your emergency fund. Correct?” Confirm the goal and constraints. Then let the agent execute. 2. Make important actions reversible If the AI sends an email, give me a meaningful grace period. If it reorganizes my calendar, let me undo the entire operation. If it changes a configuration, preserve the previous state. Reversibility is a form of control. You don’t need to prevent every mistake if you make mistakes recoverable. 3. Give me a causal narrative Not: “Done.” Instead: Done. I moved ₹25,000 because your balance was above the threshold you set. I left ₹15,000 untouched because you marked it as emergency savings. You can undo these changes until tomorrow. That’s not an explanation of the model. It’s an explanation of what happened to my world and why. That distinction matters. 4. Let autonomy be adjustable Different users want different levels of delegation. And even the same user wants different levels depending on the task. I might happily let an AI automatically organize my files. I probably want more control before it sends an angry email to my client. So instead of a binary: AI ON / AI OFF think: AI suggests → AI prepares → AI executes with approval → AI executes with veto → AI executes autonomously The user should be able to move along that spectrum as their confidence grows. 5. Design for scaffolding The best agent may not always maximize autonomy. Sometimes it should teach the user how much autonomy they can safely delegate. Start with: “Here’s what I would do.” Then: “I’ll prepare it for you.” Then: “I’ll do this automatically unless you stop me.” Eventually: “I’ll handle this within the rules you’ve established.” That’s scaffolding. The system gradually earns autonomy instead of demanding it. The real UX question For years, the dominant interaction model was: Learn the interface → perform the task. Agentic AI is pushing us toward: Express the intent → delegate the task. That is a much bigger shift than adding a chatbot to an existing product. If you’re simply taking a legacy interface, putting an AI assistant on top of it, and calling the product “agentic,” you may be missing the fundamental design problem. The question isn’t: “How do we keep humans in the loop?” The question is: “Which parts of the causal chain should humans still own?” That’s a different design brief. We may be entering the age of delegated agency I don’t think the future is humans controlling everything. That would defeat the purpose of autonomous systems. And I don’t think the future is humans surrendering everything. That would make us passengers in systems we don’t understand. The interesting middle ground is delegated agency. The human defines the intent. The human establishes the boundaries. The AI handles the execution. The system makes important decisions visible. Actions remain reversible where possible. And autonomy increases as the relationship earns it. In other words: Let the AI take over the work. Don’t make it take over the meaning of the work. That may be the real transition from software we operate to systems we delegate to. And perhaps the best agentic experiences won’t be the ones that give users the most control. They’ll be the ones that understand which control users actually need to keep. That, to me, is the UX problem worth solving.
The three conditions
It applies to qualitative testing.
Watching people attempt tasks, not measuring how long they take. The moment you want a number for a board deck, five is nowhere near enough.
It assumes one comparable group.
One kind of user with one kind of job. Two segments is two studies. A marketplace with buyers and sellers does not get to run five sessions and call it done.
It is a loop, not a sample.
The recommendation was always iterative. Several small tests beat one big one, because the point is to fix things between rounds rather than cataloguing them.
What we do about it
So when we say five users, we mean five this week, five more after the next change, and five after that. Inside a two week sprint that is five or six in week one, and two or three of the same people back on the screens in week two.
It also means we say no to the version founders sometimes ask for. If you have two segments and one week, we will tell you that you are buying half a study. That is still worth doing. It is just worth knowing.
If someone quotes the five user rule at you in a planning meeting, the useful follow up is not "is that true?" It is "which of the three conditions are we breaking?"