APoV 004Building, Keeping, and Evolving Trust

Two nights ago, I sat down for an interview about how competition among personal AI agents is shifting from capturing mindshare to earning trust. I think of myself as an AI native: whenever a chore can be handed off to an agent, whether drafting emails, organizing itineraries, or comparing prices before placing an order, I hand it over gladly. Yet when an agent actually asked for my personal ID and payment details, I hesitated for a long time. And there is one habit I still have not broken: when something truly matters, I have to be the one to hit the final confirm button.

It was past eleven by the time the interview wrapped up. Exhausted, I climbed into my car and, almost by reflex, clicked the scroll wheel on the steering wheel to engage Tesla's Full Self-Driving (FSD). I have made that motion so many times it has become muscle memory. Today, I trust FSD with 99% of my daily commute. As the car merges across lanes, takes highway interchanges, and creeps through stop-and-go traffic, I simply sit in the driver's seat, glancing out the window, queuing up a song that fits the mood, and resting my tired feet off the pedals for a bit. By the time my hands return to the wheel, I am almost always pulling up to my destination.

The car carried me south toward home. Along Highway 101, tech headquarters still blazed with light, slipping past one after another into the dark. All of a sudden, a question struck me: FSD holds my life in its hands, and I trust it almost completely; an agent only wants to manage my inbox, and I watch it like a hawk.

Aren't they both AI?

At first, I tried to talk myself out of the comparison. Self-driving cars and chatbots seemed like apples and oranges, one a feat of precision systems engineering and the other an enigmatic language model, so holding them to the same standard felt unfair. A quick look under the hood, however, showed the joke was on me. In March 2024, with FSD v12.3, Tesla ripped out hundreds of thousands of lines of engineer-written routing rules and replaced them with an end-to-end neural network: the model takes in raw camera feeds and decides how to drive on its own. Waymo has gone even further, building a Gemini-based vision-language model directly into its driving stack to handle rare, thorny edge cases, such as coming up on a car ahead that suddenly bursts into flames.

Of course, the two worlds are not identical. The models differ, and so do the rules of the road. Driving happens inside painted lanes and traffic lights, where the boundaries are clear; the world of emails, calendars, and credit cards has almost no guardrails at all. Still, from where I sit as a user, the core bargain is the same: handing a task where mistakes are costly over to a black box whose inner workings I cannot see. Why, then, am I willing to let go of the wheel with one, while keeping the other on such a short leash?

Understanding Was Never a Prerequisite

Start with a truth we rarely stop to consider: we barely understand most of the things we trust every day. General anesthesia has been in use for well over a century; more than 300 million surgeries are performed worldwide each year, and a vast share of them rely on it to put patients under. Scientists have a fairly clear picture of which molecules anesthetic drugs bind to, yet how those molecular actions combine to switch off human consciousness remains an open question. Asked how it works, one anesthesia researcher answered bluntly: "Nobody really knows." Even so, we climb onto the operating table, start counting to ten, and drift off before we finish. Commercial flight is no different. Hardly any passenger in the cabin understands the aerodynamics of a turbofan engine, yet we buckle our seatbelts, close our eyes, and sleep through the night.

Not understanding how something works, in other words, has never been a barrier to trust. There is a classic definition in management scholarship that I love: trust is "the willingness to be vulnerable." In plain English, it means: I know you could hurt me, and I am putting this in your hands anyway.

What makes us willing to take that leap? Piece together decades of research in psychology and human-factors engineering, and trust looks less like a one-time decision than a continuous loop of calibration:

We size up the initial risk by gut instinct, lean on brand reputation and word of mouth as a baseline, and then cautiously hand over a small piece of work to see how the system performs. If it delivers, we hand over a little more; if it botches the job, we pull control back. Simply put, trust is never handed out all at once; it is earned, one turn at a time.

The Passenger-Side Brake

Anyone who went to driving school in China remembers the training car. Over on the passenger floorboard sits a second brake pedal. The instructor rarely touches it during a lesson, yet knowing that pedal is there is the only reason a nervous beginner dares to merge into live traffic. Looking back at how I use FSD, what gives me peace of mind is that exact same pedal: the steering wheel in front of me is my instructor's brake. Researchers at Wharton once ran a telling experiment: when people were allowed to tweak an imperfect forecasting algorithm's output, even slightly, they became far more willing to use it, and how much they were allowed to change barely mattered. More intriguing still, the forecasts people tweaked by hand were often less accurate than the algorithm's raw numbers; yet because having a say persuaded them to use the algorithm in the first place, their overall error still shrank. Clearly, what people crave is not more control, but the presence of control. My refusal to let go of that final confirm button with an agent comes from the exact same impulse.

Alongside the steering wheel, something else keeps me calm: sitting in the driver's seat, I can watch every move FSD makes right in front of me. The split second something goes sideways, I know. An agent is different. It reads and sends emails in the background, out of sight; if it fires off the wrong message, I might not find out for days, and words, once sent, cannot be unsent. What scares people most is often not the danger itself, but what they can neither see nor stop. This August, the personal agent Instinct showed testers just how unnerving that invisibility can be. As TechCrunch reported, one beta tester revoked the app's access to her Gmail at 11 a.m., only to receive an automated email digest from it at 2 p.m. When she pressed for answers, she learned that her inbox had already been copied onto the company's servers. Had that afternoon digest not given the game away, she might never have known.

I still remember the first time I tried FSD, more than two years ago in a friend's car. Perhaps because the car was not mine, I was on pins and needles the entire way: my hands never left the wheel, my foot hovered over the brake, and my eyes stayed glued to the road ahead. Apart from not pressing the accelerator myself, I was practically driving the car the whole trip. Two years later, I hand 99% of my commute over to it. I care just as much about safety and follow the rules just as strictly, yet my trust in autonomous driving has multiplied many times over. No keynote demo can substitute for hundreds upon hundreds of uneventful drives home.

To be sure, it has given me two real scares over those two years. Once, at an intersection, three traffic lights were stacked vertically on a single pole, governing straight traffic and left turns separately. I was waiting in the left-turn lane when the middle green light lit up for the through lanes; FSD missed the distinction, mistook the green for its own, and lurched forward. The second time was in an office parking garage: the entry gate arm had not finished rising, yet the car rolled right up to the bar without slowing down, slamming on the brakes only at the last possible instant. I stomped on the pedal at the exact same moment. To this day, I cannot tell whether my foot stopped the car, FSD braked in time, or the gate's sensor caught the hood and swung the arm up out of the way.

What surprised me afterward was that neither scare drove me off. And the reason is simple: both times it slipped up, I was right there watching, with my hands and feet ready to take over in a heartbeat. Better yet, each close call taught me where its blind spots are: vertically stacked traffic lights and thin horizontal gate arms. Instead of trusting the system blindly, I now carry a mental map of where it is rock-solid and where I need to keep a closer eye on it. Good trust works like that: neither blind faith nor blanket rejection, but knowing the exact boundary of where one can rely on it.

Even so, much of my comfort with FSD is not trust in the purest sense, but reliance with my hands near the wheel. When I gaze out the window or rest my feet off the pedals, I am not far from the "Overtrust" branch in Figure 1. The real test is whether I would dare climb into the back seat and go to sleep. Of course I would not. Hundreds of uneventful commutes are enough to back up reliance while I am sitting in the driver's seat, but nowhere near enough to prove the car is safe while I am asleep. So the difference between FSD and an agent right now may lie less in which one I trust more than in what it costs to supervise it and stop it: when FSD slips up, I see it on the spot and can take over in a split second; when an agent slips up, I might not find out for days.

The Real Risk Is Not Making Mistakes

Nearly everyone who has used AI has been lied to with a straight face. A book that was never written, a link that leads nowhere, a quote no one ever said, all delivered in the exact same measured, confident tone the model uses when stating hard facts. The most famous debacle played out in a New York federal court in 2023. A lawyer used ChatGPT to draft a legal brief packed with case citations that neither opposing counsel nor the judge could find. When the court ordered him to produce the actual decisions, he went back and asked ChatGPT; the chatbot assured him the cases were real and could be found on LexisNexis and Westlaw. Every single one was made up, and the lawyer was fined $5,000.

Why do language models do this? OpenAI offered a refreshingly candid explanation last September: the way models are scored during training rewards guessing over admitting "I don't know." It is like a multiple-choice exam where a lucky guess wins points and a blank answer gets a zero; over time, the models that bluff the hardest rack up the highest benchmark scores. And the problem goes deeper than exam grading. OpenAI's own technical report on GPT-4 showed that the base pretrained model actually had a well-calibrated sense of what it knew and what it did not, only for post-training via human feedback (RLHF) to blunt that calibration; a Stanford study likewise found that human raters rarely pick hedged answers, teaching models to stop saying "I'm not sure." The silver lining is that this is a design choice made by model builders, and what is chosen can be changed.

Why, then, do humans keep falling for it? Psychologists have known for decades that the smoother a sentence reads and the more assured its delivery, the more likely we are to believe it is true. A study from the University of California, Irvine added another wrinkle: the longer an AI's answer, the more confident readers feel in it, even when the longer answer is no more accurate. And nothing comes more naturally to a large language model than sounding fluent, thorough, and unmistakably human.

That same quirk of human psychology explains why the internet is awash in half-baked takes, and why a creator who pairs them with an unflinching stare and a tone of absolute certainty can still build a following in the millions.

To my mind, though, the scariest thing is not that AI makes mistakes. Everyone makes mistakes. We have all sat across the dinner table from that one relative who claims to know everything from astrophysics to geopolitics and has never once said the words "I don't know." Before long, everyone at the table learns to discount everything he says, because he sounds just as certain when he is dead wrong as when he is right. AI has the exact same problem: its blunders look identical to its breakthroughs. The signal we rely on most to calibrate trust gets wiped out. Fortunately, that same UC Irvine study pointed to a way out: when an AI hedges its answer with a phrase like "I'm not entirely sure," people adjust their confidence accordingly. As long as the user can eventually verify whether an answer was right, knowing when to say "I don't know" turns out to be a higher form of credibility.

Who Hits the Final Button?

When I wrote about Muse in APoV 001, I argued that consumer agents are not really competing on raw intelligence; they are competing for user trust. A generational lead in model capability can be erased in one to three months, turning raw smarts into a commodity. What remains scarce are two other things: whether an agent respects boundaries, and whether it is genuinely on my side. Neither can be proven on a leaderboard; both have to be built from the ground up, one interaction at a time.

The way I treat an agent looks a lot like how I break in a new intern. For the first few weeks, I edit every email he drafts before it goes out. Once he gets a few right, I let him book conference rooms and write up meeting notes on his own without checking in. For a high-stakes client email or a decision that spends money, he can tee everything up to the very last step, but before it goes out the door, it has to cross my desk. Trust climbs that ladder one rung at a time. For a machine, though, the ladder is steeper than for an intern: those same Wharton researchers found in an earlier study that people abandon an algorithm after watching it err far faster than they abandon a human who makes the exact same mistake. An agent has to plant its feet more firmly on every single rung than an intern ever would.

A well-designed agent product should build that ladder directly into the user experience, instead of demanding the master keys upfront. Instinct is a textbook example of what not to do. As TechCrunch noted, its terms of service demanded a "perpetual and irrevocable" license to user content, along with permission to enter into legally binding transactions on the user's behalf. Trying to vault straight to the penthouse before putting up the ladder, even setting aside its clunky interface, was the main reason I stayed away from the product.

Alongside the ladder, a black box is equally vital. Commercial aviation has a wonderful irony: the most famous "black box" on an airplane exists precisely to pry open a black box. Invented for crash investigations, flight recorders are painted bright blaze-orange so search crews can spot them immediately in the wreckage. Passengers dare to board a jetliner not because they understand turbine blades, but because they know every single failure is recorded, investigated, and written into the next revision of the safety manual. According to the International Air Transport Association (IATA), the global commercial aviation accident rate dropped from 3.72 per million flights in 2005 to 1.13 in 2024. Agents need a flight recorder of their own: what the agent read, what it sent on my behalf, and how much money it spent should all be logged cleanly so I can inspect the tape at any time. As for the agent's own written explanation of why it took an action, that can only be treated as a rough reference for now: a study published by Anthropic last year showed that when reasoning models used hints planted inside a prompt, they mostly left those hints out of their chain of thought. Visible effort can be staged, too. Harvard Business School researchers have shown that people rate a service more favorably simply because they see it working hard, regardless of the actual result. Precisely because what a model says and what an interface performs cannot be taken at face value, building a tamper-proof black box for agents has become one of the hottest startup categories in cybersecurity since 2025.

Even a flight recorder is not enough on its own; someone also has to backstop the loss. Two decades ago, consumers were just as reluctant to type a credit card number into an unfamiliar website. What eventually won them over was not the ability to watch the merchant pack the box, but a hard cap on how much they could lose: under US federal law, a cardholder's liability for unauthorized charges is capped at $50, most banks waive even that, and e-commerce platforms add buyer protection on top. Agents should work the same way. Instead of trying to talk me into trusting them more, they should shrink the risk I have to take, for example by transacting through a virtual card capped by dollar amount and locked to a single merchant. The first company that dares to promise "if our agent books the wrong thing, we cover it" will find user trust following close behind. Looking back at my hesitation over handing payment details to an agent on the night of that interview, perhaps the real question was never whether I should hand them over, but why the agent needed my raw card in the first place.

Another red line exists: an agent can make a mistake, but it cannot cross the line. Management scholars have long found that a lapse in competence can usually be repaired with an honest apology and a fix, whereas a lapse in integrity is almost impossible to recover from. If an agent books me the wrong restaurant, that is a competence problem; I will forgive it and move on. If it sends an email behind my back without asking, it has crossed a line, and I will kick it off my machine on the spot. Whenever an agent is unsure, it is far better off pausing to ask one extra question than making the call for the user.

The last question is the hardest of all: whose side is it actually on? In my piece on Muse, I noted that Meta has been weighing taking a cut of transactions completed through the agent. That immediately raises an uncomfortable question: when it picks a restaurant or a flight for me, is it optimizing for my wallet, or for its own commission? Until that conflict of interest is laid bare, no amount of model IQ will convince me to let an agent make calls where my own money is on the line.

Over the next few years, the gap between agents will be defined less and less by what they can do, and more and more by what we dare let them do. They will win first in work that is visible and reversible, such as drafting, research, and scheduling, before slowly earning their way into work that is invisible and irreversible: spending our money and speaking in our name.

To reach that second stage, however, the passenger-side brake eventually has to come off. A car that requires a human to stare at the road every second does not save much effort. As early as 2013, the Google self-driving project that later became Waymo discovered that test drivers in its highway-assist cars were nodding off, checking their phones, and putting on makeup behind the wheel, prompting the team to abandon the "human as backup" approach altogether. Once the brake comes off, trust can no longer rest on my keeping watch; it has to rest on institutional guardrails like black boxes, liability guarantees, and industry-wide safety statistics, just as no passenger boards an airliner today because they plan to keep an eye on the captain. Waymo has already shown what those statistics look like: in a joint study with the reinsurer Swiss Re across 25.3 million driverless miles, Waymo's fleet generated 88% fewer property-damage claims than human drivers. FSD's safety numbers, by contrast, are still self-reported mainly by Tesla.

The Other Side of High Agency

Spend any time around Silicon Valley or the startup world today, and the highest compliment one can pay a founder or a colleague is "high agency": someone who does not wait to be told what to do, spots the problem on their own, drives the work end to end, and delivers. Yet as I was writing the paragraph above about the intern, it hit me that this definition tells only half the story. The more autonomous one is, and the more decisions made on other people's behalf, the more that person looks like an AI agent to the people around them. When a manager hands off a project and cannot see what the person running it is doing day to day, that manager is usually keeping a finger on a final confirm button of their own. Colleagues, clients, friends, and even family members size each other up the exact same way. The other half of high agency, then, is keeping the people who depend on you confident in what you are doing while the work is still in flight. Every rule we just laid out for trusting AI applies word for word to working with other human beings.

First, make work visible. Never be the person who disappears into a cave and only resurfaces at the deadline to hand in a finished product. Even if the final work is brilliant, right up until the moment it is delivered, that person is an invisible agent running silently in the background. Nothing makes people more anxious than what they cannot see. Sending quick progress updates, sharing rough drafts early, and walking people through your thinking does the exact same job as letting the driver watch FSD turn the wheel in real time. Just make sure what you share is real progress, not a performance of busyness.

Second, always leave a brake pedal on the passenger side. Instead of dropping a polished, take-it-or-leave-it plan on someone's desk, bring two or three options and invite them to mark it up. As that Wharton study showed, simply giving people a small say in the outcome makes them far more willing to get on board; how much they actually change barely matters. Humans do differ from machines in one crucial way, though: people read close monitoring as distrust and quietly dial back their effort in response, a phenomenon economists Armin Falk and Michael Kosfeld call the "hidden costs of control." So the brake offered to a human colleague should be a choice of direction, not surveillance.

Third, remember that trust is built in small deposits. Break big promises down into small deliverables, and hit them on time, one after another. When Google's Project Aristotle studied what makes teams effective, dependability came out near the very top of the list. Ten small promises kept on schedule are worth far more than one grand promise still hanging in the air.

Fourth, never be that know-it-all relative at the dinner table. When not sure about something, saying plainly, "I'm not certain; let me check," might sound like admitting weakness, but what it actually does is help the other person calibrate their trust in you. Someone who speaks with 100% certainty every time only has to be wrong once for their credibility to fall off a cliff; when one makes a habit of stating how sure they really are, people build a buffer into their expectations before anything goes wrong.

Finally, when mistakes do happen, know which kind it is. A slip in competence can almost always be mended with a straight apology and a concrete fix; the line on honesty and boundaries is one that should never be crossed even once. That is also why I put this line in the founding principles of APoV: I do not defend my past views. When I am wrong, I fix it, and corrections are always welcome.

Back to That Night

The lights of the South Bay brightened the pavement of Highway 101 as I neared home. Tomorrow morning, I will almost certainly click the scroll wheel and turn FSD back on out of sheer muscle memory. Only now I know why: I dare to put myself in its hands not because I understand how its neural network works, but because it drives right in front of my eyes, I can hit the brake at any moment, and it has brought me home safely hundreds of times. Looking back at how I work with AI agents, we are still feeling our way forward, building that ladder of trust one rung at a time. Right now, sitting in front of an agent, I am still very much that guy from two years ago in his friend's car, both hands locked tight around the wheel.