Why Do Health Systems Have Forty AI Pilots and No AI Capability?

Years ago I watched a Stripe dashboard cross a million dollars in a single day. I was running a direct-to-consumer company, I was thirty-something, and I remember thinking: we did it. We figured it out.

The product was bad.

Not catastrophically bad. Just bad enough that the whole thing was being held together by paid acquisition and the sheer velocity of money moving through it, and when the spend stopped, so did everything else. Scale hides problems. That's the lesson I paid seven figures to learn and I've been dining out on it ever since.

Health systems are learning the same lesson right now, except the currency isn't revenue. It's pilots.

What's actually broken about running dozens of AI pilots?

Pilots hide the same thing scale hides: that nobody owns the thing after it works. A recent Becker's piece on the AI operating model put it plainly, and I've been thinking about it for three days. Health systems don't need more AI pilots. They need a repeatable way to decide which opportunities to pursue, how to deploy them, who stays accountable after go-live, and whether any of it created value.

A pilot has a champion, a budget, and an end date. Capability has none of those things. It has an owner.

Why does every AI deployment stall at the handoff?

Because the pilot was scoped around proving the technology works, and nobody scoped the part where an actual human does this every Tuesday forever.

I've sat in rooms where an AI project delivered exactly what it promised and then died, because the moment it graduated from innovation to operations there was no operations team who'd agreed to take it. The nurses weren't consulted. The IT group inherited a system they'd never architected. The vendor relationship was owned by someone in strategy who'd moved on.

That's not a technology failure. That's an org chart failure wearing a technology costume.

Is an AI operating model just more governance theatre?

It can be, and often is. Governance committees that meet monthly to review a slide deck are how organizations feel accountable without being accountable.

What makes an operating model real is that it answers four questions with named people, not processes. Who decides this is worth doing. Who builds it. Who runs it on day four hundred. Who gets fired if it stops working. If your framework produces a RACI chart but can't answer the last one, you've built a ritual.

My daughter is ten. I told her once that I work for her, not the other way around, and she looked at me like I'd said something obvious and slightly stupid. Same principle applies to every system I've built since. The thing exists to serve the people using it. If your AI operating model exists to serve the committee, the committee will be the only thing it ever produces.

What should health systems build versus buy?

Buy the model. Own the workflow. Own the data.

The reason we built HANA as fully open source and self-hostable wasn't ideology, it was watching health systems get trapped in black box vendor relationships where they couldn't audit the decisions being made about their patients, couldn't move the data, and couldn't answer a compliance question without filing a ticket. If a system can't inspect what a model did on a specific call, it can't be accountable for it. And accountability that stops at the vendor boundary isn't accountability.

Our integration documentation is public for the same reason. You should be able to evaluate whether this fits your stack before anyone signs anything.

How do you know an AI deployment is creating value?

Pick the metric before you deploy, and make it a metric the organization already reports.

Not "AI interactions completed." Not a satisfaction score invented for the pilot. Something on an existing dashboard that a board member already looks at. Thirty-day readmission rate. No-show rate. Days in accounts receivable. Nurse hours on the phone.

The evidence here is genuinely encouraging when the intervention is designed around workflow rather than surveillance. A multi-site virtual nursing study in npj Digital Medicine across nine hospitals found 30-day emergency readmission rates of 3.7 percent for virtually assisted discharges against 13.3 percent for traditional ones, in matched cohorts. That's a workflow change with a technology component, not a technology looking for a workflow. We track the same category of before-and-after numbers in our case studies, and the deployments that move real dashboard metrics are always the ones where somebody redesigned the process first.

Key Takeaways

The pilot-to-capability gap isn't a funding problem or a talent problem. It's an ownership problem, and it shows up identically across every health system I've talked to this year: enthusiastic pilots, real results, and then a silence where the operational handoff was supposed to be.

If you're evaluating AI infrastructure right now, the most useful thing you can do is skip the technology comparison for a week and answer the ownership questions first. Who runs this in eighteen months. What existing metric will move. What happens to the data if the vendor disappears. Systems that answer those three before they buy tend to deploy once. Systems that don't tend to deploy forty times and build nothing.

FAQ

How many AI pilots should a health system run at once?

Fewer than it currently does. The constraint isn't budget, it's the number of operational owners you have available to take handoffs, which in most systems is a single-digit number.

Should health systems build their own AI infrastructure?

Build the workflow layer, buy or self-host the model layer. Building foundation models is an enormous distraction. Owning how your organization's clinical work flows through them is not optional.

What's the fastest way to move from pilot to production?

Name the production owner before the pilot starts and give them veto power over the pilot design. It feels like friction. It's the only thing that reliably prevents the handoff from failing. If you want to think through what that looks like for patient outreach specifically, book time with me.