.png)
A person suspects something is wrong. Maybe they’ve been sleeping too much, or not at all. Maybe they can’t remember the last time they felt like themselves. They open their health plan’s app, answer a short questionnaire, and within seconds a recommendation appears on the screen — a referral, a class, an app to download. There's no voice on the line. No one to notice the non-verbal cues.
I’ve spent much of my career on the other end of that moment. What a screening form captures is what a patient is willing to type. What a clinician captures during a patient encounter is much richer and incorporates judgment and clinical decision-making.
That gap sits at the center of a complaint a healthcare workers’ union has now filed against Kaiser Permanente. But the lesson most people are drawing from it is the wrong one.
The National Union of Healthcare Workers alleges that Kaiser’s “E-Visit” tool screens patients seeking help for anxiety or depression and then automatically generates referrals and care recommendations without a licensed clinician reviewing the answers in real time. The allegation suggests the recommendations come back too fast for a human to be in the loop. It has taken the matter to California’s Department of Managed Health Care and to the U.S. Department of Labor.
Kaiser points out that the tool doesn’t diagnose anyone, doesn’t determine medical necessity, and doesn’t make clinical decisions. Those, Kaiser maintains, remain with licensed professionals. It notes that every screen offers a phone number to reach a live person and that the E-Visit is only one path to care, not the only path.
Regulators will sort out whose account is accurate. What interests me is how we determine where to draw the line. The two organizations interpret California’s SB 1120 — the Physicians Make Decisions Act — differently: software may supplement a clinician’s judgment, but it may not supplant it.
AI in healthcare began with lower-stakes administrative tasks before moving into clinical workflows: drafting notes, summarizing discharges, and streamlining documentation.
Now, those same capabilities are moving closer to clinical reasoning and decision-making: software that triages, routes, prioritizes, and recommends.
There is enormous pressure to accelerate that shift. Health systems are under intense financial and workforce strain, making the promise of greater efficiency and productivity difficult to ignore.
The risk is that the boundary can move without anyone explicitly deciding to move it. A vendor adds new functionality to a product already embedded in a workflow. A previously manual step becomes automated. Then another.
Over time, the cumulative effect can quietly shift more clinical judgment from people to machines without the organization making a deliberate decision that this is what it wants. That is drift — and drift is an actual hazard.
Let me be clear about something: AI, correctly implemented, will improve healthcare. The sheer volume of information and work, combined with staff shortages and mounting external regulatory and financial pressures, presents a unique opportunity to incorporate AI to improve patient care, staff satisfaction, and efficiency. I’m not arguing against that tide.
A well-governed agent, correctly implemented, can improve the consistency of a human clinician who may be exhausted, rushed, and facing a flood of patients during a flu season. At certain discrete tasks, that consistency is not a fantasy. A health system could study the evidence and reasonably conclude that, for a specific step, the machine is the safer bet.
Because adoption is inevitable and the push to increase efficiency never eases, drift can be overlooked and the boundary of what is done by the machine versus licensed practitioners blurs.
A system can and should decide, “We're going to support our clinical workforce with AI at this step, here's why, and here's who owns it.” What it should not do is arrive at that decision without intentionality and human judgment.
The difference between making a decision and drifting into one isn't philosophical. It’s operational, and it surfaces in at least three places.
It’s worth noticing that Kaiser’s own defense is, in the end, a governance argument; it points to an AI framework built with labor and a joint oversight committee.
So what does making a purposeful decision actually require? In my experience, it comes down to three disciplines.
First, you have to be able to see it. You cannot govern what you can’t see, and most health systems today couldn't produce an honest map of every place AI now touches a clinical decision — including the tools they never knowingly switched on.
Second, the policy has to be enforced, not only published. It has to be live at the point of care — an actual control that stops the system from crossing it — or it isn’t a policy, it’s a hope.
Third, governance isn't a one-time launch day event. Models get updated, vendors push changes, workflows evolve. What was a deliberate, well-bounded choice on day one becomes passive drift three months in if no one's watching it continuously. Intentionality has a half-life.
The decision you didn’t know you made
Somewhere upstream, a health system decided how much of the judgment in that moment would belong to a clinician and how much to a machine. The only question — the one the Kaiser complaint should make every executive ask about their own organization — is whether that decision was made on purpose, or whether it's simply what happened while everyone was busy.
Intentionality is not the enemy of AI in medicine. It is essential to incorporating AI at scale safely and effectively. The systems that sustainably automate will be the ones that have clearly delineated where the machine helps and where the human decides.
Murali Naidu, MD, is Chief Medical Officer of Vitea, the leading AI governance software for healthcare organizations.
.png)
.png)
.png)
