In a recent healthsystemCIO.com webinar, Hartford HealthCare's Joel Vengco and Dr. Barry Stein joined Vitea CEO Shantanu Nigam to discuss a hard truth about AI governance for healthcare: approving a tool is no longer the same as governing it.
Hartford HealthCare likes to move fast with AI. It also insists on knowing exactly what that AI is doing once it's running.
That balance was the focus of "Accounting for the AI Already Inside Your Health System," a webinar moderated by Anthony Guerra, founder and editor of healthsystemCIO.com. The panel featured Joel Vengco, SVP and Chief Information and Digital Officer at Hartford HealthCare; Barry Stein, MD, MBA, SVP, Chief Clinical Innovation Officer and CMIO at Hartford HealthCare; and Shantanu Nigam, co-founder and CEO of Vitea.
Vengco framed Hartford's philosophy with a racing analogy. "We're an organization that loves to go fast," he said. "But in going fast, we know we also have to be thoughtful and focused on safety." Ask Formula 1 drivers what gives them the confidence to drive at top speed, he noted, and they'll tell you it's knowing they have great brakes.
Throughout the discussion, the panel kept returning to one idea: health systems have built rigorous processes for deciding which AI to let in, but far less for overseeing it after go-live. Below are the key insights from the conversation. You can watch the full webinar here:
For most of its history, healthcare AI behaved predictably. Nigam pointed out that neural networks and random forest models have been in use for decades. But until roughly four years ago, most applications were "more deterministic than not."
Generative AI changed that. "An application behaving in a certain way today doesn't mean it's going to behave in the exact same way for the same kind of stimuli a month later or six months later," Nigam said. These tools take context into account, he explained, and can even write queries on the fly that reach data they were never meant to touch.
Vengco said Hartford saw this firsthand, even with tools it built itself. The team tested a retrieval-based assistant on its website to answer consumer questions, along with an internal benefits agent for employees. "Even if we develop it ourselves, you still don't know how it's going to interact every single time," he said. "It changes the way that it responds to even a prompt."
That unpredictability is why legacy controls fall short. Security tools like data loss prevention still do their part, Nigam said, but "the functional side of risk is completely exposed with the existing tools."
The takeaway: A one-time review assumes the tool you approved is the same tool you'll be running next quarter. With generative AI, that assumption no longer holds.
Stein offered the analogy that anchored much of the discussion. As a physician, he said, he thinks of AI agents, whether built in-house or brought in from vendors, as "humanoid." Their arrival looks a lot like onboarding new clinicians.
"We're bringing in new residents, new fellows, new attending physicians into the clinical environment. We check out their credentials, they're trained well, they check all the boxes, then they go through the privileging process," he said. "No one, no matter how well you credentialed and privileged, would basically say, 'Off you go and do what the heck you want,' unsupervised, unobserved."
Now picture thousands of credentialed and privileged agents entering the environment. "What happens afterwards?" Stein asked. "You can build an agent today to do one thing, and an hour later it can be doing something potentially very harmful."
Nigam put the gap in sharper terms:
"We test AI like it's a new drug," he said. "And then we monitor it like a fax machine. Nobody looks at it until something really falls apart."
The organizations handling this well, Nigam said, have changed what they approve. "You're not approving the app, you're approving the usage of the app," he said. "When you approve an app, you still do not know all the things that app can do. Even if it's not doing it today, it can still do that thing tomorrow." The most advanced health systems pair that usage approval with enforced boundaries on what each application is allowed to do.
The takeaway: Intake is the credentialing step. Governance also needs the ongoing supervision that follows it.
When leaders think about AI risk in healthcare, they often picture unsanctioned tools slipping in. Nigam argued the bigger exposure may already be on the approved list. "The riskiest AI is probably not even what you're looking out for in the market," he said. "It might be an app you approved two years back. It just got an upgrade. Now it's got an LLM behind it doing something."
A visibility gap compounds the problem. Nigam cited a recent CHIME survey in which only 10% of members said they could detect shadow AI. Across the health systems Vitea has assessed, he said, teams have found 160 to more than 300 AI applications in use that nobody knew about, and the network and the official inventory have disagreed every time. Vengco said Hartford's own count is close to that range, at nearly 300.
When it was Nigam's turn to question his co-panelists, he tested this directly: What happens when a vendor tells you it's swapping the model underneath a tool you've already approved?
Vengco said Hartford is revisiting its vendor contracts so that any decision to add AI to an existing solution, or to change the model or use case behind it, comes back to Hartford for review. "The risk changes," he said, adding that a model change may be even riskier than a change in usage. Hartford's test is straightforward: "Does it match the safety and quality that we had initially seen as we approved it in the first round of governance and review?"
Stein added two points. Contracting and supply chain matter, because partners need to understand the consequences of not disclosing changes. And the industry still hasn't agreed on what counts as a model change.
Once Hartford learns of a change, the model goes back through intake. "Are you credentialed? Are you privileged?" Stein said. Then the team defines performance boundaries and monitors against them. "If you skip any of those steps, something is going to happen that's going to surprise you."
The takeaway: Approved tools need continuous monitoring, plus contract language that requires vendors to disclose AI changes.
Hartford's leaders reached a conclusion that shaped their entire approach: people alone can't supervise AI at this scale.
"If we want to unlock the potential of agentic AI and generative AI, both in the operational and clinical settings, it's not a human problem alone that we can solve for," Stein said. Vengco described the same limit from the operations side: "You could have a couple of folks reviewing those conversations, but that's hard to do when there's hundreds and thousands of conversations."
Nigam argued that putting humans in the monitoring seat cancels out the reason for adopting AI. "If there are too many humans doing it, then you've defeated all the value you're going to get from AI," he said. "You might as well have humans do it."
Timing matters, too. When an agent breaks a policy, whether in what it writes, how it interacts with patients or how it behaves with other agents, the action has to be blocked on the spot. "It's too late if it's done retrospectively," Nigam said. "Patients could be harmed."
Stein stressed that the monitoring layer must also be independent of the vendors being monitored. "Every single vendor and partner will come in and say they have the best security, they have the best controls, they have the best quality and safety," he said. "We never, ever want to be reliant on other folks telling us how safe their products are without us being able to independently verify. Twenty-four seven, always on, not sporadically."
That's the role Vitea plays at Hartford. Vengco said the organization needed a control to continuously monitor its generative AI and apply Hartford's own policies and guardrails "at every point of contact," and that need is what led to the partnership with Vitea.
Stein was direct about the impact: "As we've unlocked the potential with our patient-facing agents, we could not do it as safely without Vitea." He added that his team continually learns from the feedback the AI governance platform for healthcare surfaces.
The takeaway: Effective AI oversight runs continuously, sits independent of vendors and enforces policy in real time, as each interaction happens.
Despite the risks, no one on the panel argued for shutting AI down.
Hartford considered it early on. "We had the discussion of, ‘should we just shut everything down?’" Vengco said. "Imagine people shutting off access to the internet. The kind of innovation that you stifle, and just even the thinking that you might stifle." Instead, Hartford sent a memo to the enterprise explaining where it stood, rolled out provisioned AI platforms for staff and built guardrails designed to keep governance flexible rather than bureaucratic.
Nigam said unsanctioned use tells leaders something worth hearing. "Shadow AI is a demand signal. It's not a discipline problem," he said. "You can't just find shadow AI and block it, because that's not going to solve the problem." The goal is to find it, understand the need behind it, and then make sure the AI you keep stays within its bounds.
Stein added that governance only works when the whole organization owns it. "Risk is everybody's responsibility," he said. "It's not one person's responsibility, it's not one committee's responsibility, it's a culture."
Hartford deputizes the end users closest to each AI tool to take part in post-deployment performance monitoring, in partnership with Vitea, and it invests heavily in education. Stein compared that education to prescribing: "I'm not going to prescribe a drug that I don't understand: the indications, the contraindications, the complications."
His closing thought for the audience: "If you can educate your colleagues, it's the most powerful tool to mitigate the risk for taking care of our patients."
The takeaway: Governance that blocks everything pushes AI underground. Governance that sets clear boundaries lets people keep using it safely.
Hartford's experience shows that moving fast with AI and governing it well aren't competing goals. As Vengco put it, having the controls are what let you go fast.
The panel's message for health system leaders was consistent. Treat approval as the first step of governance. Credential AI the way you would a new clinician, then supervise it continuously, independently and in real time, especially when a vendor changes what's under the hood.
Watch the full webinar to hear the complete conversation.
Vitea is the AI governance platform built specifically for healthcare. It enforces your AI policies in real time across every AI application in your environment, including the vendor upgrades you didn't see coming. Book a call with our team to see how it works.
.png)
.png)
.png)
.png)
