Challenges and strategies in adapting healthcare governance frameworks to the rapidly evolving landscape of artificial intelligence technologies for safe and effective implementation

In 2024, state legislatures reacted to the growth of AI in healthcare by introducing more than 100 AI-related bills across the country. Out of these, 20 bills were successfully passed. States like California, Colorado, and Utah have made important laws that require transparency when AI is used in clinical settings or other regulated jobs.
For example:

  • California requires doctors and healthcare organizations to clearly say when generative AI tools are used in clinical decision-making.
  • Colorado has consumer protection laws that make AI developers and users tell people the risks when AI tools are used in high-risk situations.
  • Utah demands notifications when generative AI impacts work in regulated jobs, including healthcare.

These laws show that more people want clarity on when AI is part of healthcare decisions so patients and providers can understand its role. Medical practice administrators must keep up with these state rules to follow them and keep patient trust.
The American Medical Association (AMA) has helped by recommending that any refusal of care or limits based on AI must be reviewed by a licensed doctor who is an expert in that field. This rule makes sure decisions are not just made by machines but also consider each patient’s situation.
The AMA also strongly supports doctors being part of talks about AI rules. Doctors provide useful medical views that help prevent harm caused by relying too much on AI systems.

Key Challenges in Healthcare AI Adoption and Governance

1. Rapid Pace of AI Development

AI tools change faster than usual healthcare devices. Unlike medical devices that take a long time to get approved and stay the same for years, AI software can update quickly. This makes it hard for healthcare groups to keep AI systems safe, effective, and following rules.
Justin Norden, MD, CEO of Qualified Health, said most healthcare places are not ready for this fast change. He suggested starting AI use in low-risk areas like handling claims before using AI for clinical decisions.

2. Balancing Innovation and Patient Safety

AI can help make diagnoses more accurate, personalize treatments, and speed up work processes. But there are worries that making decisions automatically without looking at each patient may cause more care denials or delays. For example, using AI to approve insurance claims or medical needs without a doctor’s review can stop people from getting care or lower care quality.
This problem needs governance rules that keep the “do no harm” principle in medicine.

3. Ethical, Privacy, and Bias Issues

AI can accidentally continue or even make discrimination worse if it learns from biased data. Healthcare serves many different groups, so ignoring bias in AI risks unfair results. Also, patient data used by AI must be well protected and explained clearly to patients.
These ethical and privacy problems are important for healthcare leaders when changing AI rules.

4. Integration with Existing Workflows and Systems

AI tools often have to connect with complex electronic health records (EHR) and other software. If AI results do not fit well with real work processes, it can confuse workers or make doctors trust AI less.
Healthcare IT managers need to focus on making AI work smoothly with current systems, be easy to use, and follow safety and privacy laws.

Strategies for Adapting Healthcare Governance to AI

Physician-Led Oversight and Collaboration

Any AI use in healthcare must include regular input from doctors, especially when AI helps or makes clinical decisions. Doctors understand the medical context, ethics, and how to check patients, which AI cannot do.
Policies should require that AI suggestions about patient care be reviewed and approved by licensed doctors who know the specialty before final decisions. This review follows AMA rules that stop fully automatic care denials without human checks.

Transparency and Informed Consent Practices

Healthcare organizations must clearly tell patients when AI helps with their care, diagnosis, or administrative tasks.
Transparency builds trust and lets patients give informed permission. This means saying when AI is used for things like reading images, predicting risks, or scheduling.
Following state transparency laws, like those in California and Utah, is needed for legal and ethical reasons.

Phased and Risk-Based AI Deployment

Because AI changes fast, healthcare groups should start by using AI in safer, simple tasks. Examples are automating admin work like claims processing or answering phones. When AI is shown to be reliable and safe, it can then be used in harder areas like clinical decisions but with tight rules.
This plan lowers harm during early AI use.

Continuous AI Monitoring and Audit Trails

Healthcare groups should keep strong systems to regularly check AI results for quality, bias, and rule-following.
Having this data helps hold AI accountable and find problems early. IT teams need to make sure AI systems produce detailed logs for review by regulators and internal teams.

Training and Education for Staff

Healthcare leaders and IT managers must make training about AI tools, their uses, and limits a priority for doctors and staff who work with AI.
Knowing what AI can do and its risks helps people stay careful and not blindly trust computer results.

AI and Workflow Integration in Healthcare Administration

Healthcare leaders should look at how AI can make workflows smoother while following changing rules.
Front-office automation like AI phone answering can help by improving patient contact and office efficiency.
Companies like Simbo AI work in this area by using AI to reduce paperwork and repetitive tasks.
Using AI for common tasks frees staff to focus more on patients. For example, AI chatbots and voice systems can schedule appointments, send reminders, and answer common questions. This cuts down on wait times and mistakes.
AI can also help billing and claims by checking data early and pointing out errors, matching advice to start AI in low-risk areas.
However, governance must make sure AI workflows keep patients’ right to talk to real people, especially in tricky or sensitive cases.
Automated systems should quickly pass calls or issues to humans or doctors to keep care quality and follow rules.
Also, tracking AI actions is important so admins can check accuracy, privacy, and fairness.
When AI is used the right way in healthcare work, leaders can ease workloads while keeping control and following laws. This balance is key today.

The Role of Policy and Industry Collaboration

State lawmakers, groups like the AMA, and AI makers have built guiding rules that focus on clarity, safety, and doctor oversight.
Good AI governance needs teamwork among:

  • Lawmakers who make rules balancing new ideas and public safety,
  • Doctors and care providers who check AI tools meet medical needs and ethics,
  • Healthcare leaders and IT managers who use policies and controls in practice,
  • AI developers who create clear, fair, and explainable AI systems.

The many new AI bills in states like New York, Texas, Virginia, and Illinois show that laws will keep changing.
Healthcare groups must watch these changes closely and update their governance and compliance plans early.

Summary: Actions for Medical Practice Administrators and IT Managers

  • Stay updated on federal and state AI healthcare laws, especially about transparency and doctor oversight.
  • Include licensed doctors in AI decision steps to review and OK any limits AI suggests on patient care.
  • Make clear rules telling patients when AI is involved in their care or admin processes.
  • Use a careful AI rollout starting with non-clinical tasks like claims handling and office automation before clinical uses.
  • Set up ongoing checks that review AI results often to find bias or mistakes.
  • Train staff on how AI works and its risks to keep human oversight strong.
  • Pick AI tools that fit well with electronic health records and admin workflows to avoid problems and help users accept them.
  • Work with vendors who know healthcare rules, like companies that offer AI front-office automation, to safely use new technology focused on patients.

By understanding the problems caused by quick AI changes and updating governance rules, healthcare groups can keep patients safe, follow laws, and work better.
This balanced method lets hospitals and clinics in the United States use AI tools well without losing ethical care or quality.

Frequently Asked Questions

What is the current legislative focus on AI in health care according to the AMA State Advocacy Summit?

State legislatures are actively introducing bills regulating AI in health care, focusing on transparency, regulation of payer use, discrimination prevention, and clinical decision-making oversight, reflecting the rapid legislative response to balance innovation with patient protections.

Why is transparency in AI use important in healthcare?

Transparency ensures that patients and healthcare providers are aware when AI tools are used, particularly in decision-making processes, allowing for accountability, informed consent, and safeguarding against misuse or over-reliance on automated systems without human oversight.

What role do physicians have regarding AI decision-making tools in healthcare?

Physicians must oversee AI-generated recommendations, especially those limiting or denying care. Any AI decision should be reviewed by a licensed physician in the relevant specialty before final determinations to ensure individual patient needs are considered.

How are states like California, Colorado, and Utah addressing healthcare AI transparency?

California mandates disclosure of generative AI use by physicians and organizations; Colorado imposes significant requirements on AI tool developers in high-risk situations; Utah requires disclosure when generative AI is used in regulated professions, including healthcare, emphasizing consumer protections.

What concerns does the AMA have about AI’s impact on healthcare access and outcomes?

The AMA worries AI may increase denials of medically necessary care, cause delays, and create access barriers by automating decisions without nuanced understanding of individual patient conditions, threatening quality and equity in healthcare delivery.

Why is the rapid evolution of AI technology challenging for healthcare?

Healthcare is unaccustomed to the fast pace of AI changes, unlike traditional medical tools approved once for long use. This rapid change demands continuous adaptation and governance, complicating safe, effective implementation in clinical settings.

What is the AMA’s vision for AI’s role in healthcare?

The AMA envisions AI as a tool that enhances patient experience and clinical outcomes, supporting physicians rather than burdening them, ensuring technology aligns with medical standards and ethical care delivery.

How does the AMA recommend handling automated denials of care by AI?

Automated denials should be automatically referred for review by a qualified physician who can assess medical necessity considering each patient’s unique circumstances before any final decision.

What proactive steps should healthcare organizations take with AI implementation?

Organizations should start by deploying AI for low-risk tasks like claims processing and quality reporting, allowing observation of AI behavior in less critical areas before expanding its clinical use.

Why is physician involvement critical in AI policy and governance in healthcare?

Inclusion of physicians ensures AI development and use maintains clinical relevance, addresses patient safety concerns, and balances technological innovation with ethical, individualized patient care requirements.