Understanding the Unique Risks Associated with Generative AI Through the Newly Released Profile and Suggested Management Actions

Generative Artificial Intelligence (AI) is a technology that is growing fast and changing many fields like healthcare. In the United States, medical practice administrators, owners, and IT managers want to know how AI can help with running operations better, talking with patients, and making paperwork easier. But generative AI also brings new or bigger risks compared to older AI methods. It is important for healthcare groups to understand these risks and how to handle them, so they can use AI while keeping patient information safe and following the rules.

This article looks at the latest advice from the National Institute of Standards and Technology (NIST) about managing risks for generative AI systems. This advice includes the final version of the Generative Artificial Intelligence Profile (GenAI Profile), released on July 26, 2024. This profile adds to the existing AI Risk Management Framework (AI RMF). The article shares ideas that apply to healthcare and thoughts on AI use and workflow automation in medical offices.

The NIST AI Risk Management Framework and the Generative AI Profile

The AI Risk Management Framework (AI RMF), released by NIST in January 2023, is a voluntary guide to help groups find, study, and handle risks from AI technologies. The AI RMF covers different AI uses, but generative AI—systems that can make text, images, sounds, or other media—has some special risks. Because of this, the Generative Artificial Intelligence Profile (GenAI Profile) was created and published in July 2024 as an extension of the original framework.

The GenAI Profile focuses on risks that are special to generative AI, like:

• Confabulation or “hallucinations”: Generative AI machines can create wrong or made-up information.
• Large-scale misinformation: These systems can quickly make and spread false or confusing content.
• Data privacy and unauthorized data use: Because these models train on large data sets, there is a risk of leaking sensitive or personal data.
• Intellectual property infringement: Generative AI might reuse protected material without permission.
• Production of offensive or illegal content: This includes deepfake videos, images shared without consent, or harmful words.
• Environmental impacts: Training big generative language models uses a lot of energy. One model might produce as much carbon as 300 roundtrip flights between San Francisco and New York.
• Access to hazardous information: It can lower the barriers for bad actors to get data about chemical, biological, radiological, or nuclear (CBRN) weapons.

These risks matter a lot to healthcare groups in the U.S. because patient data privacy is protected by laws like the Health Insurance Portability and Accountability Act (HIPAA). It is very important that generative AI does not leak sensitive health data or give wrong medical advice. This is key for both healthcare providers and office staff.

Risk Management Actions Recommended by NIST

NIST’s GenAI Profile suggests more than 400 actions to lower risks related to generative AI. These actions are organized around four main functions introduced in the original AI RMF:

• Govern
• Map
• Measure
• Manage

Each function is an important step to include risk management when developing and using generative AI.

Govern
Governance means setting rules, clear responsibilities, and oversight. For healthcare leaders, this means creating AI ethics groups or risk teams to watch over AI use. They must make sure AI use follows HIPAA and require checks when buying generative AI tools, especially if they come from outside vendors. NIST points out the need for contracts and proof of standards from suppliers of AI models or data. Since patient information is sensitive, medical offices need strong plans to check and buy from vendors.

Map
Mapping means identifying where and how generative AI is used in an organization and understanding how data flows. In healthcare, managers should record where AI models are applied, such as patient communication, scheduling appointments, or answering calls automatically. Knowing exactly how data moves helps find spots where sensitive info might be exposed or changed by mistake.

Measure
Measuring means checking the quality, correctness, and safety of AI output. Since generative AI can create wrong or false content (called hallucinations), healthcare providers should do strong testing before full use. This testing can include AI red teaming or simulations that act like attacks. The tests should involve different people, like clinical staff, IT teams, and legal experts, to check if AI is reliable when handling patient data or calls.

Manage
Managing means watching AI constantly, having plans for incidents, and ways to fix problems. NIST recommends making formal plans to report problems with AI or data breaches. Healthcare administrators and IT managers should report inaccurate AI results quickly so they can fix them fast and be open about what happened. Being open matches guidelines for public reports to better understand AI problems in healthcare.

The Challenges of Generative AI Governance in Healthcare

Generative AI tools often work with different types of data—text, voice, and images. This makes governance harder. NIST highlights the need for more human checks, tracking, records, and higher-level management to handle risks better. Some challenges include:

• Third-party model use: Many generative AI tools use external models trained on large data sets, which may include sensitive health data. If these tools are not checked well, they could expose protected health data or spread bias.
• Inadequate testing: Old testing methods might not work well for generative AI, since its output can change and be unpredictable. Field tests before full use and ongoing feedback are needed.
• Environmental costs: Medical offices should know the energy impact of using generative AI models and think about using energy-saving options when they can.

The report by Alexander Sisto and K.C. Halm points out that organizations following NIST’s risk steps may lower chances of legal issues linked to carelessness or intellectual property problems. This is important for healthcare groups under strict rules.

AI and Workflow Automations in Medical Practices

Generative AI and other AI tools play a big role in automating front-office work in medical practices. Companies like Simbo AI improve automation by adding smart calling services into healthcare workflows. These tools use AI to answer phone calls, set up appointments, and respond to patient questions without needing a person. For medical administrators and IT managers, this means:

• Improved efficiency: Automating phone tasks lowers staff workload, helps avoid missed calls, and makes patients happier.
• Accuracy and consistency: When AI systems are trained well and managed closely, they keep communication steady, lowering human errors in appointment scheduling or patient data handling.
• Data security: Technologies that follow NIST’s AI RMF build trust in phone automation by cutting risks of data leaks or wrong information.

Moreover, using generative AI in workflow automation needs careful governance to stop the creation of wrong medical info or wrong handling of patient data. Health IT teams must watch AI outputs constantly, keep data private, and enforce strong rules.

Using generative AI for front-office automation also changes work for staff. Front-office workers can focus more on complex tasks needing human judgment while AI handles simple, repeated jobs. This change needs staff training and good policies from administrators to make sure AI adds value without hurting care or breaking rules.

Relevance of NIST’s Guidance to U.S. Healthcare Organizations

The NIST Trustworthy and Responsible AI Resource Center, started in March 2023, helps healthcare groups by offering resources and examples that follow federal rules. For U.S. clinics and offices with several providers, these rules are important to use AI that meets both work needs and legal demands.

Because AI changes fast, having over 400 actions in the GenAI Profile gives organizations a clear way to spot generative AI risks and pick the right controls. This may include:

• Doing regular risk checks that include risks special to generative AI.
• Using open AI incident reports to find and fix unexpected AI problems.
• Setting clear governance groups that handle AI ethics, privacy, and security in healthcare.

Working together with government groups like NIST, private companies, and schools like Stanford’s Institute for Human-Centered AI helps make sure these guides stay current with AI progress and useful for real-world work.

Final Notes for Healthcare Practice Administrators, Owners, and IT Managers

Medical practices thinking about using generative AI can find full and practical guidance in the NIST AI RMF and its Generative AI Profile to handle the risks carefully. Using these frameworks can keep patient privacy safe, make operations more reliable, and ensure healthcare rules are followed.

But choosing to use generative AI is not just a tech decision. It needs leaders to commit to good governance, work across teams, and watch AI use all the time. Staff need training, vendors must be checked well, and clear communication about what AI can and cannot do is important. This way, benefits grow and risks shrink.

As healthcare in the United States moves more toward digital and automatic processes, administrators and IT managers should stay updated on AI risk management and follow best practices from trusted sources like NIST. This will help safely change front-office work and lead to better patient care and smoother clinic work.

By knowing the specific challenges of generative AI and following detailed management steps in the NIST guides, healthcare groups can move forward with AI tools such as Simbo AI’s automated phone answering services. These tools can support both innovation and keeping patients safe.