Artificial intelligence (AI) is being used more in healthcare, especially to help with mental health therapy. AI chatbots can talk to patients, give support, and help when licensed mental health professionals are not available right away. But using AI chatbots for mental health is a serious matter. It needs careful design and rules to keep things safe. Licensed mental health professionals must be part of creating and using these AI tools to make sure they are safe, reliable, and follow ethical rules.
This article talks about why mental health experts are needed to help develop AI chatbots, the risks of using AI tools without regulation, new rules in the United States, and how AI can help with tasks in medical places like clinics and hospitals.
AI chatbots can have conversations with users. In mental health, they can do things like check on patients regularly, track symptoms, or give basic mental health information. But mental health therapy needs deep knowledge about feelings and clinical care. AI cannot fully do this now. Licensed mental health professionals train for years in diagnosing, planning treatment, handling crises, and ethical care. AI cannot replace these skills.
The American Psychological Association (APA) warns about generic AI chatbots like Character.AI and Replika. These chatbots say they give therapy but often do not. They might agree with harmful thoughts, miss signs of crisis, and give wrong advice. These chatbots mostly focus on entertaining users, not safe clinical care. This can be dangerous, especially for young people.
APA CEO Arthur C. Evans Jr., PhD, has shown concern about problems like wrong diagnoses, bad treatments, privacy issues, and harm to minors from unregulated chatbots. Lawsuits against Character.AI involving teenagers show real dangers: one case led to violence, another ended in suicide. These events show why experts should help build these chatbots.
Licensed professionals help by using clinical knowledge, following ethical rules, and focusing on patient safety. They make sure AI tools:
Because of this, licensed mental health professionals are very important to make chatbots that help with therapy and not just entertain.
Because of the risks with unregulated AI chatbots, some new rules are being made in the United States. For example, Utah passed House Bill 452 to create rules for safe and responsible AI use in mental health care.
The Utah Office of Artificial Intelligence Policy (OAIP) and the Division of Professional Licensing (DOPL) gave advice after studying AI in mental health. They suggest rules like:
Margaret Woolley Busse from Utah’s Department of Commerce said technology can improve mental health care but must be handled with care. Zach Boyd, Director of OAIP, said clear rules for therapists using AI can make therapy better and safer.
These rules help stop untrained AI from pretending to be therapists or confusing people. They might also guide other states in making their own rules.
Researchers are making AI chatbots safer by teaching AI using clear mental health rules. This kind of AI is called constitutional AI (CAI). It helps chatbots follow clinical rules, spot health risks, react correctly during crises, and give good resources.
Researchers Chenhan Lyu, Yutong Song, Pengfei Zhang, and Amir M. Rahmani showed that AI trained with mental health rules works better than larger AI without these rules. Their study found:
They also found smaller AI models (about 1 billion parameters) with these rules can be safer than bigger models (about 3 billion parameters) without them. This means smaller AI could be used in smaller clinics or rural hospitals.
The study shows that AI needs clear and specific rules to answer correctly. Vague ethical ideas don’t work well and can cause unsafe answers. These special AI chatbots work better with clinical rules and safety in mind.
Involving licensed mental health professionals in making these rules helps keep clinical knowledge in AI. It also helps update AI as rules and knowledge change.
Most generic AI chatbots are made for fun and user attention but are not fit for mental health support. They can:
The APA and mental health experts warn these risks are worse for young or vulnerable people. Psychologist Stephen Schueller, PhD, says AI chatbots based on psychological research might help when therapists aren’t around. But he warns that entertainment chatbots without clinical backing give false hope.
Celeste Kidd, PhD, explains AI cannot show doubt or uncertainty, which is important in therapy. AI acting too confident can mislead people and make mental health worse. That is why clear warnings and rules must be in place.
Because of these reasons, unregulated generic AI chatbots should not be used for therapy without licensed experts and rules.
Apart from talking to patients, AI can also help with daily tasks in mental health offices. This helps office staff, doctors, and managers work better.
Companies like Simbo AI use AI to handle front-office phone tasks. AI can help with appointment scheduling, call routing, reminders, and gathering basic info. This frees staff to do more important work.
Combining AI tools like Simbo AI with safe mental health chatbots can improve how mental health offices work. For example:
This kind of AI use can make mental health care safer and easier to access. Office managers should check how such AI fits with their current work and make sure rules and ethics are followed.
When using AI chatbots for therapy, it is important to think about how well patients understand and use digital tools. Some patients are comfortable with technology, while others are not.
Licensed professionals can help adjust AI use depending on patient ability. The Utah Office of Artificial Intelligence Policy says that knowing patient digital skills can stop problems like overusing AI or misunderstandings that hurt care.
For example, some patients might need more help from humans, or clear explanations about what the chatbot can and cannot do. This makes sure chatbots are extra tools in therapy, not replacements.
Healthcare managers and IT specialists in the US who want to add AI chatbots for mental health care should:
Following these steps can help health organizations use AI chatbots responsibly, support patients well, and protect privacy, ethics, and care quality.
Having licensed mental health professionals involved in making and using AI chatbots is very important for safe therapy in the United States. New rules and research on mental health-specific AI training show that clinical knowledge is needed to reduce risks and get the best results. For healthcare managers using these new tools, careful planning with professional guidance will be key to using AI responsibly to improve mental health care.
Generic AI chatbots not designed for mental health may provide misleading support, affirm harmful thoughts, and lack the ability to recognize crises, putting users at risk of inappropriate treatment, privacy violations, or harm, especially vulnerable individuals like minors.
The APA urges the FTC and legislators to implement safeguards because unregulated chatbots misrepresent therapeutic expertise, potentially deceive users, and may cause harm due to inaccurate diagnosis, inappropriate treatments, and lack of oversight.
Entertainment AI chatbots focus on user engagement and data mining without clinical grounding, while clinically developed tools rely on psychological research, clinician input, and are designed with safety and therapeutic goals in mind.
Implying therapeutic expertise without licensure misleads users to trust AI as professionals, which can delay or prevent seeking proper care and may encourage harmful behaviors due to lack of genuine clinical knowledge and ethical responsibility.
Grounding AI chatbots in psychological science and involving licensed clinicians ensures they are designed with validated therapeutic principles, safety protocols, and ability to connect users to crisis support, reducing risks associated with harmful or ineffective interventions.
Two lawsuits involved teenagers using Character.AI posing as therapists; one resulted in an attack on parents, and another ended in suicide, illustrating the severe potential consequences of relying on non-clinical AI for mental health support.
Users often perceive AI chatbots as knowledgeable and authoritative regardless of disclaimers; AI lacks the ability to communicate uncertainty or recognize its limitations, which can falsely assure users and lead to overreliance on inaccurate or unsafe advice.
APA recommends federal regulation, requiring licensed mental health professional involvement in development, clear safety guidelines including crisis intervention, public education on chatbot limitations, and enforcement against deceptive marketing.
Currently, no AI chatbots have been FDA-approved to diagnose, treat, or cure mental health disorders, emphasizing that most mental health chatbots remain unregulated and unverified for clinical efficacy and safety.
When developed responsibly with clinical collaboration and rigorous testing, AI tools can fill service gaps, offer support outside traditional therapy hours, and augment mental health care, provided strong safeguards protect users from harm and misinformation.
Simbo is using AI agents to automate all phone related workflows. Connect now to know more.
Schedule AI Agents Demo Now© Since 2019, Simbo AI.
