IBM Watson was once seen as a big AI project to help with cancer treatment choices. But it did not provide safe or useful advice. This happened because it used made-up data and did not have enough different real patient records. Since it was not trained on real cases, it gave wrong advice that might have hurt patients. After spending a lot of money, the project stopped.<\/p>\n
This failure shows why it is important to use good and complete data to teach AI in healthcare. Using only limited or fake data can cause wrong results. This may make doctors and patients lose trust.<\/p>\n
The Epic Sepsis Model is a tool used in some U.S. hospitals to find patients who might have sepsis, a serious infection. But in a study over 15 hospitals, the model missed 67% of patients with sepsis. It did not help improve treatments or patient results compared to hospitals that did not use it. This means the AI was not reliable enough to guide doctor decisions.<\/p>\n
At the start of the COVID-19 pandemic, many AI models were created to help detect infections from scan images. But many of these were trained with bad data. For example, one AI learned to spot how the patient was positioned, not how bad the infection was. This caused wrong predictions. This happened because the data was not correct or was labeled wrong. Such mistakes can delay correct diagnosis, lead to wrong treatments, and cause extra worry for patients.<\/p>\n
Apart from medical decisions, AI chatbots and virtual helpers have also had problems. Air Canada\u2019s AI chatbot gave wrong details about special fares for bereavement. This led to a court case and fines. McDonald\u2019s stopped its AI voice ordering pilot in 2024 after many order mistakes upset customers.<\/p>\n
Even though these examples are not in healthcare, they show how AI mistakes in talking to customers can break trust and cause money loss. Healthcare groups using AI to talk with patients must make sure it is correct and trustworthy to avoid the same problems.<\/p>\n
Amazon\u2019s AI tool for hiring, while not healthcare-related, showed a serious problem useful for health AI users. The tool learned to be unfair to women because it was taught on mostly male resumes from the past decade. It was stopped in 2018 after showing bias issues.<\/p>\n
For health providers, this warning shows the risk of using AI trained on incomplete or biased data. AI bias can cause unfair care or treatment of workers and patients. Ethical rules and fairness must come first when building and using AI.<\/p>\n
<\/p>\n