How Artificial Intelligence Is Revolutionizing Diagnosis in Health Care
AI is helping health care professionals during the pandemic by improving accuracy and patient safety in radiology and disease prediction. Despite concerns about job replacement and misuse, AI promises to enhance health care accessibility, affordability, and accuracy.
Trending Topics with Infection Control Today
Diagnostic errors1 are a long-standing, pervasive problem in medicine. The Agency for Healthcare Research and Quality (AHRQ) estimates that one in every 20 adults living in the United States has experienced a diagnostic error, errors that have also been implicated in 10% of all patient deaths.2 A recent study published in the British Medical Journal, Quality & Safety,3 estimates that in the United States, 795,000 patients suffer serious harm each year from diagnostic errors. The big 3 categories of diseases where these errors occurred were vascular events, infectious diseases, and cancers. There was an average medical error occurrence of 11% and a rate of patient harm of 4.4%.3
During the pandemic, the medical system was severely stressed and has not recovered. Many healthcare systems are evaluating the use of artificial intelligence (AI) to help mitigate problems of staffing shortages and diagnostic errors. AI systems are currently assisting human doctors and nurses in several major medical centers around the nation.4
When evaluating the effectiveness of artificial intelligence, it must be remembered that AI is not expected to be 100% accurate. Its performance is measured against the human counterpart. In addition, there are many versions and upgrades to artificial intelligence programs. Open AI’s Chat GPT-4’s performance is superior to its predecessor, Chat GPT-3.5 Many commercial AI programs use an even more advanced version, some of which are specialized for a particular task, such as reading X-rays or pathology slides.
Chat GPT-4 has outstanding performance on specialized tests regarding the accumulation of knowledge. For example, Chat GPT-4 easily outperformed humans on the SAT by scoring in the 93rd percentile in the reading and writing section and 89th percentile in the math section.5 On the Bar Exam, Chat GPT-4 scored in the 90th percentile, eclipsing the performance of Chat GPT -3.,5 which scored in the 10th percentile.6 Chat GPT-4 has obtained a passing score of 80.7% on a mock Radiology Board exam, surpassing the near passing score obtained by Chat GPT 3.5 of 69.3%.7
Chat GPT-3.5 responses were compared to physician responses to 195 randomly selected social media medical questions.8 The chatbot’s responses were judged to be superior to physicians 78.6% of the time. The chatbot’s responses were also judged to be more empathetic. Using physicians to measure empathy may be a low bar, but the chatbot’s performance was impressive. In a study using the records of 6 patients with a delay in diagnosis of over a month, Chat GPT-4 diagnosed 4 patients accurately, while clinicians only correctly diagnosed 2.9 In another study, Chat GPT-3.5 achieved a diagnostic accuracy of 71.7% in 36 published clinical diagnostic vignettes.10
Advances in imaging diagnosis are developing at a rapid pace. AIs can accurately differentiate between normal and abnormal chest X-rays. In a recent study, AI identified abnormal chest X-rays with a sensitivity of 99.1%, higher than the sensitivity of radiologist’s readings.11 AI may perform well in outpatient settings where potentially abnormal chest X-rays are identified for in-depth reading by radiologists. In another study, AI-supported reading of mammograms with a single radiologist was compared to double reading with two radiologists.12 AI-supported screenings detected 6.1 cancers per 1000 participants with a recall rate of 2.2%. Double reading by 2 radiologists had a cancer detection rate of 5.1 per 1000 participants with a recall rate of 2.0%. The false positive rate was 1.5% in both groups. In countries with an underdeveloped health care system, AIs may help fill a void in patient care. A recent Croatian news article touts the country’s new AI-enabled diagnostic capabilities, which will significantly aid their physicians in reviewing scans and making final treatment decisions.13 As stated in the article, “Croatian carcinoma diagnosis will now be among the best, but the weakest national point remains treatment.”
Artificial Intelligence is also used to predict the likelihood of a patient developing a dangerous illness. For example, different AIs can use clinical information and radiomics features in patients treated for prostate cancer to improve predictions of disease recurrence.14
AIs assisting the human nurse and clinician should be done one-to-one. Implementation where a single human has only a supervisory role over many AIs should be discouraged. For example, a system with 1 nurse overseeing10 AIs performing patient intake and triage should not be allowed. The recent Washington Post article titled “Hospital bosses love AI. Doctors and nurses are worried”4 is a warning of potential AI abuses. As stated in the article, there is concern that AI will become “an excuse for insurance and hospital administrators to cut staff in the name of innovation and efficiency.”
There is little doubt that AIs will be widely used to aid physicians in making diagnoses and nurses in making patient intake and triage decisions. Implementing AI promises to increase patient safety, accuracy, and efficiency. The latter, however, will mean fewer medical jobs for humans, but with the potential of patients having greater access to more affordable health care.
References
1. Outpatient Diagnostic Errors Affect 1 in 20 U.S. Adults, AHRQ Study Finds. Agency for Healthcare Research and Quality; April 16, 2014, 2014. https://archive.ahrq.gov/news/newsroom/press-releases/2014/diagnostic_errors.html
2. AHRQ. Diagnostic Safety and Quality. Agency for Healthcare Research and Quality. https://www.ahrq.gov/topics/diagnostic-safety-and-quality.html#accordions
3. Newman-Toker DE, Nassery N, Schaffer AC, et al. Burden of serious harms from diagnostic error in the USA. BMJ Qual Saf. Jul 17 2023;doi:10.1136/bmjqs-2021-014130
4. Verma P. Hospital bosses love AI. Doctors and nurses are worried. Washington Post. August 10, 2023. https://www.washingtonpost.com/technology/2023/08/10/ai-chatbots-hospital-technology/
5. Varanasi L. GPT-4 can ace the bar, but it only has a decent chance of passing the CFA exams. Here's a list of difficult exams the ChatGPT and GPT-4 have passed. Business Insider. Nov 5, 2023. https://www.businessinsider.com/list-here-are-the-exams-chatgpt-has-passed-so-far-2023-1
6. GPT-4 Passes the Bar Exam. Illinois Tech; March 15, 2023, 3023. https://www.iit.edu/news/gpt-4-passes-bar-exam
7. DePeau-Wilson M. ChatGPT Passes Board Exam. MedPage Today. May 16, 2023. https://www.medpagetoday.com/special-reports/features/104524
8. Ayers JW, Poliak A, Dredze M, et al. Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Intern Med. Jun 1 2023;183(6):589-596. doi:10.1001/jamainternmed.2023.1838
9. Shea YF, Lee CMY, Ip WCT, Luk DWA, Wong SSW. Use of GPT-4 to Analyze Medical Records of Patients With Extensive Investigations and Delayed Diagnosis. JAMA Netw Open. Aug 1 2023;6(8):e2325000. doi:10.1001/jamanetworkopen.2023.25000
10. Rao A, Pang M, Kim J, et al. Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study. J Med Internet Res. Aug 22 2023;25:e48659. doi:10.2196/48659
11. AI Accurately Identifies Normal and Abnormal Chest X-rays. RSNA Press Release; March 7, 2023, 2023. https://press.rsna.org/timssnet/media/pressreleases/14_pr_target.cfm?ID=2420
12. Lang K, Josefsson V, Larsson AM, et al. Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study. Lancet Oncol. Aug 2023;24(8):936-944. doi:10.1016/S1470-2045(23)00298-X
13. Simmonds L. Croatian carcinoma diagnosis using AI begins. Total Croatia 2023.
14. Wang YD, Huang CP, Yang YR, et al. Machine Learning and Radiomics of Bone Scintigraphy: Their Role in Predicting Recurrence of Localized or Locally Advanced Prostate Cancer. Diagnostics (Basel). Nov 3 2023;13(21)doi:10.3390/diagnostics13213380
