Summary: Researchers compared the clinical correctness of GPT-4 based ChatGPT and pharmacists using 150 brain tumor MRI reports. ChatGPT achieved 73 % accuracy, slightly outperforming neuroradiologists ( 72 % ) and general radiologists ( 68 % ).
The AI model’s accuracy was highest ( 80 % ) when interpreting reports written by neuroradiologists, suggesting its potential in supporting medical diagnoses. This study highlights the growing part of AI in imaging and its potential for reducing doctor workload and improving medical accuracy.
Important Facts:
- ChatGPT’s diagnostic accuracy was 73 %, slightly higher than pharmacists.
- Its precision was 80 % when using neuroradiologist-written information.
- According to the study, AI might aid in increasing imaging clinical efficiency.
Origin: Osaka Metropolitan University
As artificial intelligence develops, its applications and features in practical situations may actually surpass those of humans.
In the field of radiology, where a proper diagnosis is crucial to ensure correct patient care, big language models, such as ChatGPT, may increase accuracy or at least offer a great following opinion.
In order to compare the clinical performance of GPT-4 based ChatGPT and pharmacists on 150 postoperative brain tumor MRI reports, grad student Yasuhito Mitsuyama and Associate Professor Daiju Ueda’s staff at Osaka Metropolitan University’s Graduate School of Medicine led the researchers to test its potential.
Based on these normal scientific information written in Japanese, ChatGPT, two board-certified neuroradiologists, and three public radiologists were asked to give variable diagnoses and a final examination.
Consequently, their reliability was calculated based on the actual treatment of the lesion after its removal.
The results stood at 73 % for ChatGPT, a 72 % average for neuroradiologists, and 68 % average for general radiologists.
Also, ChatGPT’s last examination precision varied depending on whether the medical report was written by a neuroradiologist or a public radiologist.
Compared to the 60 % accuracy of basic doctor reports, the exactness with neuroradiologist reviews was 60 %.
These findings support the use of ChatGPT for the postoperative MRI examination of brain tumors, according to graduate student Mitsuyama.
” In the future, we intend to study huge language models in different medical imaging fields with the aims of reducing the burden on physicians, improving diagnostic precision, and using Artificial to support academic conditions”.
About this research being done on brain cancer and AI.
Author: Yung-Hsiang Kao
Source: Osaka Metropolitan University
Contact: Yung-Hsiang Kao – Osaka Metropolitan University
Image: The image is credited to Neuroscience News
Original Research: Open access.
Yasuhito Mitsuyama and colleagues ‘” Comparative analysis of GPT-4-based ChatGPT’s diagnostic performance with radiologists using real-world radiology reports of brain tumors” is the title of their paper. European Radiology
Abstract
Comparative analysis of the GPT-4-based ChatGPT’s diagnostic performance with radiologists using actual-world radiology reports of brain tumors
Objectives
Large-scale language models like GPT-4 have demonstrated potential for radiology diagnosis. Previous studies that looked at this potential primarily incorporated quizzes from academic journals. This study aimed to compare the diagnostic capabilities of the GPT-4-based Chat Generative Pre-trained Transformer ( ChatGPT ) with that of neuroradiologists and general radiologists and use real-world clinical radiology reports of brain tumors.
Methods
From January 2017 to December 2021, we collected brain MRI reports from preoperative brain tumor patients at two institutions. The MRI reports were translated into English by radiologists. The same textual findings from the reports were presented to GPT-4 and five radiologists, who were then asked to make differential and definitive diagnoses. The excised tumor’s pathological diagnosis provided the foundation for the story. McNemar’s test and Fisher’s exact test were used for statistical analysis.
Results
In a study analyzing 150 radiological reports, GPT-4 achieved a final diagnostic accuracy of 73 %, while radiologists ‘ accuracy ranged from 65 to 79 %. Compared to the final diagnostic accuracy of GPT-4 using reports from general radiologists, which was higher at 80 % and 60 %, respectively. In the realm of differential diagnoses, GPT-4’s accuracy was 94 %, while radiologists ‘ fell between 73 and 89 %. Notably, for these differential diagnoses, GPT-4’s accuracy remained consistent whether reports were from neuroradiologists or general radiologists.
Conclusion
GPT-4 exhibited good diagnostic capability, comparable to neuroradiologists in differentiating brain tumors from MRI reports. GPT-4 can serve as a guide for general radiologists and residents as well as a second opinion for neuroradiologists regarding final diagnoses.
Clinical relevance statement
This study compared the diagnostic abilities of the GPT-4-based ChatGPT using real-world clinical MRI reports from brain tumor cases, finding that its accuracy in interpreting brain tumors from MRI findings is comparable to that of radiologists.
