OpenAI’s most recent iteration of its expansive language model, ChatGPT 4.0, recently underwent a clinical neurology assessment, achieving an 85% accuracy rate in a proof-of-concept study. The study’s authors foresee promising potential for ChatGPTs in the realm of clinical neurology after some necessary adjustments.
The outcomes of this study, conducted by researchers from the University Hospital Heidelberg and the German Cancer Research Center in Heidelberg, were disclosed on December 7th. The examination carried out on May 31st featured two versions of the language model: ChatGPT 3.5 and its successor, ChatGPT 4.0.
The researchers utilized a compilation of questions from the American Board of Psychiatry and Neurology’s neurology exam, along with a smaller set of queries from the European Board for Neurology. ChatGPT 3.5 scored 66.8%, accurately answering 1306 out of 1956 questions, while the newer iteration, ChatGPT 4.0, achieved an 85% accuracy rate with 1662 correct answers. In comparison, the average human score stood at 73.8%. ChatGPT 4.0 outperformed human participants in questions related to behavioral, cognitive, and psychological aspects, effectively passing the neurology exam, given that a 70% accuracy rate is typically considered a passing grade in educational settings.
However, both versions of the model exhibited lower performance in tasks demanding higher-order thinking compared to those necessitating lower-order thinking.
The researchers leading the experiment view these results as an endorsement for the potential utilization of large language models in clinical neurology following further enhancements: These findings suggest that, with further refinements, large language models could have significant applications in clinical neurology.
Nonetheless, they caution that reservations persist. Despite the potential for implementing these models in documentation and decision-making support systems within neurology, their use in practical applications should be approached with caution, particularly in tasks involving high-order cognitive functions. AI has already been involved in significant healthcare tasks, such as aiding AstraZeneca in cancer research or combating antibiotic over prescription in Hong Kong.
Image by freepik
Disclosure Statement: Miami Crypto does not take any external funding, or support to bring crypto news to the readers. We do not have any conflicts of interest while writing news stories on Miami Crypto.