Medicine & AI to Improve Outcomes by Discovering Undiagnosed Patients- NHS Clinicians Presenting at HIMSS

Predicting Length-of-Stay & Risk of Mortality for ICU Patients

Pangaea Data Awarded Top Tier Co-Sell Partnership Status by Microsoft

Pangaea Named ‘Digital Solution of the Year’ by UK Government’s Department for International Trade

WHITEPAPER: How can Pharmaceutical & Healthcare Providers Collaborate in a Financially Sustainable & Scalable Manner to Improve Patient Outcomes?

Go back

The Potential and Pitfalls of using a LLM as a Clinical Assistant

This publication explores the application of ChatGPT and GPT-4 in the context of Chronic Obstructive Pulmonary Disease (COPD), Chronic Kidney Disease (CKD), Herpes Simplex Virus (HSV) and Primary Biliary Cirrhosis (PBC).

Recent studies have shown promising performance of ChatGPT and GPT-4 in various medical tasks. However, their utility in large-scale real-world electronic health record database and providing comprehensive diagnostic assistance has not been fully assessed. Despite our analysis using ChatGPT and GPT-4 revealed high F1 scores (up to 96%) for disease classification tasks and accurate diagnosis (75%), there were factually incorrect statements, overlooking critical medical findings and recommendations for unnecessary investigations and overtreatment. These issues coupled with privacy concerns limit the real-world clinical use of such models. Nonetheless, their potential for scalability in healthcare applications is notable as less data and prompt engineering is required compared to conventional machine learning workflows.

Request Publication