Abstract
Free-text analysis using machine learning (ML)-based natural language processing (NLP) shows promise for diagnosing psychiatric conditions. Chat Generative Pre-trained Transformer (ChatGPT) has demonstrated preliminary initial feasibility for this purpose; however, whether it can accurately assess mental illness remains to be determined. This study evaluates the effectiveness of ChatGPT and the text-embedding-ada-002 (ADA) model in detecting post-traumatic stress disorder following childbirth (CB-PTSD), a maternal postpartum mental illness affecting millions of women annually, with no standard screening protocol. Using a sample of 1295 women who gave birth in the last six months and were 18+ years old, recruited through hospital announcements, social media, and professional organizations, we explore ChatGPT’s and ADA’s potential to screen for CB-PTSD by analyzing maternal childbirth narratives. The PTSD Checklist for DSM-5 (PCL-5; cutoff 31) was used to assess CB-PTSD. By developing an ML model that utilizes numerical vector representation of the ADA model, we identify CB-PTSD via narrative classification. Our model outperformed (F1 score: 0.81) ChatGPT and six previously published large text-embedding models trained on mental health or clinical domains data, suggesting that the ADA model can be harnessed to identify CB-PTSD. Our modeling approach could be generalized to assess other mental health disorders.
Original language | English |
---|---|
Article number | 8336 |
Journal | Scientific Reports |
Volume | 14 |
Issue number | 1 |
DOIs | |
State | Published - 11 Apr 2024 |
Bibliographical note
Publisher Copyright:© The Author(s) 2024.
Keywords
- Birth narratives
- Birth trauma
- ChatGPT
- Childbirth-related post-traumatic stress disorder (CB-PTSD)
- Maternal mental health
- Natural language processing (NLP)
- Postpartum PTSD
- Pre-trained large language model (PLM)