Factors affecting reliability and confidence of DSM-III-R psychosis-related diagnosis

The relationships between interrater diagnostic reliability, confidence in diagnosis, DSM-III-R criteria ambiguity and case report data quality were examined in 20 case vignettes describing psychosis-related symptomatology. Each of seven diagnosticians made DSM-III-R diagnoses and gave confidence ratings for those diagnoses, as well as ratings on quality of data presented and clarity of diagnostic criteria for each vignette. As hypothesized, confidence ratings significantly predicted interrater diagnostic agreement. Clarity of DSM-III-R criteria positively correlated with both interrater reliability and confidence. Case report data quality correlated with diagnostic confidence but, contrary to the authors' hypothesis, did not correlate with interrater agreement. The authors conclude that confidence ratings may be useful indices of diagnostic reliability among experienced clinicians and suggest that the case report method allows for reliable diagnoses to be made.