Consistency checks to improve measurement with the Young Mania Rating Scale (YMRS)

Jonathan Rabinowitz; Robert C. Young; Christian Yavorsky; Janet B.W. Williams; Jan Sedway; Patricia Marino; Christopher Matteo; Atul Mahableshwarkar; Alan Kott; Nanco Hefting; Jenicka Engler; Chris Brady

doi:10.1016/j.jad.2023.10.098

Consistency checks to improve measurement with the Young Mania Rating Scale (YMRS)

Jonathan Rabinowitz, Robert C. Young, Christian Yavorsky, Janet B.W. Williams, Jan Sedway, Patricia Marino, Christopher Matteo, Atul Mahableshwarkar, Alan Kott, Nanco Hefting, Jenicka Engler, Chris Brady

The Louis & Gabi Weisfeld School of Social Work

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

Background: Mitigating rating inconsistency can improve measurement fidelity and detection of treatment response. Methods: The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that developed logical consistency (LC) checks for ratings of the Young Mania Rating Scale (YMRS), which is widely used in studies of mood and bipolar disorders. LC and statistical outlier-response pattern checks (SC) were applied to 63,228 YMRS administrations from 14 clinical trials evaluating treatments for bipolar disorder. Checks were also applied to Monte Carlo-simulated data as a proxy for their use under conditions of inconsistency. Results: 42 LC flags were developed, and four SC flags were created from the data set (n = 14). Almost 20 % of the rating administrations had at least one LC flag, 6.7 % had two or more, 1.7 % had three or more; 17.3 % percent of the administrations had at least one SC flag and 4.6 % percent had two or more. Overall, 31 % of administrations had at least one flag of any type, 12.1 % had two or more and 5.3 % had three or more. In acute antimanic treatment trials (n = 10) there were more flags of any type compared to relapse prevention trials (n = 4). Limitations: Flagged ratings may represent less-common presentations assessed correctly. Conclusions: Using established methods, we illustrate development and application of consistency flags for YMRS ratings. Applying flags and mitigation during trials may improve the value of YMRS data, help focus attention on rater training, and improve reliability and validity of trial data.

Original language	English
Pages (from-to)	24-31
Number of pages	8
Journal	Journal of Affective Disorders
Volume	345
DOIs	https://doi.org/10.1016/j.jad.2023.10.098
State	Published - 15 Jan 2024

Bibliographical note

Publisher Copyright:
© 2023 The Author(s)

Funding

The research leading to these results has received support from the Innovative Medicine Initiative Joint Undertaking under grant agreement n° 115008 of which resources are composed of European Federation of Pharmaceutical Industries and Associations (EFPIA) in-kind contribution and financial contribution from the European Union's Seventh Framework Programme (FP7/2007-2013) and the Elie Wiesel Chair at Bar Ilan University. Funding source was not involved in the collection, analysis, and interpretation of data; in the writing of the report; and nor in the decision to submit the paper for publication. Data acknowledgment: This study, carried out under YODA Project 2022-5090, used data obtained from the Yale University Open Data Access Project, which has an agreement with Janssen Research & Development, L.L.C. The interpretation and reporting of research using this data are solely the responsibility of the authors and does not necessarily represent the official views of the Yale University Open Data Access Project or Janssen Research & Development, L.L.C.” The research leading to these results has received support from the Innovative Medicine Initiative Joint Undertaking under grant agreement n° 115008 of which resources are composed of European Federation of Pharmaceutical Industries and Associations (EFPIA) in-kind contribution and financial contribution from the European Union's Seventh Framework Programme (FP7/2007-2013) and the Elie Wiesel Chair at Bar Ilan University . Funding source was not involved in the collection, analysis, and interpretation of data; in the writing of the report; and nor in the decision to submit the paper for publication. The research leading to these results has received support from the Innovative Medicine Initiative Joint Undertaking under grant agreement n° 115008 of which resources are composed of European Federation of Pharmaceutical Industries and Associations (EFPIA) in-kind contribution and financial contribution from the European Union's Seventh Framework Programme (FP7/2007-2013) and the Elie Wiesel Chair at Bar Ilan University. Funding sources were not involved in the collection, analysis, and interpretation of data; in the writing of the report; and nor in the decision to submit the paper for publication. Funding source was not involved in the collection, analysis, and interpretation of data; in the writing of the report; and nor in the decision to submit the paper for publication.

Funders	Funder number
Yale University
Seventh Framework Programme
Bar-Ilan University	2022-5090
Seventh Framework Programme

Keywords

Careless ratings
Consistency of measurement
Inconsistent ratings
YMRS
Young Mania Rating Scale

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1016/j.jad.2023.10.098License: CC BY

Cite this

@article{c13152e7001442f6948e5425fefa6f76,

title = "Consistency checks to improve measurement with the Young Mania Rating Scale (YMRS)",

abstract = "Background: Mitigating rating inconsistency can improve measurement fidelity and detection of treatment response. Methods: The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that developed logical consistency (LC) checks for ratings of the Young Mania Rating Scale (YMRS), which is widely used in studies of mood and bipolar disorders. LC and statistical outlier-response pattern checks (SC) were applied to 63,228 YMRS administrations from 14 clinical trials evaluating treatments for bipolar disorder. Checks were also applied to Monte Carlo-simulated data as a proxy for their use under conditions of inconsistency. Results: 42 LC flags were developed, and four SC flags were created from the data set (n = 14). Almost 20 \% of the rating administrations had at least one LC flag, 6.7 \% had two or more, 1.7 \% had three or more; 17.3 \% percent of the administrations had at least one SC flag and 4.6 \% percent had two or more. Overall, 31 \% of administrations had at least one flag of any type, 12.1 \% had two or more and 5.3 \% had three or more. In acute antimanic treatment trials (n = 10) there were more flags of any type compared to relapse prevention trials (n = 4). Limitations: Flagged ratings may represent less-common presentations assessed correctly. Conclusions: Using established methods, we illustrate development and application of consistency flags for YMRS ratings. Applying flags and mitigation during trials may improve the value of YMRS data, help focus attention on rater training, and improve reliability and validity of trial data.",

keywords = "Careless ratings, Consistency of measurement, Inconsistent ratings, YMRS, Young Mania Rating Scale",

author = "Jonathan Rabinowitz and Young, \{Robert C.\} and Christian Yavorsky and Williams, \{Janet B.W.\} and Jan Sedway and Patricia Marino and Christopher Matteo and Atul Mahableshwarkar and Alan Kott and Nanco Hefting and Jenicka Engler and Chris Brady",

note = "Publisher Copyright: {\textcopyright} 2023 The Author(s)",

year = "2024",

month = jan,

day = "15",

doi = "10.1016/j.jad.2023.10.098",

language = "אנגלית",

volume = "345",

pages = "24--31",

journal = "Journal of Affective Disorders",

issn = "0165-0327",

publisher = "Elsevier B.V.",

}

TY - JOUR

T1 - Consistency checks to improve measurement with the Young Mania Rating Scale (YMRS)

AU - Rabinowitz, Jonathan

AU - Young, Robert C.

AU - Yavorsky, Christian

AU - Williams, Janet B.W.

AU - Sedway, Jan

AU - Marino, Patricia

AU - Matteo, Christopher

AU - Mahableshwarkar, Atul

AU - Kott, Alan

AU - Hefting, Nanco

AU - Engler, Jenicka

AU - Brady, Chris

PY - 2024/1/15

Y1 - 2024/1/15

N2 - Background: Mitigating rating inconsistency can improve measurement fidelity and detection of treatment response. Methods: The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that developed logical consistency (LC) checks for ratings of the Young Mania Rating Scale (YMRS), which is widely used in studies of mood and bipolar disorders. LC and statistical outlier-response pattern checks (SC) were applied to 63,228 YMRS administrations from 14 clinical trials evaluating treatments for bipolar disorder. Checks were also applied to Monte Carlo-simulated data as a proxy for their use under conditions of inconsistency. Results: 42 LC flags were developed, and four SC flags were created from the data set (n = 14). Almost 20 % of the rating administrations had at least one LC flag, 6.7 % had two or more, 1.7 % had three or more; 17.3 % percent of the administrations had at least one SC flag and 4.6 % percent had two or more. Overall, 31 % of administrations had at least one flag of any type, 12.1 % had two or more and 5.3 % had three or more. In acute antimanic treatment trials (n = 10) there were more flags of any type compared to relapse prevention trials (n = 4). Limitations: Flagged ratings may represent less-common presentations assessed correctly. Conclusions: Using established methods, we illustrate development and application of consistency flags for YMRS ratings. Applying flags and mitigation during trials may improve the value of YMRS data, help focus attention on rater training, and improve reliability and validity of trial data.

AB - Background: Mitigating rating inconsistency can improve measurement fidelity and detection of treatment response. Methods: The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that developed logical consistency (LC) checks for ratings of the Young Mania Rating Scale (YMRS), which is widely used in studies of mood and bipolar disorders. LC and statistical outlier-response pattern checks (SC) were applied to 63,228 YMRS administrations from 14 clinical trials evaluating treatments for bipolar disorder. Checks were also applied to Monte Carlo-simulated data as a proxy for their use under conditions of inconsistency. Results: 42 LC flags were developed, and four SC flags were created from the data set (n = 14). Almost 20 % of the rating administrations had at least one LC flag, 6.7 % had two or more, 1.7 % had three or more; 17.3 % percent of the administrations had at least one SC flag and 4.6 % percent had two or more. Overall, 31 % of administrations had at least one flag of any type, 12.1 % had two or more and 5.3 % had three or more. In acute antimanic treatment trials (n = 10) there were more flags of any type compared to relapse prevention trials (n = 4). Limitations: Flagged ratings may represent less-common presentations assessed correctly. Conclusions: Using established methods, we illustrate development and application of consistency flags for YMRS ratings. Applying flags and mitigation during trials may improve the value of YMRS data, help focus attention on rater training, and improve reliability and validity of trial data.

KW - Careless ratings

KW - Consistency of measurement

KW - Inconsistent ratings

KW - YMRS

KW - Young Mania Rating Scale

UR - https://www.scopus.com/pages/publications/85174519191

U2 - 10.1016/j.jad.2023.10.098

DO - 10.1016/j.jad.2023.10.098

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

C2 - 37865349

AN - SCOPUS:85174519191

SN - 0165-0327

VL - 345

SP - 24

EP - 31

JO - Journal of Affective Disorders

JF - Journal of Affective Disorders

ER -

Consistency checks to improve measurement with the Young Mania Rating Scale (YMRS)

Abstract

Bibliographical note

Funding

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this