Prevalence and implications of significance testing for baseline covariate imbalance in randomised cancer clinical trials: The Table 1 Fallacy

Alexander D. Sherry, Pavlos Msaouel, Zachary R. McCaw, Joseph Abi Jaoude, Eric J. Hsu, Ramez Kouzy, Roshal Patel, Yumeng Yang, Timothy A. Lin, Cullen M. Taniguchi, Claus Rödel, Emmanouil Fokas, Chad Tang, Clifton David Fuller, Bruce Minsky, Tomer Meirson, Ryan Sun, Ethan B. Ludmir

Research output: Contribution to journalArticlepeer-review

Abstract

Background: The ‘Table 1 Fallacy’ refers to the unsound use of significance testing for comparing the distributions of baseline variables between randomised groups to draw erroneous conclusions about balance or imbalance. We performed a cross-sectional study of the Table 1 Fallacy in phase III oncology trials. Methods: From ClinicalTrials.gov, 1877 randomised trials were screened. Multivariable logistic regressions evaluated predictors of the Table 1 Fallacy. Results: A total of 765 randomised controlled trials involving 553,405 patients were analysed. The Table 1 Fallacy was observed in 25% of trials (188 of 765), with 3% of comparisons deemed significant (59 of 2353), approximating the typical 5% type I error assertion probability. Application of trial-level multiplicity corrections reduced the rate of significant findings to 0.3% (six of 2345 tests). Factors associated with lower odds of the Table 1 Fallacy included industry sponsorship (adjusted odds ratio [aOR] 0.29, 95% confidence interval [CI] 0.18–0.47; multiplicity-corrected P < 0.0001), larger trial size (≥795 versus <280 patients; aOR 0.32, 95% CI 0.19–0.53; multiplicity-corrected P = 0.0008), and publication in a European versus American journal (aOR 0.06, 95% CI 0.03–0.13; multiplicity-corrected P < 0.0001). Conclusions: This study highlights the persistence of the Table 1 Fallacy in contemporary oncology randomised controlled trials, with one of every four trials testing for baseline differences after randomisation. Significance testing is a suboptimal method for identifying unsound randomisation procedures and may encourage misleading inferences. Journal-level enforcement is a possible strategy to help mitigate this fallacy.

Original languageEnglish
Article number113357
JournalEuropean Journal of Cancer
Volume194
DOIs
StatePublished - Nov 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2023 The Authors

Funding

Christine Wogan, MS, ELS (Division of Radiation Oncology, The University of Texas MD Anderson Cancer Centre), provided editorial contributions. She received no compensation beyond her salary for this contribution. We thank Carrie L. Sherry, BS, for assistance with the figures. Research data are stored in an institutional repository and will be shared upon reasonable request to the corresponding author. This work was supported in part by National Cancer Institute, National Institutes of Health grant P30CA016672. This work was supported in part by the Sabin Family Fellowship Foundation (EBL) and the Fund for Innovation in Cancer Informatics (EBL).

FundersFunder number
National Institutes of HealthP30CA016672
National Cancer Institute
University of Texas MD Anderson Cancer Center
Fund for Innovation in Cancer Informatics

    Keywords

    • Covariate imbalance
    • Oncology
    • Phase III
    • Randomised controlled trials
    • Significance testing for baseline characteristics
    • Table 1 Fallacy
    • Testing for baseline differences

    Fingerprint

    Dive into the research topics of 'Prevalence and implications of significance testing for baseline covariate imbalance in randomised cancer clinical trials: The Table 1 Fallacy'. Together they form a unique fingerprint.

    Cite this