Predictive tools for major bleeding (MB) using machine learning (ML) might be advantageous over traditional methods. We used data from the Registro Informatizado de Enfermedad TromboEmbólica (RIETE) to develop ML algorithms to identify patients with venous thromboembolism (VTE) at increased risk of MB during the first 3 months of anticoagulation. A total of 55 baseline variables were used as predictors. New data prospectively collected from the RIETE were used for further validation. The RIETE and VTE-BLEED scores were used for comparisons. External validation was performed with the COMMAND-VTE database. Learning was carried out with data from 49 587 patients, of whom 873 (1.8%) had MB. The best performing ML method was XGBoost. In the prospective validation cohort the sensitivity, specificity, positive predictive value and F1 score were: 33.2%, 93%, 10%, and 15.4% respectively. F1 value for the RIETE and VTE-BLEED scores were 8.6% and 6.4% respectively. In the external validation cohort the metrics were 10.3%, 87.6%, 3.5% and 5.2% respectively. In that cohort, the F1 value for the RIETE score was 17.3% and for the VTE-BLEED score 9.75%. The performance of the XGBoost algorithm was better than that from the RIETE and VTE-BLEED scores only in the prospective validation cohort, but not in the external validation cohort.

Original languageEnglish
Pages (from-to)971-981
Number of pages11
JournalBritish Journal of Haematology
Issue number5
Early online date21 Mar 2023
StatePublished - Jun 2023

Bibliographical note

Publisher Copyright:
© 2023 The Authors. British Journal of Haematology published by British Society for Haematology and John Wiley & Sons Ltd.


We express our gratitude to Sanofi Spain and ROVI for supporting this Registry with an unrestricted educational grant. We also thank the RIETE Registry Co-ordinating Center, S&H Medical Science Service, for their quality control data, logistic and administrative support. Dr Bikdeli reports that he is a consulting expert, on behalf of the plaintiff, for litigation related to two specific brand models of inferior vena cava filters. Dr Bikdeli is supported by the Scott Schoen and Nancy Adams IGNITE Award, and the Mary Ann Tynan Research Scientist award through the Mary Horrigan Connors Center for Women's Health and Gender Biology, as well as the Heart and Vascular Center Junior Faculty Award, all at Brigham and Women's Hospital. Dr Bikdeli is a recipient of a Career Development Award by the American Heart Association and VIVA Physicians (no. #938814).

FundersFunder number
Mary Ann Tynan
Mary Horrigan Connors Center for Women's Health and Gender Biology
S&H Medical Science Service
VIVA Physicians938814
American Heart Association


    • haemorrhage
    • machine learning
    • outcomes
    • pulmonary embolism
    • venous thrombosis


    Dive into the research topics of 'Machine learning to predict major bleeding during anticoagulation for venous thromboembolism: possibilities and limitations'. Together they form a unique fingerprint.

    Cite this