Customization scenarios for de-identification of clinical notes

Tzvika Hartman, Michael D. Howell, Jeff Dean, Shlomo Hoory, Ronit Slyper, Itay Laish, Oren Gilon, Danny Vainstein, Greg Corrado, Katherine Chou, Ming Jack Po, Jutta Williams, Scott Ellis, Gavin Bee, Avinatan Hassidim, Rony Amira, Genady Beryozkin, Idan Szpektor, Yossi Matias

Research output: Contribution to journalArticlepeer-review

20 Scopus citations


Background: Automated machine-learning systems are able to de-identify electronic medical records, including free-text clinical notes. Use of such systems would greatly boost the amount of data available to researchers, yet their deployment has been limited due to uncertainty about their performance when applied to new datasets. Objective: We present practical options for clinical note de-identification, assessing performance of machine learning systems ranging from off-the-shelf to fully customized. Methods: We implement a state-of-the-art machine learning de-identification system, training and testing on pairs of datasets that match the deployment scenarios. We use clinical notes from two i2b2 competition corpora, the Physionet Gold Standard corpus, and parts of the MIMIC-III dataset. Results: Fully customized systems remove 97-99% of personally identifying information. Performance of off-the-shelf systems varies by dataset, with performance mostly above 90%. Providing a small labeled dataset or large unlabeled dataset allows for fine-tuning that improves performance over off-the-shelf systems. Conclusion: Health organizations should be aware of the levels of customization available when selecting a de-identification deployment solution, in order to choose the one that best matches their resources and target performance level.

Original languageEnglish
Article number14
JournalBMC Medical Informatics and Decision Making
Issue number1
StatePublished - 30 Jan 2020
Externally publishedYes

Bibliographical note

Funding Information:
The study was funded by Google, LLC. The design of the study, collection, analysis, interpretation of data, and writing of the manuscript were all performed by Google employees.

Publisher Copyright:
© 2020 The Author(s).


  • Clinical notes
  • De-identification
  • Electronic health records
  • Free text
  • Natural language processing
  • Recurrent neural networks


Dive into the research topics of 'Customization scenarios for de-identification of clinical notes'. Together they form a unique fingerprint.

Cite this