Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation

Nimrod Sagie; Hayit Greenspan; Jacob Goldberger

doi:10.1007/978-3-030-87589-3_17

Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation

Nimrod Sagie, Hayit Greenspan, Jacob Goldberger

Tel Aviv University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Transfer learning is a machine learning technique where a model trained on one task is used to initialize the learning procedure of a second related task which has only a small amount of training data. Transfer learning can also be used as a regularization procedure by penalizing the learned parameters if they deviate too much from their initial values. In this study we show that the learned parameters move apart from the source task as the image processing progresses along the network layers. To cope with this behaviour we propose a transfer regularization method based on monotonically decreasing regularization coefficients. We demonstrate the power of the proposed regularized transfer learning scheme on COVID-19 opacity task. Specifically, we show that it can improve the segmentation of coronavirus lesions in chest CT scans.

Original language	English
Title of host publication	Machine Learning in Medical Imaging - 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Proceedings
Editors	Chunfeng Lian, Xiaohuan Cao, Islem Rekik, Xuanang Xu, Pingkun Yan
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	161-170
Number of pages	10
ISBN (Print)	9783030875886
DOIs	https://doi.org/10.1007/978-3-030-87589-3_17
State	Published - 2021
Event	12th International Workshop on Machine Learning in Medical Imaging, MLMI 2021, held in conjunction with 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 - Virtual, Online Duration: 27 Sep 2021 → 27 Sep 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12966 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	12th International Workshop on Machine Learning in Medical Imaging, MLMI 2021, held in conjunction with 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021
City	Virtual, Online
Period	27/09/21 → 27/09/21

Bibliographical note

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.

Funding

This research was supported by the Ministry of Science & Technology, Israel.

Funders	Funder number
Ministry of science and technology, Israel

Keywords

COVID-19 opacity
Regularization
Transfer learning

Access to Document

10.1007/978-3-030-87589-3_17

Cite this

Sagie, N., Greenspan, H., & Goldberger, J. (2021). Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation. In C. Lian, X. Cao, I. Rekik, X. Xu, & P. Yan (Eds.), Machine Learning in Medical Imaging - 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Proceedings (pp. 161-170). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12966 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-87589-3_17

Sagie, Nimrod ; Greenspan, Hayit ; Goldberger, Jacob. / Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation. Machine Learning in Medical Imaging - 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Proceedings. editor / Chunfeng Lian ; Xiaohuan Cao ; Islem Rekik ; Xuanang Xu ; Pingkun Yan. Springer Science and Business Media Deutschland GmbH, 2021. pp. 161-170 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{6052030b06604687940b6c9a1972c70c,

title = "Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation",

abstract = "Transfer learning is a machine learning technique where a model trained on one task is used to initialize the learning procedure of a second related task which has only a small amount of training data. Transfer learning can also be used as a regularization procedure by penalizing the learned parameters if they deviate too much from their initial values. In this study we show that the learned parameters move apart from the source task as the image processing progresses along the network layers. To cope with this behaviour we propose a transfer regularization method based on monotonically decreasing regularization coefficients. We demonstrate the power of the proposed regularized transfer learning scheme on COVID-19 opacity task. Specifically, we show that it can improve the segmentation of coronavirus lesions in chest CT scans.",

keywords = "COVID-19 opacity, Regularization, Transfer learning",

author = "Nimrod Sagie and Hayit Greenspan and Jacob Goldberger",

note = "Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 12th International Workshop on Machine Learning in Medical Imaging, MLMI 2021, held in conjunction with 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 ; Conference date: 27-09-2021 Through 27-09-2021",

year = "2021",

doi = "10.1007/978-3-030-87589-3_17",

language = "אנגלית",

isbn = "9783030875886",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "161--170",

editor = "Chunfeng Lian and Xiaohuan Cao and Islem Rekik and Xuanang Xu and Pingkun Yan",

booktitle = "Machine Learning in Medical Imaging - 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Proceedings",

address = "גרמניה",

}

Sagie, N, Greenspan, H & Goldberger, J 2021, Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation. in C Lian, X Cao, I Rekik, X Xu & P Yan (eds), Machine Learning in Medical Imaging - 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12966 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 161-170, 12th International Workshop on Machine Learning in Medical Imaging, MLMI 2021, held in conjunction with 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021, Virtual, Online, 27/09/21. https://doi.org/10.1007/978-3-030-87589-3_17

Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation. / Sagie, Nimrod; Greenspan, Hayit; Goldberger, Jacob.
Machine Learning in Medical Imaging - 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Proceedings. ed. / Chunfeng Lian; Xiaohuan Cao; Islem Rekik; Xuanang Xu; Pingkun Yan. Springer Science and Business Media Deutschland GmbH, 2021. p. 161-170 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12966 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation

AU - Sagie, Nimrod

AU - Greenspan, Hayit

AU - Goldberger, Jacob

PY - 2021

Y1 - 2021

N2 - Transfer learning is a machine learning technique where a model trained on one task is used to initialize the learning procedure of a second related task which has only a small amount of training data. Transfer learning can also be used as a regularization procedure by penalizing the learned parameters if they deviate too much from their initial values. In this study we show that the learned parameters move apart from the source task as the image processing progresses along the network layers. To cope with this behaviour we propose a transfer regularization method based on monotonically decreasing regularization coefficients. We demonstrate the power of the proposed regularized transfer learning scheme on COVID-19 opacity task. Specifically, we show that it can improve the segmentation of coronavirus lesions in chest CT scans.

AB - Transfer learning is a machine learning technique where a model trained on one task is used to initialize the learning procedure of a second related task which has only a small amount of training data. Transfer learning can also be used as a regularization procedure by penalizing the learned parameters if they deviate too much from their initial values. In this study we show that the learned parameters move apart from the source task as the image processing progresses along the network layers. To cope with this behaviour we propose a transfer regularization method based on monotonically decreasing regularization coefficients. We demonstrate the power of the proposed regularized transfer learning scheme on COVID-19 opacity task. Specifically, we show that it can improve the segmentation of coronavirus lesions in chest CT scans.

KW - COVID-19 opacity

KW - Regularization

KW - Transfer learning

UR - http://www.scopus.com/inward/record.url?scp=85116447274&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-87589-3_17

DO - 10.1007/978-3-030-87589-3_17

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85116447274

SN - 9783030875886

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 161

EP - 170

BT - Machine Learning in Medical Imaging - 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Proceedings

A2 - Lian, Chunfeng

A2 - Cao, Xiaohuan

A2 - Rekik, Islem

A2 - Xu, Xuanang

A2 - Yan, Pingkun

PB - Springer Science and Business Media Deutschland GmbH

T2 - 12th International Workshop on Machine Learning in Medical Imaging, MLMI 2021, held in conjunction with 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021

Y2 - 27 September 2021 through 27 September 2021

ER -

Sagie N, Greenspan H, Goldberger J. Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation. In Lian C, Cao X, Rekik I, Xu X, Yan P, editors, Machine Learning in Medical Imaging - 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 161-170. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-87589-3_17

Transfer Learning with a Layer Dependent Regularization for Medical Image Segmentation

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this