Self-supervised learning for domain adaptation on point clouds

Idan Achituve; Haggai Maron; Gal Chechik

doi:10.1109/WACV48630.2021.00017

Self-supervised learning for domain adaptation on point clouds

Idan Achituve, Haggai Maron, Gal Chechik

NVIDIA

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

146 Scopus citations

Abstract

Self-supervised learning (SSL) is a technique for learning useful representations from unlabeled data. It has been applied effectively to domain adaptation (DA) on images and videos. It is still unknown if and how it can be leveraged for domain adaptation in 3D perception problems. Here we describe the first study of SSL for DA on point clouds. We introduce a new family of pretext tasks, Deformation Reconstruction, inspired by the deformations encountered in sim-to-real transformations. In addition, we propose a novel training procedure for labeled point cloud data motivated by the MixUp method called Point cloud Mixup (PCM). Evaluations on domain adaptations datasets for classification and segmentation, demonstrate a large improvement over existing and baseline methods.

Original language	English
Title of host publication	Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	123-133
Number of pages	11
ISBN (Electronic)	9780738142661
DOIs	https://doi.org/10.1109/WACV48630.2021.00017
State	Published - Jan 2021
Event	2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021 - Virtual, Online, United States Duration: 5 Jan 2021 → 9 Jan 2021

Publication series

Name	Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021

Conference

Conference	2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021
Country/Territory	United States
City	Virtual, Online
Period	5/01/21 → 9/01/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Funding

IA was funded by the Israel innovation authority as part of the AVATAR consortium, and by a grant from the Israel Science Foundation (ISF 737/2018).

Funders	Funder number
Israel Innovation Authority
Israel Science Foundation	ISF 737/2018

Access to Document

10.1109/WACV48630.2021.00017

Cite this

Achituve, I., Maron, H., & Chechik, G. (2021). Self-supervised learning for domain adaptation on point clouds. In Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021 (pp. 123-133). (Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV48630.2021.00017

@inproceedings{2a650b974d8b4c1caac0202836496879,

title = "Self-supervised learning for domain adaptation on point clouds",

abstract = "Self-supervised learning (SSL) is a technique for learning useful representations from unlabeled data. It has been applied effectively to domain adaptation (DA) on images and videos. It is still unknown if and how it can be leveraged for domain adaptation in 3D perception problems. Here we describe the first study of SSL for DA on point clouds. We introduce a new family of pretext tasks, Deformation Reconstruction, inspired by the deformations encountered in sim-to-real transformations. In addition, we propose a novel training procedure for labeled point cloud data motivated by the MixUp method called Point cloud Mixup (PCM). Evaluations on domain adaptations datasets for classification and segmentation, demonstrate a large improvement over existing and baseline methods.",

author = "Idan Achituve and Haggai Maron and Gal Chechik",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021 ; Conference date: 05-01-2021 Through 09-01-2021",

year = "2021",

month = jan,

doi = "10.1109/WACV48630.2021.00017",

language = "אנגלית",

series = "Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "123--133",

booktitle = "Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021",

address = "ארצות הברית",

}

Achituve, I, Maron, H & Chechik, G 2021, Self-supervised learning for domain adaptation on point clouds. in Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021. Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021, Institute of Electrical and Electronics Engineers Inc., pp. 123-133, 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021, Virtual, Online, United States, 5/01/21. https://doi.org/10.1109/WACV48630.2021.00017

Self-supervised learning for domain adaptation on point clouds. / Achituve, Idan; Maron, Haggai; Chechik, Gal.
Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 123-133 (Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Self-supervised learning for domain adaptation on point clouds

AU - Achituve, Idan

AU - Maron, Haggai

AU - Chechik, Gal

PY - 2021/1

Y1 - 2021/1

N2 - Self-supervised learning (SSL) is a technique for learning useful representations from unlabeled data. It has been applied effectively to domain adaptation (DA) on images and videos. It is still unknown if and how it can be leveraged for domain adaptation in 3D perception problems. Here we describe the first study of SSL for DA on point clouds. We introduce a new family of pretext tasks, Deformation Reconstruction, inspired by the deformations encountered in sim-to-real transformations. In addition, we propose a novel training procedure for labeled point cloud data motivated by the MixUp method called Point cloud Mixup (PCM). Evaluations on domain adaptations datasets for classification and segmentation, demonstrate a large improvement over existing and baseline methods.

AB - Self-supervised learning (SSL) is a technique for learning useful representations from unlabeled data. It has been applied effectively to domain adaptation (DA) on images and videos. It is still unknown if and how it can be leveraged for domain adaptation in 3D perception problems. Here we describe the first study of SSL for DA on point clouds. We introduce a new family of pretext tasks, Deformation Reconstruction, inspired by the deformations encountered in sim-to-real transformations. In addition, we propose a novel training procedure for labeled point cloud data motivated by the MixUp method called Point cloud Mixup (PCM). Evaluations on domain adaptations datasets for classification and segmentation, demonstrate a large improvement over existing and baseline methods.

UR - http://www.scopus.com/inward/record.url?scp=85116114872&partnerID=8YFLogxK

U2 - 10.1109/WACV48630.2021.00017

DO - 10.1109/WACV48630.2021.00017

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

T3 - Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021

SP - 123

EP - 133

BT - Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021

Y2 - 5 January 2021 through 9 January 2021

ER -

Achituve I, Maron H, Chechik G. Self-supervised learning for domain adaptation on point clouds. In Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021. Institute of Electrical and Electronics Engineers Inc. 2021. p. 123-133. (Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021). doi: 10.1109/WACV48630.2021.00017

Self-supervised learning for domain adaptation on point clouds

Abstract

Publication series

Conference

Bibliographical note

Funding

Access to Document

Other files and links

Fingerprint

Cite this