CALM: Conditional Adversarial Latent Models for Directable Virtual Characters

Chen Tessler; Yoni Kasten; Yunrong Guo; Shie Mannor; Gal Chechik; Xue Bin Peng

doi:10.1145/3588432.3591541

CALM: Conditional Adversarial Latent Models for Directable Virtual Characters

Chen Tessler, Yoni Kasten, Yunrong Guo, Shie Mannor, Gal Chechik, Xue Bin Peng

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

35 Scopus citations

Abstract

In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for user-controlled interactive virtual characters. Using imitation learning, CALM learns a representation of movement that captures the complexity and diversity of human motion, and enables direct control over character movements. The approach jointly learns a control policy and a motion encoder that reconstructs key characteristics of a given motion without merely replicating it. The results show that CALM learns a semantic motion representation, enabling control over the generated motions and style-conditioning for higher-level task training. Once trained, the character can be controlled using intuitive interfaces, akin to those found in video games.

Original language	English
Title of host publication	Proceedings - SIGGRAPH 2023 Conference Papers
Editors	Stephen N. Spencer
Publisher	Association for Computing Machinery, Inc
ISBN (Electronic)	9798400701597
DOIs	https://doi.org/10.1145/3588432.3591541
State	Published - 23 Jul 2023
Event	2023 Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH 2023 - Los Angeles, United States Duration: 6 Aug 2023 → 10 Aug 2023

Publication series

Name	Proceedings - SIGGRAPH 2023 Conference Papers

Conference

Conference	2023 Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH 2023
Country/Territory	United States
City	Los Angeles
Period	6/08/23 → 10/08/23

Bibliographical note

Publisher Copyright:
© 2023 ACM.

Keywords

adversarial training
animated character control
motion capture data
reinforcement learning

Access to Document

10.1145/3588432.3591541

Cite this

Tessler, C., Kasten, Y., Guo, Y., Mannor, S., Chechik, G., & Peng, X. B. (2023). CALM: Conditional Adversarial Latent Models for Directable Virtual Characters. In S. N. Spencer (Ed.), Proceedings - SIGGRAPH 2023 Conference Papers Article 37 (Proceedings - SIGGRAPH 2023 Conference Papers). Association for Computing Machinery, Inc. https://doi.org/10.1145/3588432.3591541

@inproceedings{aff975eae6974048a257ce33028592de,

title = "CALM: Conditional Adversarial Latent Models for Directable Virtual Characters",

abstract = "In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for user-controlled interactive virtual characters. Using imitation learning, CALM learns a representation of movement that captures the complexity and diversity of human motion, and enables direct control over character movements. The approach jointly learns a control policy and a motion encoder that reconstructs key characteristics of a given motion without merely replicating it. The results show that CALM learns a semantic motion representation, enabling control over the generated motions and style-conditioning for higher-level task training. Once trained, the character can be controlled using intuitive interfaces, akin to those found in video games.",

keywords = "adversarial training, animated character control, motion capture data, reinforcement learning",

author = "Chen Tessler and Yoni Kasten and Yunrong Guo and Shie Mannor and Gal Chechik and Peng, \{Xue Bin\}",

note = "Publisher Copyright: {\textcopyright} 2023 ACM.; 2023 Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH 2023 ; Conference date: 06-08-2023 Through 10-08-2023",

year = "2023",

month = jul,

day = "23",

doi = "10.1145/3588432.3591541",

language = "אנגלית",

series = "Proceedings - SIGGRAPH 2023 Conference Papers",

publisher = "Association for Computing Machinery, Inc",

editor = "Spencer, \{Stephen N.\}",

booktitle = "Proceedings - SIGGRAPH 2023 Conference Papers",

}

Tessler, C, Kasten, Y, Guo, Y, Mannor, S, Chechik, G & Peng, XB 2023, CALM: Conditional Adversarial Latent Models for Directable Virtual Characters. in SN Spencer (ed.), Proceedings - SIGGRAPH 2023 Conference Papers., 37, Proceedings - SIGGRAPH 2023 Conference Papers, Association for Computing Machinery, Inc, 2023 Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH 2023, Los Angeles, United States, 6/08/23. https://doi.org/10.1145/3588432.3591541

CALM: Conditional Adversarial Latent Models for Directable Virtual Characters. / Tessler, Chen; Kasten, Yoni; Guo, Yunrong et al.
Proceedings - SIGGRAPH 2023 Conference Papers. ed. / Stephen N. Spencer. Association for Computing Machinery, Inc, 2023. 37 (Proceedings - SIGGRAPH 2023 Conference Papers).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - CALM

T2 - 2023 Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH 2023

AU - Tessler, Chen

AU - Kasten, Yoni

AU - Guo, Yunrong

AU - Mannor, Shie

AU - Chechik, Gal

AU - Peng, Xue Bin

PY - 2023/7/23

Y1 - 2023/7/23

N2 - In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for user-controlled interactive virtual characters. Using imitation learning, CALM learns a representation of movement that captures the complexity and diversity of human motion, and enables direct control over character movements. The approach jointly learns a control policy and a motion encoder that reconstructs key characteristics of a given motion without merely replicating it. The results show that CALM learns a semantic motion representation, enabling control over the generated motions and style-conditioning for higher-level task training. Once trained, the character can be controlled using intuitive interfaces, akin to those found in video games.

AB - In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for user-controlled interactive virtual characters. Using imitation learning, CALM learns a representation of movement that captures the complexity and diversity of human motion, and enables direct control over character movements. The approach jointly learns a control policy and a motion encoder that reconstructs key characteristics of a given motion without merely replicating it. The results show that CALM learns a semantic motion representation, enabling control over the generated motions and style-conditioning for higher-level task training. Once trained, the character can be controlled using intuitive interfaces, akin to those found in video games.

KW - adversarial training

KW - animated character control

KW - motion capture data

KW - reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85167971943&partnerID=8YFLogxK

U2 - 10.1145/3588432.3591541

DO - 10.1145/3588432.3591541

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85167971943

T3 - Proceedings - SIGGRAPH 2023 Conference Papers

BT - Proceedings - SIGGRAPH 2023 Conference Papers

A2 - Spencer, Stephen N.

PB - Association for Computing Machinery, Inc

Y2 - 6 August 2023 through 10 August 2023

ER -

CALM: Conditional Adversarial Latent Models for Directable Virtual Characters

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this