The ASC-inclusion perceptual serious gaming platform for autistic children

Erik Marchi; Björn Schuller; Alice Baird; Simon Baron-Cohen; Amandine Lassalle; Helen O'Reilly; Delia Pigat; Peter Robinson; Ian Davies; Tadas Baltrušaitis; Andra Adams; Marwa Mahmoud; Ofer Golan; Shimrit Fridenson-Hayo; Shahar Tal; Shai Newman; Noga Meir-Goren; Antonio Camurri; Stefano Piana; Sven Bölte; Metin Sezgin; Nese Alyuz; Agnieszka Rynkiewicz; Aurelie Baranger

doi:10.1109/TG.2018.2864640

The ASC-inclusion perceptual serious gaming platform for autistic children

Erik Marchi, Björn Schuller, Alice Baird, Simon Baron-Cohen, Amandine Lassalle, Helen O'Reilly, Delia Pigat, Peter Robinson, Ian Davies, Tadas Baltrušaitis, Andra Adams, Marwa Mahmoud, Ofer Golan, Shimrit Fridenson-Hayo, Shahar Tal, Shai Newman, Noga Meir-Goren, Antonio Camurri, Stefano Piana, Sven BölteMetin Sezgin, Nese Alyuz, Agnieszka Rynkiewicz, Aurelie Baranger

Department of Psychology

Research output: Contribution to journal › Article › peer-review

15 Scopus citations

Abstract

“Serious games” are becoming extremely relevant to individuals who have specific needs, such as children with an autism spectrum condition (ASC). Often, individuals with an ASC have difficulties in interpreting verbal and nonverbal communication cues during social interactions. The ASC-Inclusion EU-FP7 funded project aims to provide children who have an ASC with a platform to learn emotion expression and recognition, through play in the virtual world. In particular, the ASC-Inclusion platform focuses on the expression of emotion via facial, vocal, and bodily gestures. The platform combines multiple analysis tools, using onboard microphone and webcam capabilities. The platform utilizes these capabilities via training games, text-based communication, animations, video, and audio clips. This paper introduces current findings and evaluations of the ASC-Inclusion platform and provides detailed description for the different modalities.

Original language	English
Article number	2864640
Pages (from-to)	328-339
Number of pages	12
Journal	IEEE Transactions on Games
Volume	11
Issue number	4
DOIs	https://doi.org/10.1109/TG.2018.2864640
State	Published - Dec 2019

Bibliographical note

Funding Information:
Manuscript received January 2, 2017; revised October 7, 2017, March 10, 2018, and June 23, 2018; accepted July 16, 2018. Date of publication August 9, 2018; date of current version December 23, 2019. This work was supported by the European Community’s Seventh Framework Programme (FP7/2007–2013) and Horizon 2020 under Grant 289021 (ASC-Inclusion) and Grant 688835 (RIA DE-ENIGMA). (Corresponding author: Erik Marchi.) E. Marchi is with Siri Speech, Apple Inc., CA, USA. The work was done while at the Machine Intelligence and Signal Processing Group, MMK, Technische Universität München, Munich Germany, and also with audEERING GmbH, Gilching Germany (e-mail:,erik.marchi@tum.de).

Publisher Copyright:
© 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

Keywords

AI in games
Autism spectrum condition (ASC)
Emotion recognition
Inclusion
Virtual computerized environment

Access to Document

10.1109/TG.2018.2864640

Cite this

Marchi, E., Schuller, B., Baird, A., Baron-Cohen, S., Lassalle, A., O'Reilly, H., Pigat, D., Robinson, P., Davies, I., Baltrušaitis, T., Adams, A., Mahmoud, M., Golan, O., Fridenson-Hayo, S., Tal, S., Newman, S., Meir-Goren, N., Camurri, A., Piana, S., ... Baranger, A. (2019). The ASC-inclusion perceptual serious gaming platform for autistic children. IEEE Transactions on Games, 11(4), 328-339. [2864640]. https://doi.org/10.1109/TG.2018.2864640

@article{de32fa13dd3d4587ac77b9e6b663214e,

title = "The ASC-inclusion perceptual serious gaming platform for autistic children",

abstract = "“Serious games” are becoming extremely relevant to individuals who have specific needs, such as children with an autism spectrum condition (ASC). Often, individuals with an ASC have difficulties in interpreting verbal and nonverbal communication cues during social interactions. The ASC-Inclusion EU-FP7 funded project aims to provide children who have an ASC with a platform to learn emotion expression and recognition, through play in the virtual world. In particular, the ASC-Inclusion platform focuses on the expression of emotion via facial, vocal, and bodily gestures. The platform combines multiple analysis tools, using onboard microphone and webcam capabilities. The platform utilizes these capabilities via training games, text-based communication, animations, video, and audio clips. This paper introduces current findings and evaluations of the ASC-Inclusion platform and provides detailed description for the different modalities.",

keywords = "AI in games, Autism spectrum condition (ASC), Emotion recognition, Inclusion, Virtual computerized environment",

author = "Erik Marchi and Bj{\"o}rn Schuller and Alice Baird and Simon Baron-Cohen and Amandine Lassalle and Helen O'Reilly and Delia Pigat and Peter Robinson and Ian Davies and Tadas Baltru{\v s}aitis and Andra Adams and Marwa Mahmoud and Ofer Golan and Shimrit Fridenson-Hayo and Shahar Tal and Shai Newman and Noga Meir-Goren and Antonio Camurri and Stefano Piana and Sven B{\"o}lte and Metin Sezgin and Nese Alyuz and Agnieszka Rynkiewicz and Aurelie Baranger",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.",

year = "2019",

month = dec,

doi = "10.1109/TG.2018.2864640",

language = "אנגלית",

volume = "11",

pages = "328--339",

journal = "IEEE Transactions on Games",

issn = "2475-1502",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "4",

}

Marchi, E, Schuller, B, Baird, A, Baron-Cohen, S, Lassalle, A, O'Reilly, H, Pigat, D, Robinson, P, Davies, I, Baltrušaitis, T, Adams, A, Mahmoud, M, Golan, O, Fridenson-Hayo, S, Tal, S, Newman, S, Meir-Goren, N, Camurri, A, Piana, S, Bölte, S, Sezgin, M, Alyuz, N, Rynkiewicz, A & Baranger, A 2019, 'The ASC-inclusion perceptual serious gaming platform for autistic children', IEEE Transactions on Games, vol. 11, no. 4, 2864640, pp. 328-339. https://doi.org/10.1109/TG.2018.2864640

TY - JOUR

T1 - The ASC-inclusion perceptual serious gaming platform for autistic children

AU - Marchi, Erik

AU - Schuller, Björn

AU - Baird, Alice

AU - Baron-Cohen, Simon

AU - Lassalle, Amandine

AU - O'Reilly, Helen

AU - Pigat, Delia

AU - Robinson, Peter

AU - Davies, Ian

AU - Baltrušaitis, Tadas

AU - Adams, Andra

AU - Mahmoud, Marwa

AU - Golan, Ofer

AU - Fridenson-Hayo, Shimrit

AU - Tal, Shahar

AU - Newman, Shai

AU - Meir-Goren, Noga

AU - Camurri, Antonio

AU - Piana, Stefano

AU - Bölte, Sven

AU - Sezgin, Metin

AU - Alyuz, Nese

AU - Rynkiewicz, Agnieszka

AU - Baranger, Aurelie

N1 - Publisher Copyright: © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

PY - 2019/12

Y1 - 2019/12

N2 - “Serious games” are becoming extremely relevant to individuals who have specific needs, such as children with an autism spectrum condition (ASC). Often, individuals with an ASC have difficulties in interpreting verbal and nonverbal communication cues during social interactions. The ASC-Inclusion EU-FP7 funded project aims to provide children who have an ASC with a platform to learn emotion expression and recognition, through play in the virtual world. In particular, the ASC-Inclusion platform focuses on the expression of emotion via facial, vocal, and bodily gestures. The platform combines multiple analysis tools, using onboard microphone and webcam capabilities. The platform utilizes these capabilities via training games, text-based communication, animations, video, and audio clips. This paper introduces current findings and evaluations of the ASC-Inclusion platform and provides detailed description for the different modalities.

AB - “Serious games” are becoming extremely relevant to individuals who have specific needs, such as children with an autism spectrum condition (ASC). Often, individuals with an ASC have difficulties in interpreting verbal and nonverbal communication cues during social interactions. The ASC-Inclusion EU-FP7 funded project aims to provide children who have an ASC with a platform to learn emotion expression and recognition, through play in the virtual world. In particular, the ASC-Inclusion platform focuses on the expression of emotion via facial, vocal, and bodily gestures. The platform combines multiple analysis tools, using onboard microphone and webcam capabilities. The platform utilizes these capabilities via training games, text-based communication, animations, video, and audio clips. This paper introduces current findings and evaluations of the ASC-Inclusion platform and provides detailed description for the different modalities.

KW - AI in games

KW - Autism spectrum condition (ASC)

KW - Emotion recognition

KW - Inclusion

KW - Virtual computerized environment

UR - http://www.scopus.com/inward/record.url?scp=85089075222&partnerID=8YFLogxK

U2 - 10.1109/TG.2018.2864640

DO - 10.1109/TG.2018.2864640

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

SN - 2475-1502

VL - 11

SP - 328

EP - 339

JO - IEEE Transactions on Games

JF - IEEE Transactions on Games

IS - 4

M1 - 2864640

ER -

The ASC-inclusion perceptual serious gaming platform for autistic children

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this