Comparing Agents' Success against People in Security Domains

Raz Lin, Sarit Kraus, Noa Agmon, Samuel Barrett, Peter Stone

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

The interaction of people with autonomous agents has become increasingly prevalent. Some of these settings include security domains, where people can be characterized as uncooperative, hostile, manipulative, and tending to take advantage of the situation for their own needs. This makes it challenging to design proficient agents to interact with people in such environments. Evaluating the success of the agents automatically before evaluating them with people or deploying them could alleviate this challenge and result in better designed agents. In this paper we show how Peer Designed Agents (PDAs) - computer agents developed by human subjects - can be used as a method for evaluating autonomous agents in security domains. Such evaluation can reduce the effort and costs involved in evaluating autonomous agents interacting with people to validate their efficacy. Our experiments included more than 70 human subjects and 40 PDAs developed by students. The study provides empirical support that PDAs can be used to compare the proficiency of autonomous agents when matched with people in security domains.

Original languageEnglish
Title of host publicationProceedings of the 25th AAAI Conference on Artificial Intelligence, AAAI 2011
PublisherAAAI press
Pages809-814
Number of pages6
ISBN (Electronic)9781577355083
StatePublished - 11 Aug 2011
Event25th AAAI Conference on Artificial Intelligence, AAAI 2011 - San Francisco, United States
Duration: 7 Aug 201111 Aug 2011

Publication series

NameProceedings of the 25th AAAI Conference on Artificial Intelligence, AAAI 2011

Conference

Conference25th AAAI Conference on Artificial Intelligence, AAAI 2011
Country/TerritoryUnited States
CitySan Francisco
Period7/08/1111/08/11

Bibliographical note

Publisher Copyright:
Copyright © 2011, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Funding

∗This research is based upon work supported in part by the U.S. Army Research Laboratory and the U.S. Army Research Office under grant number W911NF-08-1-0144 and under NSF grant 0705587. Copyright ©c 2011, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

FundersFunder number
National Science Foundation0705587
Army Research OfficeW911NF-08-1-0144
Army Research Laboratory

    Fingerprint

    Dive into the research topics of 'Comparing Agents' Success against People in Security Domains'. Together they form a unique fingerprint.

    Cite this