Collecting Better Training Data using Biased Agent Policies in Negotiation Dialogues

Vasily Konovalov, Oren Melamud, Ron Artstein, Ido Dagan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


When naturally occurring data is characterized by a highly skewed class distribution, supervised learning often benefits from reducing this skew. Human-agent dialogue data is commonly highly skewed when using standard agent policies. Hence, we suggest that agent policies need to be reconsidered in the context of training data collection. Specifically, in this work we implemented biased agent policies that are optimized for data collection in the negotiation domain. Empirical evaluations show that our method is successful in collecting a reasonably balanced corpus in the highly skewed Job-Candidate domain. Furthermore, using this balanced corpus to train a negotiation intent classifier yields notable performance improvements relative to naturally distributed data.
Original languageAmerican English
Title of host publicationProceedings of WOCHAT, the Second Workshop on Chatbots and Conversational Agent Technologies
Place of PublicationLos Angeles
Number of pages12
StatePublished - 1 Sep 2016


  • Virtual Humans


Dive into the research topics of 'Collecting Better Training Data using Biased Agent Policies in Negotiation Dialogues'. Together they form a unique fingerprint.

Cite this