Improving task-oriented dialogue systems in production with conversation logs

Alon Jacovi, Ori Bar El, Ofer Lavi, David Boaz, David Amid, Inbal Ronen, Ateret Anaby-Tavor

Research output: Contribution to journalConference articlepeer-review


In this work we propose a solution to a significant limitation of task-oriented dialogue systems - their inability to learn and improve over time during deployment. Although current popular task-oriented systems are implemented as rule-based execution graphs, the available solutions for improvement incorporate neural network modules, either fully or partially, despite the poor performance of neural architectures for the task-oriented use-case. We present an algorithm to modify the graph-based system directly, in a manner which improves the system automatically and is simultaneously easy to understand by the system expert. To our knowledge, this is the first method of this type towards automatically improving a dialogue system's coverage in production, without additional explicit labels. Though the system is still evidential, our experiments already show promising results in its ability to usefully modify an existing dialogue system, while improving its coverage.

Original languageEnglish
JournalCEUR Workshop Proceedings
StatePublished - 2020
EventKDD 2020 Workshop on Conversational Systems Towards Mainstream Adoption, KDD-Converse 2020 - Virtual, Online, United States
Duration: 24 Aug 2020 → …

Bibliographical note

Publisher Copyright:
© 2020 Copyright held by the owner/author(s).


  • Closed domain
  • Dialogue systems
  • Machine learning
  • Rule based systems
  • Task oriented
  • Virtual agent


Dive into the research topics of 'Improving task-oriented dialogue systems in production with conversation logs'. Together they form a unique fingerprint.

Cite this