Exploitation vs. exploration: Choosing a supplier in an environment of incomplete information

Rina Azoulay-Schwartz, Sarit Kraus, Jonathan Wilkenfeld

Research output: Contribution to journalArticlepeer-review

42 Scopus citations


An agent operating in the real world must often choose between maximizing its expected utility according to its current knowledge about the world and trying to learn more about the world, since this may improve its future gains. This problem is known as the trade-off between exploitation and exploration. In this research, we consider this problem in the context of electronic commerce. An agent intends to buy a particular product (goods or service). There are several potential suppliers of this product, but they differ in their quality and in the price charged. The buyer cannot observe the average quality of each product, but he has some knowledge about the quality of previous goods purchased from the suppliers. On the one hand, the buyer is motivated to buy the goods from the supplier with the highest expected product quality, deducting the product price. However, when buying from a lesser known supplier, the buyer can learn about its quality and this can help him in the future, when he will purchase more products of this type. We show the similarity of the suppliers problem to the k-armed bandit problem, and we suggest solving the suppliers problem by evaluating Gittins indices and choosing the supplier with the optimal index. We demonstrate how Gittins indices are calculated in real world situations, where deals of different magnitudes may exist, and where product prices may vary. Finally, we consider the existence of suppliers with no history and suggest how the original Gittins indices can be adapted in order to consider this extension.

Original languageEnglish
Pages (from-to)1-18
Number of pages18
JournalDecision Support Systems
Issue number1
StatePublished - Oct 2004

Bibliographical note

Funding Information:
Jonathan Wilkenfeld (PhD Political Science, Indiana University, 1969) is Professor and Director of the Center for International Development and Conflict Management, University of Maryland. He is a specialist in foreign policy decision making and crisis behavior, as well as simulation and experimental techniques in political science. He is the author of numerous articles and seven books, the most recent A Study of Crisis (with Michael Brecher, 1997, 2000) and Negotiating a Complex World (with Brigid Starkey and Mark Boyer, 1999). His research has been supported over the years by the National Science Foundation, the US Department of Education, the US Institute of Peace, Sun Microsystems, NCR and IBM, among others. His current work focuses on the combined use of aggregate and experimental techniques to study the mediation process in international crisis negotiations, and how decision makers learn from previous crisis experience.

Funding Information:
She has published over 120 papers in leading journals and major conferences and is an author of the book Strategic Negotiation in Multi-Agent Environments (2001) and a co-author of a book on Heterogeneous Active Agents (2000), both published in MIT Press. Her research has been supported over the years by GIF, NSF, GM, IBM and NDS, among others. She is an Associate Editor of the Annals of Mathematics and Artificial Intelligence Journal and on the Editorial Board of the Artificial Intelligence Journal and the Journal of Autonomous Agents and Multi-Agent Systems.

Funding Information:
This material is based upon work supported in part by NSF under grants no. IIS-9820657 and IIS-0208608.


  • Choosing a supplier
  • Decision making
  • Electronic commerce
  • Incomplete information


Dive into the research topics of 'Exploitation vs. exploration: Choosing a supplier in an environment of incomplete information'. Together they form a unique fingerprint.

Cite this