Efficient bidding strategies for Cliff-Edge problems

Rina Azoulay, Ron Katz, Sarit Kraus

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


In this paper, we propose an efficient agent for competing in Cliff-Edge (CE) and simultaneous Cliff-Edge (SCE) situations. In CE interactions, which include common interactions such as sealed-bid auctions, dynamic pricing and the ultimatum game (UG), the probability of success decreases monotonically as the reward for success increases. This trade-off exists also in SCE interactions, which include simultaneous auctions and various multi-player ultimatum games, where the agent has to decide about more than one offer or bid simultaneously. Our agent competes repeatedly in one-shot interactions, each time against different human opponents. The agent learns the general pattern of the population's behavior, and its performance is evaluated based on all of the interactions in which it participates. We propose a generic approach which may help the agent compete against unknown opponents in different environments where CE and SCE interactions exist, where the agent has a relatively large number of alternatives and where its achievements in the first several dozen interactions are important. The underlying mechanism we propose for CE interactions is a new meta-algorithm, deviated virtual learning (DVL), which extends existing methods to efficiently cope with environments comprising a large number of alternative decisions at each decision point. Another competitive approach is the Bayesian approach, which learns the opponents' statistical distribution, given prior knowledge about the type of distribution. For the SCE, we propose the simultaneous deviated virtual reinforcement learning algorithm (SDVRL), the segmentation meta-algorithm as a method for extending different basic algorithms, and a heuristic called fixed success probabilities (FSP). Experiments comparing the performance of the proposed algorithms with algorithms taken from the literature, as well as other intuitive meta-algorithms, reveal superiority of the proposed algorithms in average payoff and stability as well as in accuracy in converging to the optimal action, both in CE and SCE problems.

Original languageEnglish
Pages (from-to)290-336
Number of pages47
JournalAutonomous Agents and Multi-Agent Systems
Issue number2
StatePublished - Mar 2014

Bibliographical note

Funding Information:
Acknowledgments We thank the anonymous reviewers for their helpful remarks which enabled us to significantly improve the quality of this paper. This work is supported in part by the following Grants: ERC Grant # 267523, MURI Grant Number W911NF-08-1-0144, ARO Grants W911NF0910206 and W911NF1110344, MOST # 3-6797 and the Google Inter-university Center for Electronic Markets and Auctions.


  • Cliff-Edge problems
  • Dynamic pricing
  • Human-agent interactions
  • Reinforcement learning
  • Ultimatum game
  • Virtual learning


Dive into the research topics of 'Efficient bidding strategies for Cliff-Edge problems'. Together they form a unique fingerprint.

Cite this