A statistical decision-making model for choosing among multiple alternatives

Automated agents often have several alternatives to choose from in order to solve a problem. Usually the agent does not know in advance which alternative is the best one, so some exploration is required. However, in most cases there is a cost associated with exploring the domain, which must be minimized in order to be worthwhile. We concentrate on cases where the agent has some prior knowledge about each alternative, which is expressed in terms of units of information. A unit of information about an alternative is the result of choosing the alternative - for example, in the e-commerce domain one unit of information can be a customer's impression or feedback about a supplier; in the heuristic domain one unit of information can be the observed result of running one simulation with a given heuristic function. In our environments the agent has a-priori only a small number of units of information about each alternative, and it would like to use this knowledge in deciding between its alternatives. Nevertheless, since the agent has only a limited number of units of information, deciding between the alternatives solely based on these units may be risky. In extreme cases, they can even mislead the agent to choose the worst alternative rather than the best one.