Abstract
When a new treatment is considered for use, whether a pharmaceutical drug or a search engine ranking algorithm, a typical question that arises is, will its performance exceed that of the current treatment? The conventional way to answer this counterfactual question is to estimate the effect of the new treatment in comparison to that of the conventional treatment by running a controlled, randomized experiment. While this approach theoretically ensures an unbiased estimator, it suffers from several drawbacks, including the difficulty in finding representative experimental populations as well as the cost of running randomized trials. Moreover, such trials neglect the huge quantities of available control-condition data, which in principle can be utilized for the harder task of predicting individualized effects. In this paper we propose a discriminative framework for predicting the outcomes of a new treatment from a large dataset of the control condition and data from a small (and possibly unrepresentative) randomized trial comparing new and old treatments. Our learning objective, which requires minimal assumptions on the treatments, models the relation between the outcomes of the different conditions. This allows us to not only estimate mean effects but also to generate individual predictions for examples outside the small randomized sample. We demonstrate the utility of our approach through experiments in three areas: search engine operation, treatments to diabetes patients, and market value estimation of houses. Our results demonstrate that our approach can reduce the number and size of the currently performed randomized controlled experiments, thus saving significant time, money and effort on the part of practitioners.
Original language | English |
---|---|
Title of host publication | 26th International World Wide Web Conference 2017, WWW 2017 Companion |
Publisher | International World Wide Web Conferences Steering Committee |
Pages | 602-609 |
Number of pages | 8 |
ISBN (Electronic) | 9781450349147 |
DOIs | |
State | Published - 2017 |
Externally published | Yes |
Event | 26th International World Wide Web Conference, WWW 2017 Companion - Perth, Australia Duration: 3 Apr 2017 → 7 Apr 2017 |
Publication series
Name | 26th International World Wide Web Conference 2017, WWW 2017 Companion |
---|
Conference
Conference | 26th International World Wide Web Conference, WWW 2017 Companion |
---|---|
Country/Territory | Australia |
City | Perth |
Period | 3/04/17 → 7/04/17 |
Bibliographical note
Publisher Copyright:© 2017 International World Wide Web Conference Committee (IW3C2), published under Creative Commons CC BY 4.0 License.