Journal
15TH ACM CONFERENCE ON RECOMMENDER SYSTEMS (RECSYS 2021)
Volume -, Issue -, Pages 372-379Publisher
ASSOC COMPUTING MACHINERY
DOI: 10.1145/3460231.3474231
Keywords
ad design; off-policy evaluation; bandit; reinforcement learning
Ask authors/readers for more resources
The paper proposes an alternative method to evaluate new algorithms by predicting their performance using historical data, validates the method through a simulation experiment and an advertisement design, and shows smaller mean squared errors compared to state-of-the-art methods.
Efficient methods to evaluate new algorithms are critical for improving interactive bandit and reinforcement learning systems such as recommendation systems. A/B tests are reliable, but are time- and money-consuming, and entail a risk of failure. In this paper, we develop an alternative method, which predicts the performance of algorithms given historical data that may have been generated by a different algorithm. Our estimator has the property that its prediction converges in probability to the true performance of a counterfactual algorithm at a rate of root N, as the sample size N increases. We also show a correct way to estimate the variance of our prediction, thus allowing the analyst to quantify the uncertainty in the prediction. These properties hold even when the analyst does not know which among a large number of potentially important state variables are actually important. We validate our method by a simulation experiment about reinforcement learning. We finally apply it to improve advertisement design by a major advertisement company. We find that our method produces smaller mean squared errors than state-of-the-art methods.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available