Improving Online Marketing Experiments with Drifting Multi-armed Bandits

Giuseppe Burtini, Jason Loeppky, Ramon Lawrence

2015

Abstract

Restless bandits model the exploration vs. exploitation trade-off in a changing (non-stationary) world. Restless bandits have been studied in both the context of continuously-changing (drifting) and change-point (sudden) restlessness. In this work, we study specific classes of drifting restless bandits selected for their relevance to modelling an online website optimization process. The contribution in this work is a simple, feasible weighted least squares technique capable of utilizing contextual arm parameters while considering the parameter space drifting non-stationary within reasonable bounds. We produce a reference implementation, then evaluate and compare its performance in several different true world states, finding experimentally that performance is robust to time drifting factors similar to those seen in many real world cases.

Paper Citation

in Harvard Style

Burtini G., Loeppky J. and Lawrence R. (2015). Improving Online Marketing Experiments with Drifting Multi-armed Bandits . In Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-758-096-3, pages 630-636. DOI: 10.5220/0005458706300636

in Bibtex Style

@conference{iceis15,
author={Giuseppe Burtini and Jason Loeppky and Ramon Lawrence},
title={Improving Online Marketing Experiments with Drifting Multi-armed Bandits},
booktitle={Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2015},
pages={630-636},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005458706300636},
isbn={978-989-758-096-3},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - Improving Online Marketing Experiments with Drifting Multi-armed Bandits
SN - 978-989-758-096-3
AU - Burtini G.
AU - Loeppky J.
AU - Lawrence R.
PY - 2015
SP - 630
EP - 636
DO - 10.5220/0005458706300636