Mandel, T.; Liu, Y.-E.; Brunskill, E.; Popović, Z. The Queue Method: Handling Delay, Heuristics, Prior Data, and Evaluation in Bandits. AAAI 2015, 29.