Dynamic Allocation Optimization in A/B-Tests Using Classification-Based Preprocessing

doi:10.1109/TKDE.2021.3076025

Dynamic Allocation Optimization in A/B-Tests Using Classification-Based Preprocessing

2021-01-01

5

OA

AI

P

Pierre Gançarski

M

Myriam Maumy‐Bertrand

DOI：10.1109/TKDE.2021.3076025

Original

Save

View PDF

Abstract

En 中文

An A/B-Test evaluates the impact of a new technology by running it in a real production environment and testing its performance on a set of items. Recent development efforts around A/B-Tests revolve around dynamic allocation. They allow for quicker determination of the best variation (A or B), thus saving money for the user. However, dynamic allocation by traditional methods requires certain assumptions, which are not always valid in reality. This is often due to the fact that the populations being tested are not homogeneous. This article reports on a new reinforcement learning methodology which has been deployed by the commercial A/B-Test platform AB Tasty. We provide a new method that not only builds homogeneous groups of users, but also allows the best variation for these groups to be found in a short period of time. This article provides numerical results on AB Tasty's data, in addition to public datasets, tha demonstrate an improvement over traditional methods.

Keywords:

A/B-TEST

bandit strategies

UCB strategies

conditional inference tree

non linear bandit

regret minimisation

Dynamic Allocation Optimization in A/B-Tests Using Classification-Based Preprocessing

Abstract

AI Summary

Journal

Organization