Tanya Zyabkina

One of the most important principles of this design is making sure the control is representative of the treatment group. This is typically achieved by random assignment. However, sometimes random selection of customers into the groups is impossible.

we must make sure the control group mimics the behavior of the test group. This phenomena is described as groups being matched or representative of each other.

Measurement of overlapping campaigns when using controlled experiment design has been a bit of a tricky point for many controlled experiment proponents. Some claim that you have to hold other campaigns when measuring, which is a difficult proposition, especially if the goal is to measure every campaign as BAU.

In compound experiments universal control groups are used in combination individual campaign control groups, providing very powerful tools for sales attribution.