How to Create Effective Control Groups

  1. Control group must be representative of the treated group. This is most commonly achieved by random assignment of customers or subjects.
  2. In cases when random assignment is not possible, for example, your group has to cover the whole DMA or organizational unit, you want to find several units that are similar on parameters that impact your outcome/measure. In this case, it is best incorporate pre-test trends into the understanding of the test period outcome.
  3. If using matched groups is not possible for legal or organizational reasons, your best bet is to transform your control group to match your treatment group by selecting/dropping certain members and/or reweighing the outcome. This approach is called synthetic control group. This is the most analytically intensive way to measure test performance.
  4. The role of a control group is to show what would have happened given everything else that is going on in the marketplace except for the treatment you are holding them out from. There is no need to “rest” either of the groups or keep them clean from any program that impacts both test and control groups. This is one of the most important principles of the test design.
  5. Control group measures only the type of treatment withheld. If you withhold control from more than one treatment, it becomes a control for all of the withheld treatment put together.
  6. The measurement of the outcome should be identical for both groups

Leave a Reply