Uplift (or incremental lift) modeling is generally harder to execute than response modeling. While response follows known customer traits (demographics, lifestage, transience, change in circumstances), uplift can be dependent on variables not commonly used in response modeling. After watching a few failed attempts to create uplift models, I can identify the most common barrier in creating an valid uplift model: marketing programs that are ineffective.
Why would it matter? Let’s review in short what an uplift model is. The dependent variable of the uplift model is the difference in response between test (treatment) and control groups. The independent variables can be all sorts of things that impact the effectiveness of the action, from demographics of the segments to variables related to the treatment itself. For the math behind the model to work, we need to see variable rates of incremental response for different levels of independent variables. If there is no variance in response, the model fails because no independent variable impacts the outcome.
Let’s consider two charts below. Which experiment is likely to produce a good uplift model?
Program #1
Program #2
Program #1 delivers no incremental lift over control. It is a poor program, and trying to figure out which factors drive positive change is impossible, because there is no positive change.
Program #2 may not be a food fit for some segments, but there are strong positive results in other segments. This suggests that we have an output that not only positive, but has some variability that changes in predictable ways. This program is a good candidate for uplift modeling.
Why would a situation where there is no lift over control happen? Based on my experience, this happens either because the marketing channel you chose is not a good fit with your business or because you have already oversaturated the market with communications.
The second case is common in large companies. Reliance on direct response gets the market oversatirated to the point of zero marginal sales, so the first attempt at uplift measurement fails miserably. When faced with a situation like that, the amount of marketing communication (most often – communication frequency) should be taken into account when measuring uplift, and that’s the variable that needs to be optimized first. Unfortunately, most of the time, this variable is not being considered for the analysis or the analytics team has no means to influence this variable.
When marketers ask “Which segment do I need to mail?”, the right answer is often “At this frequency, none”.