Insights

Understand Bias

Are you pitting winners against losers? See how selection bias can mislead you.

Controlled Experiments

Too much data? Learn how to use experiment design to create simple and elegant analytics.

Churn reduction

Why do churn reduction programs fail? See how data can make us pursue wrong targets for churn reduction.

See all insights

Find out more

Find out more about my background

Want to talk?

Latest Thoughts

Why do uplift models fail?

Uplift (or incremental lift) modeling is generally harder to execute than response modeling. While response follows known customer traits (lifestage, transience, change in circumstances), uplift can be dependent on variables not commonly used in response modeling.After watching a few failed attempts at creating uplift models, I can identify the most common barrier in creating an valid uplift model: marketing programs that are extremely ineffective. Why would it matter? Let’s review in short what an uplift model is. The dependent variable of the uplift model is the difference in response between test (treatment) and control groups. The independent variables can be […]

Avinash Kaushik on controlled experiments

Back in 2011 Avinash wrote an excellent blog post about the value of controlled experiments called Measuring Incrementality: Controlled Experiments to the Rescue! In his post he describes the application of a test vs control design to breaking down how different channels of communication perform separately vs when used together. This is a good application of the controlled experiment methods. Avinash is clearly very impressed with the methodology. I was really surprised that he called it “advanced”. As far as methods of measurement goes, simple test vs control design like he describes does not require the use of advanced methods, which is […]

How to Create Effective Control Groups

Control group must be representative of the treated group. This is most commonly achieved by random assignment of customers or subjects. In cases when random assignment is not possible, for example, your group has to cover the whole DMA or organizational unit, you want to find several units that are similar on parameters that impact your outcome/measure. In this case, it is best incorporate pre-test trends into the understanding of the test period outcome. If using matched groups is not possible for legal or organizational reasons, your best bet is to transform your control group to match your treatment group […]

See all thoughts

The is site was created by Tanya Zyabkina using instructions by NYC Tech Club. Free images by Unsplash.