26. Let’s try to learn from our models

In this part of the book we look a little closer at how we can interpret the results of our supervised learning models. In particular we’ll make the distinction between knowing how our models are making predictions, and whether those patterns the model finds useful are causal in some way. We’ll start off with a real world example of trying to understand if there is a systematic bias in acceptance rates by gender at a prominent university. We’ll show that getting the best predictive model won’t really help us answer the question. This is meant to help us communicate the difference between how the model is making predictions and causal interpretations to non-specialists. We’ll also take a look at a more experimental approach, called A-B testing, that tries to get at causal questions using machine learning and experimental design.