Inference and Insights
This came up in a recent meeting with the Product team at work. We were talking about data and how in recent times, we have gotten a ton of it. We were also talking about how some of this data was actionable, and some was just there. In some areas, data was just becoming very hairy and unmanageable.We spoke about Inferences vs Insights.
Inferences : You look at all your data. You crunch what you require. You ignore/delete what does not matter to you. You separate out key metrics and secondary metrics.
Insights : These are trends and patterns that you spot in your inferences. You then distill them, cross-reference them, and derive insights out of them. Because of a drop in metric x1, and a corresponding metric x2, the resulting derived metric had a double increase to x3. Add to it, seasonal variations, of a corresponding period in the last year, you get an insight as to whether what you have 'built' has been worth it or not.
Getting inferences is more or less a science, but distilling insights is almost an art. It takes experience, and an open perspective to effectively derive insights.
Insights also serve another purpose. They serve as basis for hypothesis, and experiments. You get insights using a subset of data, for a period of time, which leads you to make a hypothesis, which in turn you experiment for a different period of time (or a different subset of data), to prove it.
The above was the effective crux of the discussion that we had, and I thought this might be valuable as food-for-thought (if not anything else), for the few folks who read my blog. img src: https://www.linkedin.com/pulse/big-data-insights-found-unexpected-places-austin-wentzlaff