In my first blog post I introduced the general framework of Bayesian $$A/B$$ testing, and went into gory details on how to make it happen with binary data: 0, 1, 0, 1, 0, 0, 0…
This post is all about dealing with Gaussians in a Bayesian way; it’s a prelude to the next post: “Bayesian A/B Testing with a Log-Normal Model.” So don’t go analyzing the non-binary data from your $$A/B$$ tests until you read both blog posts!
Here at RichRelevance we regularly run live tests to ensure that our algorithms are providing top-notch performance. Our RichRecs engine, for example, displays personalized product recommendations to consumers, and the $$A/B$$ tests we run pit our recommendations against recommendations generated by a competing algorithm, or no recommendations at all.