Algorithmic Testing: The Evolution of Experimentation
“Test Everything” screams a headline from one of the A/B testing vendors.
This is the world that marketers live in today… hundreds of A/B and MVT Tests, statistical significance, traffic allocation, optimize, rinse and repeat!
But what if that advice is outdated, inapplicable to your business or so widely peanut-buttered across everything that its being used as a substitute to your human intuition and experience? What if it doesn’t apply to ecommerce sites at all?!
The world retail and brand marketers operate in today is too complex to tackle with manual testing tools alone. There aren’t that many people in most teams to tackle every different flow for every individual. If if there were (in some land of unicorns), there simply isn’t enough traffic.
So how do you ensure that every single customer gets a unique, tailored shopping experience without the need to instrument experiences manually with segments?
To meet the challenge, RichRelevance provides clients with a spectrum of native testing tools – from traditional A/B and MVT tests to full-fledged continuous algorithmic testing and optimization.
What is Algorithmic Testing?
Algorithmic testing is the next generation of A/B testing and experimentation that makes use of data science with continuous, always-on testing for improving conversions.
It’s a critical need if you have any form of a cart at the end of your customer journey.
How does Algorithmic Testing Differ?
The ability to continually test every single digital interaction with a focus on revenue conversion, and self-optimize to an objective function with the use of AI/Machine Learning to scale to every individual.
Algorithmic testing is a core tenet of hyper-personalization.
How it Works
Algorithmic testing is driven by a two-tier AI decisioning engine, the Experience Optimizer, unique to RichRelevance. The Experience Optimizer automatically, tests and picks the best experience for each individual, in real time while factoring in context.
Customers get to choose from a library of over 150+ pre-defined strategies that have been fine-tuned over time with the best brands in the Retail industry. These strategies are a part of Xen AI and are built using a spectrum of algorithms – from statistical to machine learning to deep learning, such as natural language processing or NLP.
The Experience Optimizer is continuously testing, pitting multiple strategies against each other competitively, and predicting the most likely winner for this particular interaction all in real-time. It is looking at metrics such as RPS, RPV and CTR. Thus it explores new content or recommendation strategies to determine the best performing algorithm for the visitor.
It also has an exploration mode, fully configurable by the business, which ensures that results never hit a plateau, and instead continue to grow over time.
Clients can add new content or strategies into the mix and let Experience Optimizer determine when they should be used.
Marketers can also use RichRelevance’s native A/B and MVT tools to test specific content, placements, strategies, rules and even the merchandising of category and browse pages. Clients can target the tests and allocate traffic toward specific segments or all visitors, and unlike traditional A/B testing tools, each visitor is treated uniquely with algorithmic tests.
Clients can test strategies; optimization metrics for different areas of the funnel; and test other configuration changes, campaigns, or promotions.
They can then monitor the results via the insights dashboard or via the Experience Browser. RichRelevance automatically shows the results across multiple metrics, and the user does not have to include those metrics in their test. Results can be filtered by segment and other dimensions. Also, the impact of outliers can be gauged with outlier filtering.
In summary, algorithmic testing is the next evolution of experimentation, and needs to work in conjunction with your regular A/B testing tools. Don’t be misled into thinking that if you have one, you have the other.
Learn more about how RichRelevance can help your business.
