“The problem with that is that when your goal is to optimise the overall ROI on a campaign-to-campaign basis, there are some inherent inefficiencies in your methodology – and we’ve recognised this for some time.”
Malm references a multi-variant optimisation methodology called the “multi-armed bandit” methodology – a more complex version of A/B testing in which marketers have different proportions of different variants operating alongside each other. “Much like a horse race – if you have ten horses running down the track at Ascot, then you invest more in the horse at the front, and you invest less in the horse at the back. And over time, we can add in more horses to the race, so we’re constantly doing the champion versus challenger thing,” he explains. So, for email subject line optimisation, substitute ‘horses’ with ‘email subject line variants’.
The workload like this whatsapp number list allows both the vendor and the affiliate to focus on. Clicks are the number of clicks coming to your website’s URL from organic search results.
However, assessing performance in the context of email marketing presents some unprecedented challenges. “The delivery of response metrics – opens, clicks, conversions – are not uniform,” says Malm. “You can’t just roll out a multi-armed bandit and hope for the best. We had to do a huge amount of research and engineering work to make it feasible.”
On top of this basic methodology is the real-time language optimisation that makes Phrasee X unique. “We’ve got all of these different variants running simultaneously, being optimised in the statistically most efficient way possible; but we can then in real-time amend, augment and inject a new language into that competition,” Malm explains. “So, you’re constantly learning and constantly improving as time goes on.