Normally running a front-end A/B test I just reach for Google Experiments which are easy to setup in Google Analytics, and it does all the heavy lifting around user selection, statistics and confidence rating on the results for you.
However, Google Experiments only work if the front-end test is all about measuring the goal completion in the same visit/session. The test I needed to run on the front-end, didn’t fit this use case; Instead I needed to:-
- A/B test how showing a content creator one of two variants of a content editor would affect the quality of content produced.
- Measure the product quality by how content consumers read and interacted with the content when they visited the site at some point later, ideally leading to a purchase. (time on page, bounce rate, conversion rate)
So it was go it alone with experiment selection, in this case simply selecting even userid’s to see the new version, while keeping users with odd numbered userid’s on the existing content creation editor. This was tracked in a MySQL table.