Abakira

Part 3. Measure, then multiply

11 Test one offer against another

Up to four versions of the same offer, split fairly, judged on revenue rather than on the rate that flatters you.

Contents 11. A/B test an offer

By now you have opinions about what works. A test replaces them with an answer. This is on the Scale plan.

What a test is here

You pick one offer in a funnel and give it up to four versions. Same position in the tree, same accept and decline branches, different content: another product, another discount, another headline.

Buyers are split between the versions by a stable draw, so the same buyer always sees the same version even if the page reloads. The discount served always matches the version shown, which sounds obvious and is the part most homemade tests get wrong.

The A/B test panel with its A and B tabs and the goal selector.
Version A is the offer you already had. B, C and D are the challengers.

Change one thing

Test a different product, or a different discount, not both. If you change two things and the challenger wins, you have learned that something worked and you will never know which.

One test per funnel at a time, which the app enforces. That restriction is doing you a favour.

Reading the verdict

The results page states a winner in a sentence, and it refuses to state one before the numbers can support it. A version ahead after 30 impressions is noise, and calling it a winner is how merchants end up adopting a worse offer with total confidence.

The test results table declaring a tie, with shown, added, net revenue and revenue per view for each version.
A verdict in plain language, or an honest wait. Never a winner declared to look decisive.

Watch for the cannibalisation warning. A version can win on acceptance rate and still bring in less money, usually because it is cheaper. The results page flags that instead of letting you celebrate a number that costs you revenue. When you see the flag, pick the version that earns more, not the one that gets clicked more.

Then adopt it

When there is a winner, one button copies it into the live offer. The test stops, and the funnel keeps running with the better content.

Set yourself a date to come back. The app reminds you after a week, and a test nobody reads is just a slower funnel.

What you should have now

One test running on your best-performing offer, using the second candidate from your chapter 5 shortlist, and a date in your calendar to go and read it.

Last updated