Just received a newsletter from someone I follow (and admire).
The subject line was: Improving my sales conversion by 39.7%
Like the sucker I am, I dived in, keen for the learnings…
Here’s what the results were based on:
- 301 page views, across two variations (yes, that’s just 150 page views on each)
- One had 6 conversions
- The other had 8 conversions
- That’s it!
This is such a small test size it’s bordering on useless.
The only time you’d start responding on a test size this small is if one of them had had 0 sales – that might be telling you something.
But as it is, this is misleading.
I’m hoping he shows the results once it’s hit more than 10,000 page views, or more than 200 sales.
Key takeaway: don’t jump to insignificant conclusions.
