List Building
Email Test Sample Size: The Real Math
Most email A/B tests cannot detect the difference they report finding. Here is the arithmetic, and what to do when your list is too small for it.
Oct 15, 20265 min read
Tag
#sample size
2 articles tagged with sample size.
Metrics & Benchmarks
How to A/B Test Emails Properly
Most email A/B tests are not tests — they are noise read as signal. This guide covers writing a real hypothesis, choosing variables by impact, splitting the list correctly, the sample sizes needed to detect a given difference, why open rate is the wrong success metric, and how to build a testing program instead of a testing habit.
Sep 18, 20269 min read