What is the minimum recommended duration for an A/B test to achieve a 95% confidence level?

• A. One day
• B. Three days
• C. At least four weeks
• D. Nine weeks

Answer: (C)

Reaching 95% confidence requires enough data to reliably estimate the effect of the change and to smooth out random fluctuations. Short runs are easily swayed by daily or weekly patterns, traffic swings, and promotions, which can make a false impression of significance or hide a real effect. Running the test for at least four weeks captures multiple weeks of user behavior, reduces the impact of weekly seasonality, and gives a larger, more stable sample to base the decision on. While the exact duration depends on traffic, baseline conversion rate, and the expected lift, four weeks is the commonly recommended minimum to aim for when targeting 95% confidence.