What can A/B tests reveal that other methods might not?

• A. They can isolate and measure the impact of a single variable on customer behavior
• B. They always yield qualitative results
• C. They cannot establish causality
• D. They require large sample sizes only

Answer: (B)

A/B testing is about running controlled experiments that isolate a single variable and measure its direct impact on customer behavior. By randomly assigning visitors to two versions and keeping everything else constant, any difference in outcomes—like conversion rate or revenue per user—can be attributed to that specific change. This gives you quantitative evidence of the effect and its magnitude, along with statistical significance, rather than just opinions or correlations.

Qualitative results describe perceptions or feelings, not actual actions, so A/B tests provide numerical metrics instead of only descriptive feedback. Properly designed tests can establish causality because random assignment helps rule out confounding factors. And while having a large sample helps increase confidence, the necessary size depends on the expected effect and the desired power, not on a fixed requirement that only large samples are useful.