When an A/B test yields a clear winner, what should be inferred?

• A. The better performing variant should be chosen for deployment
• B. The test should be repeated with the same variants
• C. The result should be ignored due to randomness
• D. The worse performing variant should be scaled

Answer: (A)

When an A/B test shows a clear winner with statistical significance, you should deploy the higher-performing variant. The goal of A/B testing is to distinguish real improvements from random noise, so a winner indicates the variant is likely to deliver better results in production, such as more conversions or revenue. Re-running the test with the same variants wastes time and resources and isn’t necessary once significance is reached. Ignoring the result because of perceived randomness goes against what the test is designed to reveal. Scaling the worse performing variant would reduce overall performance, so that’s not a rational action.