What should teams decide before beginning testing?

• A. Which tools to use for scheduling
• B. The target audience and segmentation strategy
• C. How to measure results, including primary and secondary metrics
• D. The duration and number of variants

Answer: (C)

Deciding how to measure results is essential before you start testing because it pins down what counts as a win and guides every other design choice. By selecting a primary metric that aligns with the business goal (for example, conversion rate or revenue per visitor) and a set of secondary metrics to watch for side effects (like engagement or bounce rate), you create a clear, unbiased standard for judging the test outcome. This upfront measurement plan also informs how much data you need, how long to run the test, and how to interpret any changes—ensuring you’re looking at meaningful signals rather than noise or vanity numbers. Operational details like which scheduling tools to use or how to segment audiences matter, but they flow from the measurement plan rather than defining success itself. Even the number of variants and the test duration should be driven by how large of a difference you need to detect in the chosen metrics, and the reliability you want in your result.