What is Testing Optimization?

Through optimization testing, we try to discover which webpage or email message will perform best by trying each version with a random sample of target prospects.

We decide during test design how much difference we must see to warrant making a change, and what level of confidence is required to consider the data valid using statistical methods.

But is sample size the only factor that should be considered when assessing the validity of test results?