Statistical Significance

Statistical significance shows whether a difference in quantitative results—like task completion times between two designs—is likely due to the design itself, not random chance. It helps determine if a change truly impacted user behavior or if the observed effect could have happened by coincidence.

Statistical significance isn’t a metric—it doesn’t give you a number to track. Instead, it’s a way to test whether the difference between two metrics is likely real. In UX research, this is usually done using a p-value.

A p-value shows the chance of seeing the results you got (or something more extreme) if there were actually no real difference. A p-value under 0.05 (or 5%) means the result is statistically significant—so it probably didn’t happen by chance.