Glossary/Benchmark/Run-to-run variance
Run-to-run variance
Run-to-run variance is the spread in results when the same benchmark is repeated unchanged, and it sets the smallest difference you can trust.
Also known as variance, run variance, repeatability
Measured by SystemCheck
Part of the four-stage run: a 60-second sustained CPU load reporting throughput, decay and consistency; a memory stage reporting bandwidth and latency; a calibrated GPU raster load reporting frame time, pacing and decay; and a path tracer reporting samples per second. The paragraphs below say which metric carries this term.
No two runs are identical. Starting temperature, OS scheduling, background services, memory layout, and browser JIT state all shift slightly. A healthy desktop typically lands within 1-3%; a laptop with aggressive thermal management can vary 5-10% depending on how warm it was when you started.
This number defines your measurement resolution. If your variance is 4%, a 3% improvement after a tweak is indistinguishable from noise no matter how much you want it to be real. Establishing variance first (three to five runs under identical conditions) is the step that separates useful tuning from superstition.
The practical protocol with SystemCheck is to run three times back to back, note the spread, and treat that spread as your noise floor. Then compare any change against it. SystemCheck's consistency figure describes variation within a single run; run-to-run variance describes variation between runs, and you need both.
Related terms