High Fidelity - Listening Tests

Listening Tests

Blind tests refer to experiments where researchers have sighted knowledge on the tested (audio) components, but not individuals undergoing the experiments. In a double-blind experiment, neither the individuals nor the researchers know who belongs to the control group and the experimental group. Only after all the data has been recorded (and in some cases, analyzed) do the researchers learn which individuals are which. A commonly used variant of this test is the ABX test. A subject is presented with two known samples (sample A, the reference, and sample B, an alternative), and one unknown sample X, for three samples total. X is randomly selected from A and B, and the subject identifies X as being either A or B. Although there is no way to prove that a certain lossy methodology is transparent, a properly conducted double-blind test can prove that a lossy method is not transparent.

Scientific double-blind tests are often used to disprove certain audio components (such as expensive, exotic cables) have any perceivable effect on sound quality. These tests are not accepted by some "audiophile" magazines in their evaluation of audio equipment, such as Stereophile and The Absolute Sound. John Atkinson (editor), current editor of Stereophile, stated (in a 2005 July editorial named Blind Tests & Bus Stops) that he once purchased a solid-state amplifier, the Quad 405, in 1978 after blind tests, but came to realize months later that "the magic was gone" until he replaced it with a tube amp. Robert Harley of The Absolute Sound wrote, in a 2008 editorial (on Issue 183), that "blind listening tests fundamentally distort the listening process and are worthless in determining the audibility of a certain phenomenon."

Doug Schneider, editor of the online Soundstage network, refuted this position with two editorials in 2009. He stated: "Blind tests are at the core of the decades’ worth of research into loudspeaker design done at Canada’s National Research Council (NRC). The NRC researchers knew that for their results to be credible within the scientific community and to have the most meaningful results, they had to eliminate bias, and blind testing was the only way to do so." Many Canadian companies such as Axiom, Energy, Mirage, Paradigm, PSB and Revel use blind testing extensively in designing their loudspeakers. Many audio insiders like Sean Olive of Harman International share this view.