Tuesday, April 21, 2009

Humorous Aside: The significance of beer and statistics

Did you know that scientists regularly require each other to show that their experiments are ‘significant’? This is typically done with a particular test (nerd alert! the t-test) that was developed over 100 years ago in a Guinness beer factory. The test was created long before computers but everyone still uses it despite the fact that if you collect many more samples than you could have by hand, the test will almost certainly give you the ‘significance’ that you’re looking for in your experiments.

The test is like asking the question: Did our experiments do better than randomness? If so we call that ‘significant’ and our work is done – never mind that doing better than randomness is a very low bar to set for ourselves.