5-sigma signals come and go!

Slowly I am managing to return to the land of physics blogging, and much of what I see now is great!

One item which strikes me is a post written in Charm, etc. in which the author describes how a very nice 5-sigma pentaquark signal came and went! This may well serve as a cautionary tale for all of us at high-energy colliding beam experiments. If this peak had been the Higgs boson, the collaboration(s) might have been delighted to announce discovery. (Some would argue that the original W boson and top quark discoveries were on shakier ground, statistically speaking.)

I think it is great that the CLAS Collaboration wrote a article demonstrating the results from their first set of data, and the second, which is fives times larger than the first. This plot sums up the situation:

The points represent the first data sample, on the basis of which a 5-sigma significance was claimed. The solid line represents the second data sample, which is more than five times larger than the first, which contradicts that claim. Both data samples were taken with the same apparatus under the same conditions, by the same people. The collaboration wrote a paper exploring the statistical issues involved (see arXiv:0709.3154). See the clear and concise article for details, or or Charm’s original peak-finding post.