## Archive for January 2, 2008

### Judging a theoretical speculation by data

It strikes me that using theoretical priors to evaluate the importance of a discovery is, to a certain extent, putting the cart before the horse. Canonically, we would like to collect data *impartially*, and then confront our hypotheses with those data. Ideally, we would place all hypotheses on an equal footing (which is one reason why null-hypothesis tests are bad), and let the data tell us which ones to discard.

We all know that HEP does not work that way, not quite. First, there are so many facts that we need to agglomerate them in some quantitative way. The chi-squared or likelihood tests based on precision electroweak measurements (see the “blue-band” plot below) or the constraints on the unitary matrix from key measurements in the flavor sector are good examples of this. Furthermore, our hypotheses are really models and theories, some more speculative, some less, which are already based on a chain of reasoning and experimental results, and rarely can be rejected with just a few more data. (There are exceptions, though – the non-observation of a SM-like Higgs boson with a mass below 150 GeV would be very bad for minimal low-energy supersymmetry.) So there is a huge difference between overthrowing the standard model, and ruling out the theory of universal extra dimensions (attractive though it may be). We expect one to (continue to) succeed, and the other we hope to succeed, but expect to fail. As a community, we have our “priors” as to which theory is more likely to be correct.

To take the step of constructing a function which represents these priors and using it to place numbers on discoveries, or *potential* discoveries, goes well beyond common practice or thinking. It seems to me like an unwanted and perhaps dangerous feed-back loop. It supposes that we can quantify the very things that we can’t anticipate by the level at which we can’t anticipate them. Since this is clearly impossible, we should substitute deviation from our priors for how successfully an experiment guides us to a better understanding. Aside from overlooking the role played by measurement, this approach would not have helped us to overcome the quagmire of the Bootstrap Model (a.k.a. particle democracy) or to move from Regge Theory to quantum field theory, and ultimately, to QCD.

I do believe strongly in searching for new physics (deviations from the standard model) in a way that is broad and unfettered by theoretical prejudice, so I am sympathetic to many of Bruce’s ideas. But I don’ think we can quantify our potential results in the manner he has suggested.