Judging experiments by a priori theoretical expectations

December 29, 2007 at 7:35 pm 7 comments

Charm etc. posted an interesting discussion on Information Entropy and Experiments. The bloggers describe an attempt to evaluate the worth of experiments on a statistical basis, comparing the results they produce against a priori expectations from theory. (See arXiv:0712.3572 from Bruce Knuteson.) They point out very perceptively that this procedure relies too much, if not entirely, on those expectations from theory. One might wonder where else we should get our expectations, but this is not the point – the most important discoveries are the ones for which there was no expectation. And after the discovery was made, theory was altered radically, therefore changing the priors. See Charm etc. for a succinct discussion.

I think there is another problem with the approach. How does one evaluate the accumulated value of crucial measurements of particle properties and interactions? How much does a factor two improvement in the W mass measurement increase the value of a Tevatron experiment? What about the Bs oscillation frequency? These measurements fit in the standard model – in fact, they are important if not crucial empirical inputs to model calculations, without which one could not move beyond the standard model. I don’t see how to set a value for such information in the sense of theoretical expectations; if we discard the standard model in the next ten years, and replace it with something much better, is the value of the W mass measurement diminished or enhanced? At what point does the W mass (for example) become mundane or less than crucial, in contrast to the present time? I suppose there will be a day when the W mass can be predicted sufficiently well, or when it ceases to provide any insights into new physics, and these developments might be reflected in the nature of the theoretical priors required by this evaluation of scientific merit, but I doubt this could be made clear or concrete.

The bloggers at Charm etc. are also skeptical of this approach, interesting though it may be to talk or blog about it. But if someone takes it more seriously, then perhaps the next question would be: how do you place a quantitative value on individual, and improving, measurements, taking into account the possibility that some measurements are wrong?

Update: (5-Jan-2007) There are nice discussions of this issue at Quantum Diaries Survivor and Deep Thoughts and Silliness.


Entry filed under: Particle Physics.

After the budget debacle What is the value of measurement?

7 Comments Add your own

  • 1. Arunn  |  January 2, 2008 at 7:02 pm

    I don’t work in particle physics, but do experiments for my porous medium research. Of late I am interested in Bayesian methods in evaluating experimental results and in that context had doubts about the approach of assigning priors and the dependability of the concept of surprisal

    Expository work by G D’Agostini [arXiv link] may clear some of our doubts. It did for me. [ arXiv:physics/9811046 may be relevant to you. ]

    And best wishes for your renewed interest in blogging. I shall be on the look out for your expository on physics topics of (my) interest.


  • 2. Michael Schmitt  |  January 2, 2008 at 7:38 pm

    Hi Arunn,

    thanks for the pointer to d’Agostini’s work. Indeed his articles look very interesting, even edifying. Let’s see whether he can convert me to a Baysian (it may take a while)…


  • 3. dorigo  |  January 5, 2008 at 4:49 am

    Hi Michael,

    I realize only now you had discussed here the same topic of my post of today… Apologies, ignoring others’ views is not something I usually like to indulge with.


  • 5. Michael Schmitt  |  January 5, 2008 at 9:15 am

    No problem, Tommaso. Your post is very interesting and it is good, in a way, that you made a post independent of mine. Thanks for connecting my discussion back to your blog. 🙂

  • 6. Eclipse 2010  |  September 21, 2009 at 5:02 am

    It’s great that you have a great history for this blog since 2006

  • 7. Judging experiments – Evolving Thoughts  |  December 18, 2017 at 4:10 pm

    […] Collider blog has a discussion of an idea reported by Charm &c. in a paper at arXiv by Bruce Knudsen, proposing that experiments should be assessed using Shannon entropy based on prior expectations. […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed

December 2007
« Jun   Jan »

%d bloggers like this: