Don’t Argue: Just Make the Plot. Physics Controversies and How They Arise
By Prof. Jon Butterworth. (Feb. 2018)
Prof. Butterworth is a high-energy particle phycisist at CERN/LHC. Damn, he’s head of department at UCL.
Apparently at ATLAS and CMS people are constantly saying “don’t argue, just make a plot” and the plot will settle it. Apparently a lot of scientific controversies can be solved this way… so why do controversies exist to begin with?
We will see that some controversies are really proxy wars for academic or national politics, but still, why do scientific controversies persist to begin with?
Wanna read the Paul Dirac book, and the Joy of Sex author’s one.
There’s a book called Nobel Dreams, about a guy who was a historian embedded at CERN in the early 80s, and he wrote the book afterwards and it made a lot of the scientists very upset. And to this day! The book is controversial at CERN and banned from the library and people bringing it up have a word with (friendly-ly). So controversies are about scientists as well as science.
When do plots work?
Higgs boson discovery: collide the beams, measure the photons coming off. The x-axis is the invariant mass (ie, rest mass/energy) of the photons, which is directly related to the mass of the particles colliding. If we see more events compared to background, then a new particle shows up as a bump compared to the previous theories’ background.
When the problem is something like a bump in such a plot, the easy thing to do is just to go and collect more data and see what happens. If it stays, yay, if it goes away, ah well, the plot solved the argument. (In the meantime, theorists may theorise)
When they were doing precise muon magnetic moment calculations, they moved the whole experiment from Brookhaven to Chicago (big magnet to move!) because Chicago had more muons. It was thought the experiment/theory discrepency was due to not taking into account GR, but a better calculation showed that to not be the case.
“Just do a better experiment/calculation and get more people to check it” and that will solve the controversy… or will it?!?!
When doing high-count-rate particle physics, you have to make a decision about what data to even keep in the first place. (This is theory-led!) This is occasionally controversial, because you can’t measure everything so you have to decide to discount certain things.
When don’t plots work?
For electron-positron accelerators, the limiting factor is the RF power in. As they accelerate around the beam, they shed synchrotron radiation, and eventually they’re doing this faster than you can put more energy in.
That’s cool, the electron-positron experiment ALEPH that was there before LHC might have had a bit of evidence for the Higgs’ too, but it wasn’t! And they couldn’t afford an extra year delay to the LHC to collect more data and “make the plot”. The decision to turn off the positron-electron accelerator was controversial, and not revolved by a plot.
Sometimes you might make the wrong plot! The BICEP2 group in the Antarctic was looking for twists in the EM spectrum which would be a sign of primordial gravitational waves from the big-bang. They thought they’d found them! But it turns out that BICEP2 had used the same background from a preliminary Planck result, and it wasn’t entirely correct. Later, when the full data was released, their discovery went away.
In some sense, the people who measure last win - the difference between the prior measurement (the background) and the second measurement (which measures the deviation) is what makes the discovery, and can only be done by the second team.
Another example is someone, Miller, trying to replicate the Michelson-Morely experiment by going up a mountain (where he felt the ether drift would be stronger) and collecting more data. His data was anomalous to every other measurement, and suggested he had measured the absolute motion of the Earth, which special relativity prohibits.
Incredibly, ~100 years later, a historian of science/phycisist (Thomas Roberts) was able to completely re-analyse the raw data and apply present day statistical and signal processing procedures. It turns out Miller’s results were an artifact of his analogue/pen-and-paper signal processing. Sometimes you make the best plot you can, and later someone comes along and makes a better one! Dayton Miller, fantastic experimentalist that this was possible in the first place!
So to summarise, plots aren’t enough when: - Trying to decide what to do next - When the arguments are unscientific/political &c., aesthetics, naturalness and so on - When people deny the science in the first place (@climate change deniers) - You don’t have the right statistical/signal processing tools to make the right plot - When you make the wrong plot
This whole thing seems to be related to “shut up and calculate” but actually is not, because quantum interpretations are not particularly scientific to begin with and not amenable to being resolved by plots (unless you’re Bell)
Huh, fun fact: the sum of the squared masses of the bosons is the same as the sum of the squared mass of the fermions (to 1 GeV)
my takeaways:
- high-energy particle physics is less annoying than I thought. This evidence appraisal stuff is quite fun!
- Thinking about medical physics, when I’m doing research, I want to have really quite rigourous statistics and analytic procedures, and I want to understand them. Don’t want to just be using p-values and such.
- This talk was really more about “when is scientific evidence” enough, plots being the main form that that takes. But you can have very good scientific evidence and a crap plot that doesn’t communicate it well. You can also lie with graphs.
- Ultimately, record and communicate everything you do as clearly as possible