Summertime, and in Rabett Labs, a new bunch of undergraduate bunnies are researching away. In an attempt to impose structure on bedlam and please the funding agencies, Eli and his friends are running a weekly seminar in scientific ethics which met for the first time last week. Eli had asked Ben Hale to recommend a book, and he did, Scientific Integrity, by Francis Macrina, more of a casebook than an academic text and, except for its biomedical focus, a very good thing indeed.
Ben also told Eli that if all the students agreed, something was wrong, but to start the discussion the Rabett put a question, how does your research compare to what you thought your research would be, and there was agreement:
It's a lot more chaotic.
and that captures a great deal of the disjunction in discussions of science, not only climate science, between practicing scientists and others. Macrina follows
Henry Bauer in dissecting the myth of the scientific method, differentiating between "textbook science", old tried and true stuff, the kind of things we do in undergraduate teaching labs, and "frontier science" stuff that people are still working on.
What we learn about in junior high school is an idealization of textbook science. The "noble scientist" whosoever she is, thinks deeply about all that is known, identifies something unknown, formulates a hypothesis, tests the hypothesis, int
erprets the data and publishes.
Would that it were ever so, indeed, the textbook science today that was frontier science back then, was not done in that way, but we contribute to the myth by the way in which we describe our work in publications. As Peter Medawar put it, the scientific paper is a fraud (Eli is channeling Macrina with some caveats in all this), it misrepresents
the thought processes that led to the work reported. He points out that the results section is written to present facts without and mention of significance or interpretation. These are saved for the discussion section. Medawar snickers that this is where scientists "adopt the ludicrous pretense of asking yourself if the information you have collected actually means anything" and "if any general truths are going to emerge from the contemplation of the evidence you brandished in the section called results"
whereas, in the real lab, the results are tested and interpreted as they come in against the scientist's expectations, which are formed by previous study and yes, previous understanding. Macrina quotes Goodstein
...every scientific paper is written as if that particular investigation were a triumphant procession from one truth to another. All scientists who perform research, however, know that every scientific experiment is chaotic, like war. You never know what is going on; you cannot usually understand what the data mean. But in the end you figure out what it was all about and then, with hindsight, you write it up describing it as one clear and certain step after the other. This is a kind of hypocrisy, but it is deeply embedded in the way we do science.
This is not unknown to scientists and is frequently remarked on, for example, quoting Francois Jacob
Scientific writing "substitute(s) an orderly train of concepts and experiments for a jumble of disordered efforts. . .In short writing a paper is to substitute order for the disorder and agitation that animate life in the laboratory
and without which life in the laboratory would be exceedingly boring. Macrina concludes that scientific papers do not describe what actually happened, but omit the details of how conclusions were reached, don't include the
wrong turns, dead ends, and "broken test tubes" that may have been crucial to the overall body of work. Scientific papers rarely describe or put into perspective the pure luck and mistakes that were also part of the work. .
Looking back at many of the attacks on science from our dear friends, they are wails that climate, and tobacco, and ozone scientists are not doing textbook science, and, of course, since most people only have learned textbook science, this can look like a pretty convincing argument. It is also why demands for
regulatory science can be deadly to real science and why "auditing" is a distraction and a fraud.
On the other side, scientists, point to the triumphant march through their published papers, although over the course of time methods change, some conclusions are modified and the glorious adventure, the stories of how we reached our understanding, are swept under the rug. There are a thousand interesting stories out there in the Naked Laboratory, and they need to be told. Explaining the stops and starts would put a human dimension on science that is far more interesting and convincing to the public than what we have now. It is where blogs and open review can and will play important roles.
Stoat and
Gareth have something to say on the matter, but of course, not as well.
Posts
