Sunday, June 13, 2010

Summertime

Summertime, and in Rabett Labs, a new bunch of undergraduate bunnies are researching away. In an attempt to impose structure on bedlam and please the funding agencies, Eli and his friends are running a weekly seminar in scientific ethics which met for the first time last week. Eli had asked Ben Hale to recommend a book, and he did, Scientific Integrity, by Francis Macrina, more of a casebook than an academic text and, except for its biomedical focus, a very good thing indeed.

Ben also told Eli that if all the students agreed, something was wrong, but to start the discussion the Rabett put a question, how does your research compare to what you thought your research would be, and there was agreement:

It's a lot more chaotic.
and that captures a great deal of the disjunction in discussions of science, not only climate science, between practicing scientists and others. Macrina follows Henry Bauer in dissecting the myth of the scientific method, differentiating between "textbook science", old tried and true stuff, the kind of things we do in undergraduate teaching labs, and "frontier science" stuff that people are still working on.

What we learn about in junior high school is an idealization of textbook science. The "noble scientist" whosoever she is, thinks deeply about all that is known, identifies something unknown, formulates a hypothesis, tests the hypothesis, interprets the data and publishes.

Would that it were ever so, indeed, the textbook science today that was frontier science back then, was not done in that way, but we contribute to the myth by the way in which we describe our work in publications. As Peter Medawar put it, the scientific paper is a fraud (Eli is channeling Macrina with some caveats in all this), it misrepresents
the thought processes that led to the work reported. He points out that the results section is written to present facts without and mention of significance or interpretation. These are saved for the discussion section. Medawar snickers that this is where scientists "adopt the ludicrous pretense of asking yourself if the information you have collected actually means anything" and "if any general truths are going to emerge from the contemplation of the evidence you brandished in the section called results"
whereas, in the real lab, the results are tested and interpreted as they come in against the scientist's expectations, which are formed by previous study and yes, previous understanding. Macrina quotes Goodstein
...every scientific paper is written as if that particular investigation were a triumphant procession from one truth to another. All scientists who perform research, however, know that every scientific experiment is chaotic, like war. You never know what is going on; you cannot usually understand what the data mean. But in the end you figure out what it was all about and then, with hindsight, you write it up describing it as one clear and certain step after the other. This is a kind of hypocrisy, but it is deeply embedded in the way we do science.
This is not unknown to scientists and is frequently remarked on, for example, quoting Francois Jacob
Scientific writing "substitute(s) an orderly train of concepts and experiments for a jumble of disordered efforts. . .In short writing a paper is to substitute order for the disorder and agitation that animate life in the laboratory
and without which life in the laboratory would be exceedingly boring. Macrina concludes that scientific papers do not describe what actually happened, but omit the details of how conclusions were reached, don't include the
wrong turns, dead ends, and "broken test tubes" that may have been crucial to the overall body of work. Scientific papers rarely describe or put into perspective the pure luck and mistakes that were also part of the work. .
Looking back at many of the attacks on science from our dear friends, they are wails that climate, and tobacco, and ozone scientists are not doing textbook science, and, of course, since most people only have learned textbook science, this can look like a pretty convincing argument. It is also why demands for regulatory science can be deadly to real science and why "auditing" is a distraction and a fraud.

On the other side, scientists, point to the triumphant march through their published papers, although over the course of time methods change, some conclusions are modified and the glorious adventure, the stories of how we reached our understanding, are swept under the rug. There are a thousand interesting stories out there in the Naked Laboratory, and they need to be told. Explaining the stops and starts would put a human dimension on science that is far more interesting and convincing to the public than what we have now. It is where blogs and open review can and will play important roles.

Stoat and Gareth have something to say on the matter, but of course, not as well.

11 comments:

Arthur said...

At least the average scientific paper does better on this than the average mathematics paper... In math it's a steady procession of lemmas, theorems and their proofs with no hint of wrong turns or purposes or the fact that the mathematician went through hundreds of pages of scribbled equations and perhaps some computer analyses to arrive at this hyper-sanitized version of the truth.

But I think it's natural that the story of science differs substantially from the output of science - those are really two quite different things. Science journalism is where the story should be told...

John said...

I experienced science education in the US, albeit some time ago. I am now retired from a genuinely nondescript "career" in science. I have not kept close tabs on the state of current science education.

From this entry it appears that perhaps the "story of science" AND the "funding of science" should be rolled into a formal course that is finally required of all first term graduate students. I know such a dose of reality would have helped me at that stage.

John Puma

mlyle said...

Of course, one of the hard things to teach graduate students is to avoid trying to publish all the twists and turns they had to follow to get to the end. Just because they had to suffer, the reader needn't also.

In the grand scheme of things, to move forward in research we need to mark the correct path, not all the dead ends.

Nevertheless, I agree with Peter Medawar and you--we spend too much time making it look like we knew where we were going from the beginning

a_ray_in_dilbert_space said...

I think that the real distinction here is between "science" and "doing science". They really aren't the same thing at all. Indeed, if you look at it, science is really a fairly orderly progression in our state of knowledge of the system under study. We begin with an impenetrable mystery and solve it by slowly penetrating it. We explore it almost dendritically, look at which branches yield fruit and expand our efforts there. What we report is what survives, as that is probably what is most useful to our peers.

The almost stochastic exploration we make on the way to a discovery--that's doing science. It is what we as scientists experience in our daily lives. But it isn't science any more than the daily experience of a lawyer is The Law.

Steve Bloom said...

BTW, ray, I've been meaning to say: Best handle ever.

a_ray_in_dilbert_space said...

Steve Bloom,
Thanks, the moniker is the product of my wife's fertile (febrile?) imagination. She is much more creative than I am. It is symbolic of my research--to find the transformation that has been applied to reality (a ray in Hilbert space) that has transformed it into the surreality it has become.

Anonymous said...

Snow Bunny says:

The Davisson-Germer experiment, that verified De Broglie's suggestion of electron diffraction, is a classic.

Their paper describes candidly how they came by a result they had not intended to look for. I read the paper when I was an undergraduate and was much amused at the description of broken equipment.

http://www.acolytescience.co.uk/origins/DandG.html

Chris Noble said...

I would not cite Henry Bauer as an authority in the philosophy of science.

He has become a conspiracy theorist, an HIV Denier and an AGW 'skeptic'.

http://hivskeptic.wordpress.com/

Marco said...

Let's not forget that Bauer also has made a case for the Loch Ness monster. I kid you not
http://henryhbauer.homestead.com/LochNessFacts.html

crandles said...

So are you actually in favour of seeing number 1 and 6 of

http://www.phdcomics.com/comics/archive.php?comicid=1312

?
crandles

Rattus Norvegicus said...

Phd comics is pretty funny.

I've especially come up with #6 when smoking out a bug (this is in an engineering context). Initial hypothesis(s) were all wrong and after through investigation finding you are all wet on your initial ideas is not unusual. But like the comic says, you'll never find it in a published paper, and if you can help it, you'll never tell your manager.