Friday, December 16, 2011

Telephone Churnalism

In a comment appearing in the American Scientist, Andrew Gelman and Kaiser Fung take the Steves, Levitt and Dubner to task for some very sloppy work

It’s hard to be sure what process an author uses. But by appearances, the way the authors of the Freakonomics series make their work is too linear to provide adequate vetting of research. In SuperFreakonomics, for instance, economist Steven Levitt trusts authors of primary research whom he knows or respects. Journalist Stephen J. Dubner trusts Levitt’s assessment of their work, and together they create narratives about it. The book’s editors seem by and large to have trusted the authors’ account, delivering it to readers who place trust in the Freakonomics brand. Although there may be more opportunities for feedback along the way than outsiders can discern, the problems and errors encountered in the authors’ work suggest that there is room for improvement.
Bunnies may recall the Freakazoid Franchise's encounter with Ray Pierrehumbert, and Gelman is exactly right, that people who are not expert in an area rely on one who knows, even a trusted one who knows, at their own peril.
The climate-change chapter in SuperFreakonomics is a case in point. In it, Levitt and Dubner throw their weight behind geoengineering, a climate-remediation concept championed at the time by Nathan Myhrvold, a billionaire and former chief technology officer of Microsoft. Unfortunately, having moved outside the comfort zone of his own research, Levitt is in no better a position to evaluate Myhrvold’s proposal than we are.

When an actual expert, University of Chicago climate scientist Raymond Pierrehumbert, questioned the claims in Levitt and Dubner’s writing on climate, Levitt retorted that he enjoyed Pierrehumbert’s “intentional misreading” of the chapter. Referring to his own writings on the subject, Levitt wrote, “I’m not sure why that is blasphemy.” We’re not sure on this point either—we could not find a place where Pierrehumbert described Levitt’s writings in those terms. It is easy to be preemptively defensive of one’s own work, or of researchers whose work one has covered. Viewing alternative points of view as useful rather than threatening can help take the sting out of critiques. And if you’re covering subject matter outside your expertise, it pays to get second—and third and fourth—opinions.

Still Gelman and Fung miss the Churnalism driver. Leveraging their initial best-seller, the Steves created a franchise, and like Mickey D's, franchises require tons of raw meat, though quality is not necessary Any cow to be slaughtered is welcome and they have to grind up a lot of it. Careful testing and research are optional in churnalism.

The interested can read more at Gelman's blog, Statistical Modeling, Causal Interference and Social Science

6 comments:

John Mashey said...

Yes, Gelman's blog is worth reading.
Some topics are pretty heavy for non-statisticians, but many are not. The occasional discussions of statistics and graphical expressions are often helpful. He also doesn't much like plagiarism and says so, often.

Anonymous said...

Dr. Jay Cadbury, phd.

Any solution that doesn't involve a new tax receives 0 consideration.

Aaron said...

I read that chapter, sat dumbfounded for a few minutes, then went out and burned the book in the BBQ. First time in my life I ever burned a book.

I did research on the life-cycle costs of waste. It is always cheaper to avoid producing hazardous and nuclear waste than to treat or isolate the waste after it is released. That is true for the various industrial and transportation processes that produce CO2. Always!

Note that CO2 produced as a product and captured before release has positive economic value. At one point some beer makers were making more profit on the CO2 that they produced than they made on the beer. Likewise, low metals sewage sludge can have a significant positive value, but a sewage spill is costly.

Anonymous said...

Dr. phd.,

I'm sure there's a reason why your credential is always lower-cased (must have come that way from the factory), but I'm not sure it bears repeating.

Are you sure that geoengineering a "solution" to human-caused climate change in order to continue maximum government-subsidized development and consumption of fossil fuels wouldn't involve any public expense or taxes?

Taylor B

David B. Benson said...

Talyor B --- It stands for

piled
higher &
deeper

EWI said...

Speaking of all things climate change, anyone else read the latest Richard Tol funny?

"I indeed believe that people are equally worthy [...]. That belief affects the way I think, speak and write about climate change and climate policy."

http://www.irisheconomy.ie/index.php/2011/12/15/mcdonald-in-nature/#comment-210936