Sunday, June 08, 2014

A Paper Named Sue

Many have commented on the problems with a recent paper by some marketing types claiming that hurricanes with female names are deadlier than those with male ones because of the perception that the male storms are fiercer.  Kiju Jung, Sharon Shavitt, Madhu Viswanathan, and Joseph M. Hilbe
Female hurricanes are deadlier than male hurricanes PNAS 2014 : 1402786111v1-201402786.

Eli's POV is that this was pitched to the freakonomics crowd masquerading as journalists who were happy to roll with it.  There are, as they say, issues, and Eli will simply outsource those to Jeremy Freese, Andrew  Gelman, Harold Brooks and Bob  O'Hara.

Bottom line, as Brooks points out is that these clowns are building a model for a fictitious, but churn baiting effect.  Why churn? Well the bunnies remember the churnalists, the post that earned Eli the undying affection of John Fleck.  If you wondered why every piece of crazy gets its day in the headlines here is one answer, the churnalists walking, nay sitting on their butts and printing everything that is spoon fed to them without working (shudder) to figure out whether there is any there there.  At root, churnalism is an abdication of responsibility and moral failing worn proudly by an entire profession.  Some do their job and some have useful bullshit filters, witness the E-mail Gelman got from a journalist working on the story
A colleague is working on a quick piece on a soon-to-be-released paper which argues that female hurricane names have higher death tolls since people take them less seriously. In looking through the methodology, we’re seeing a couple of potential red flags, but could use a professional’s judgment on this. Any chance you’re available to take a quick look?

One thing that caught our eye was the whole no-male-hurricanes-before-the-late-1970s thing. Does that leave us with enough male hurricanes to say anything substantive here? Is their method of rating not just whether but how feminine the earlier names were good enough? I’m also curious, of course, just how much statistical tweaking they had to do to get those results. More broadly, a lab setting in which people rate names doesn’t do it for me – in the real world these sexist impulses, which I’m sure exist, are likely swamped by a million other factors. Surely whether or not your five neighbors evacuate has a bigger impact on your behavior than storm name.
Gelman and Freese go a level deeper, something that Eli has also remarked on.  Behind every press release is a press office, aka often the University Communications Office.  As Freese, points out
The authors’ university issued a press release with a dramatic presentation of results. The release includes quotes from authors and a photo, as well as a quote from a prominent social psychologist calling the study “proof positive.” So this isn’t something that the media just stumbled across and made viral. My view is that when researchers actively seek media attention for dramatic claims about real deaths, they make their work available for especial scrutiny by others.

As a coda that may or may not be relevant to the case at hand, I will confess that I [Freese] have become especially impatient by the two-step in which a breathless set of claims about findings is provided in a press release, but then the authors backtrack when talking to other scientists about how of course this is just one study and of course more work needs to be done. In particular, I have lost patience with the idea the media are to blame for extreme presentations of scientists’ work, when extreme presentations of the scientists’ work are distributed to the media by the scientists’ employers [emphasis in the original].
 and Gelman adds
As the saying goes, +1. The news media are what gets us hearing about these studies (and indeed I’m contributing to it now), but the tabloid science journals such as PNAS provide incentives for researchers to engage in hype so as to get their papers published, and of course once a paper is published, with whatever errors it happens to contain, researchers have an understandable tendency to hang tough and not acknowledge problems with their claims.
The real question is what to do.  Eli had one suggestion a long time ago, any grant renewal had to attach press releases issued by the University about the research and the project should be evaluated on the basis not only of the scientific impact but the public impact.

Eli now has another.  Call it public shaming.  Write to the Communications Officer listed as contact on the press release.  Remark about how they have dragged their university's name into the mud.  Point to some of the substantive refutations and maybe explain a bit about why they are substantive.


Dano said...

The issue of press releases being partially written as puffery for the department/uni is one of the major concerns in making scientific information usable, useful and actionable. Increasing pressure for revenue and rich alums may be signs of a market failure.

That is: who communicates findings to the publics and policy-makers? And how is it communicated?

We aren't close to figuring this out, IMHO.



J Bowers said...

Dano -- "Increasing pressure for revenue and rich alums may be signs of a market failure."

We have a new saying here in Blighty: Making universities more like the commercial sector is turning first rate academic institutions into third rate companies.

Press releases in a commercial context are just a marketing tool.

Hank Roberts said...

U. Illinois Press Release:

"... this research is the first to demonstrate that gender stereotypes can have deadly consequences."

These press release writers are not just assholes, they're ignoramuses.

Bernard J. said...

The story was given wide coverage in Australia - one example here:

It seems that it's not just press offices piling on the bandwagon - authors and their colleagues are happily taking the reins.

I haven't read much of the commentary on this myself, but perhaps someone could answer a question that has been niggling since I heard the story - how soon before a low reaches its maximum intensity is it actually named?

Anonymous said...

Increasing pressure for revenue and rich alums may be signs of a market failure."

No, those are sure signs of market success, at least in the US, where junk is the national product and marketing of junk the national pastime.

Anonymous said...

"[next] time, the announcements should be made after submission to journals and vetting by expert referees. If there must be a press conference, hopefully the scientific community and the media will demand that it is accompanied by a complete set of documents, including details of the systematic analysis and sufficient data to enable objective verification." --

"Big Bang blunder bursts the multiverse bubble"

Premature hype over gravitational waves highlights gaping holes in models for the origins and evolution of the Universe, argues Paul Steinhardt.

Anonymous said...

Male hurricanes are less deadline because they won't stop for directions when they get lost.


Hank Roberts said...

Heading off Coastline Hazards

Boulder, CO, USA - Sea level is rising, yet Americans continue to develop beaches with little regard for this fact. This new Special Paper from The Geological Society of America presents a collection of case studies that convey the startling saga of ten U.S. East Coast and Gulf Coast shoreline communities and the problems created by their inevitable interaction with the natural processes of these highly dynamic geologic environments. Examples from Puerto Rico and Europe are also highlighted.

According to volume editors Joseph T. Kelley of the University of Maine, Orrin H. Pilkey of Duke University, and J. Andrew G. Cooper of the University of Ulster, the 11 case studies in this volume foreshadow the impending rise of sea level and the myriad of shoreline responses and political controversies that rise will provoke....

America’s Most Vulnerable Coastal Communities
Joseph T. Kelley, Orrin H. Pilkey, and J. Andrew G. Cooper (editors)
Geological Society of America Special Paper 460
SPE460, 179 p., $65.00, GSA Member price $50.00
ISBN 978-0-8137-2460-7