A Paper Named Sue

Many have commented on the problems with a recent paper by some marketing types claiming that hurricanes with female names are deadlier than those with male ones because of the perception that the male storms are fiercer.  Kiju Jung, Sharon Shavitt, Madhu Viswanathan, and Joseph M. Hilbe
Female hurricanes are deadlier than male hurricanes PNAS 2014 : 1402786111v1-201402786.

Eli's POV is that this was pitched to the freakonomics crowd masquerading as journalists who were happy to roll with it.  There are, as they say, issues, and Eli will simply outsource those to Jeremy Freese, Andrew  Gelman, Harold Brooks and Bob  O'Hara.

Bottom line, as Brooks points out is that these clowns are building a model for a fictitious, but churn baiting effect.  Why churn? Well the bunnies remember the churnalists, the post that earned Eli the undying affection of John Fleck.  If you wondered why every piece of crazy gets its day in the headlines here is one answer, the churnalists walking, nay sitting on their butts and printing everything that is spoon fed to them without working (shudder) to figure out whether there is any there there.  At root, churnalism is an abdication of responsibility and moral failing worn proudly by an entire profession.  Some do their job and some have useful bullshit filters, witness the E-mail Gelman got from a journalist working on the story

A colleague is working on a quick piece on a soon-to-be-released paper which argues that female hurricane names have higher death tolls since people take them less seriously. In looking through the methodology, we’re seeing a couple of potential red flags, but could use a professional’s judgment on this. Any chance you’re available to take a quick look?

One thing that caught our eye was the whole no-male-hurricanes-before-the-late-1970s thing. Does that leave us with enough male hurricanes to say anything substantive here? Is their method of rating not just whether but how feminine the earlier names were good enough? I’m also curious, of course, just how much statistical tweaking they had to do to get those results. More broadly, a lab setting in which people rate names doesn’t do it for me – in the real world these sexist impulses, which I’m sure exist, are likely swamped by a million other factors. Surely whether or not your five neighbors evacuate has a bigger impact on your behavior than storm name.

Gelman and Freese go a level deeper, something that Eli has also remarked on.  Behind every press release is a press office, aka often the University Communications Office.  As Freese, points out

The authors’ university issued a press release with a dramatic presentation of results. The release includes quotes from authors and a photo, as well as a quote from a prominent social psychologist calling the study “proof positive.” So this isn’t something that the media just stumbled across and made viral. My view is that when researchers actively seek media attention for dramatic claims about real deaths, they make their work available for especial scrutiny by others.

As a coda that may or may not be relevant to the case at hand, I will confess that I [Freese] have become especially impatient by the two-step in which a breathless set of claims about findings is provided in a press release, but then the authors backtrack when talking to other scientists about how of course this is just one study and of course more work needs to be done. In particular, I have lost patience with the idea the media are to blame for extreme presentations of scientists’ work, when extreme presentations of the scientists’ work are distributed to the media by the scientists’ employers [emphasis in the original].

 and Gelman adds

As the saying goes, +1. The news media are what gets us hearing about these studies (and indeed I’m contributing to it now), but the tabloid science journals such as PNAS provide incentives for researchers to engage in hype so as to get their papers published, and of course once a paper is published, with whatever errors it happens to contain, researchers have an understandable tendency to hang tough and not acknowledge problems with their claims.

The real question is what to do.  Eli had one suggestion a long time ago, any grant renewal had to attach press releases issued by the University about the research and the project should be evaluated on the basis not only of the scientific impact but the public impact.

Eli now has another.  Call it public shaming.  Write to the Communications Officer listed as contact on the press release.  Remark about how they have dragged their university's name into the mud.  Point to some of the substantive refutations and maybe explain a bit about why they are substantive.