The whispers of David Kane

Our friend David Kane and Iraq Body Count have been busily whispering into the ears of Neil Munro and Carl Cannon at National Journal, the result being a long article now spreading through right wing blogs where Eli picked it up. There isn't much new from David

"The authors refuse to provide anyone with the underlying data," said David Kane, a statistician and a fellow at the Institute for Quantitative Social Statistics at Harvard University.

And we see a lot of the arguments that have been knocked down by various folks at Deltoid (Tim, Robert D^2). But the National Journal walks right up to and probably beyond the line accusing Roberts and Co. of fraud

Some critics have wondered whether the Iraqi researchers engaged in a practice known as "curb-stoning," sitting on a curb and filling out the forms to reach a desired result. Another possibility is that the teams went primarily into neighborhoods controlled by anti-American militias and were steered to homes that would provide information about the "crimes" committed by the Americans.

And, of course, calling for an audit.

Surprisingly, not one of the peer reviewers seems to have thought to ask a basic question: Are the data in the two studies even true? The possibility of fakery, editor Horton told NJ, "did not come up in peer review." Medical journals can't afford to repeat every scientific study, he said, because "if for every paper we published we had to think, 'Is this fraud?' ... honestly, we would fold tomorrow."

In Belgium, Guha-Sapir's team is completing a paper outlining numerous mathematical and procedural errors in the Lancet II article, and its corrections will likely lower the estimate of dead Iraqis to 450,000, even without consideration of possible fraud during the surveying, a source said.

Perhaps medical journals, like respected news organizations, will learn that they have to factor the possibility of wartime fraud into their fact-checking. Horton knows the peacetime risks only too well: In a Lancet article in October 2005, exactly halfway between the two Iraq mortality studies, a Norwegian physician named Jon Sudbo wrote that a review of 454 patients showed that such common painkillers as ibuprofen and naproxen reduced smokers' risk of contracting oral cancer while increasing their risk for heart disease; it later turned out that Sudbo had faked his research.

And the Lancet editor tries to make nice which is going to come back and bite him

Today, the journal's editor tacitly concedes discomfort with the Iraqi death estimates. "Anything [the authors] can do to strengthen the credibility of the Lancet paper," Horton told NJ, "would be very welcome." If clear evidence of misconduct is presented to The Lancet, "we would be happy to go ask the authors and the institution for an official inquiry, and we would then abide by the conclusion of that inquiry."

Guha-Sapir's criticisms are that

First, according to Burnham and colleagues' results, there were nearly 600 war deaths per day—an unusually high number compared with almost any other armed conflict or indeed with other Iraqi mortality estimates.2 Burnham and colleagues' figure 4, in which cumulated Iraq Body Count deaths parallel their study's mortality rates, is misleading. Rates cannot be compared with numbers, much less with cumulative numbers. The correct comparison would be the one presented here (figure), in which the Iraq Body Count numbers are transformed into rates by period. In that case, there is no similarity between the trends in the study and Iraq Body Count.

Second, the study suggests that, over a 3-year period, around 90% of the deaths were directly related to violence. However, experience from other conflicts indicates that indirect causes (disease, malnutrition) typically outnumber the deaths due to violence (bombs, gunshots, etc).3 Burnham and colleagues' figure remained high for a long period of time. By comparison, only one of 17 surveys in Darfur reported a similar level of violent deaths, and this level only persisted for 3 months of a 6-month period.4

Third, the heterogeneity of the pattern of violence in Iraq argues for a differentiated estimation across the governorates. Insurgency and coalition action is still concentrated mainly in the Sunni triangle, but large tracts in the rest of the country are relatively peaceful. A better accounting for differences in violence by governorate separately and the effect of excluding the Sunni triangle would have strengthened the study.

more criticisms can be found at the link for the interested.