Tuesday, April 09, 2013

Meta-analysis in the middle

My general view both in general and as a policymaker in my small local-office pond is that a widely held (few expert dissenters) and strongly held (high confidence) consensus position might as well be a fact as far as the policymaking is concerned. A contradictory study is just that, a study. If it turns out to be accurate, then the consensus will eventually fracture soon enough and create problems for policymakers, but there's no need or even a rational way for policymaking to jump the gun.

In the gray zone between just one study and a consensus is the meta-analysis, and I'm still trying to figure out what policymakers should do with them. If the meta-analysis reinforces the consensus position - say, Oreskes' review of scientific abstracts on climate change - then it's just a helpful tool in examining the consensus. OTOH, if the meta-analysis seems to point in a different direction - say, Choi et al. on adverse neurological effects of high-fluoride exposure in China and Iran - that presents a bit of a problem.

My assumption, generally, is that the consensus should be handling this. The main Canadian health research arm, Health Canada, looked at a lot of the same studies and reached a different conclusion:

These studies were included in the review conducted by the Expert Panel on fluoride convened by Health Canada in 2007. Despite the consistency in the results from these studies, the panel agreed that the weight of evidence does not support a link between fluoride and IQ deficit. There are significant concerns regarding the available studies, including quality, credibility, and methodological weaknesses, such as the lack of control for confounding factors, the small number of subjects, and the dose of exposure (Health Canada, 2008). Most of these studies performed in China were also included in the reviews conducted by other organizations and/or committees, which also mentioned that the significance of these studies is uncertain (IPCS, 2002; ATSDR, 2003; NRC, 2006).
I think Health Canada is probably closer to showing what the fluoride consensus is than Choi, although I think there's reason to believe the consensus, that therapeutic levels of fluoridated water are generally safe, isn't as solid as the climate consensus. Worth noting one difference between meta-analysis and consensus is the meta-analysis is only as good as the meta-analyst, while consensus hopefully has a broader base.

Maybe the consensus will change, but this is what we've got.


Anonymous said...

a more relevant example imo when it comes to climate change is indirect land use change and biofuels. a real hornet's nest if there ever was one.


Anonymous said...

I'm spamming the usual suspects hoping that one of you will lay a Watts smackdown as a top-level post:

His crime: not understanding that "gasoline retail sales by refiners" is a subset of "gasoline retail sales" and that US gasoline usage has not dropped by more than 50% since 2004. And point out how, perhaps, a real "skeptic" might have realized that there's no way we could have failed to notice a 50% drop in consumption, and that therefore, just maybe, the chart doesn't say what it seems to say?


I think http://www.eia.gov/dnav/pet/hist/LeafHandler.ashx?n=PET&s=MGFUPUS1&f=M is a better graph of actual consumption. Watts might also have been tipped off that his chart was.. inappropriate... because US gasoline consumption is about 368 million gallons per day (http://en.wikipedia.org/wiki/Gasoline_and_diesel_usage_and_pricing) and his chart shows a peak of 60 million gallons per day...


Anonymous said...


Unlike your favorite sites like Closed Mind and Wrong Climate, Watts acknowledges mistakes, credits the person who pointed out the error and the end result is better for everyone.

Your favorite blogs never admit to error and delete comments that show anything counter to a narrative the site wants to follow.

Even here at Rubbish Run the "History.." article below stopped dead in its tracks when it was shown that a favorite liberal myth about Medicare Part D was shown false and Obamacare costs are far exceeding predictions.

Step out of your bubble and stop chasing Watts like he is your Moby Dick.

EliRabett said...

But he makes so many of them, a 5% acknowledgement rate is, shall Eli say. . . .

Oh yes, Eli is one of the legions of the banned for pointing out stuff oh so politely (for real:). Tony has this neat trick he learned from Keith Kloor of declaring anything any one he does not like says, disrespectful, nasty, etc. and then banning. Tamino just says screw off. It's their blogs.

Anonymous said...

So many mistakes are made here and rarely acknowledged.

But like you said it is your blog.

dhogaza said...

"Step out of your bubble and stop chasing Watts like he is your Moby Dick."

But he does rather resemble a great white whale.

Jeffrey Davis said...

re: anonymous at 11:07

The "liberal myth" involved the purposeful suppression by W of cost estimates until the Congressional vote was taken.

Anonymous said...

It is a myth as the Medicare part D has cost $304 billion which is less than what was put before Congress at the time of the vote.

As for Obamacare and deception from the party pushing that under estmiated unmitigated disaster.


David B. Benson said...

Brian --- Congratulations on your downtown beavers.

Brian said...

Thanks David! I hope they can stay. If they build a dam, there might be a long term problem with sediment buildup in the resulting pond. I'm not sure if the dam itself would be a problem because a major flood would just blow it out, I think.

My guess is they won't build a dam - that's not a piddling little creek they're on, so it would be too difficult, and no real need for it because they can swim all over the place right now without a dam.

a_ray_in_dilbert_space said...

I should translate for anonytroll above. When he or one of his fellow bots says "shown false" it means that one of their collective brains on the mothership has told them what talking points to regurgitate and they have practiced them to the point where they can get them out with a straight face.

Anonymous said...


So much projection in your words. You are giving too much of yourself away.

Go back into Dilbert's space, back to the nether dark regions of Dilbert's posterior. And take a deep breath and hold.