A Dummy's Guide to Strange Scholarship in the Wegman Report II
A further example of research misconduct is the inclusion of a doctored version of a graphic that appeared in the 1990 IPCC assessment report (see page 137 of Strange Scholarship). Many have misinterpreted this as being a data based global record, but at least in his testimony, Prof. Wegman initially stated that it that it was a cartoon of European temperatures reflecting what was believed in 1990. What raises this from lunacy to misconduct is the distortion of the original drawing and the bootstrapping in the text. See below for Wegman's stretched figure and the original. First the WR displays their distorted graph saying
"Figure 4.5: Here we have digitized the temperature profile as presented in the IPCC Assessment Report 1990. The early period between 1100 to about 1400 of above average temperatures is known as the Medieval Warm Period and the period from about 1500 to 1900 is known as the Little Ice Age."Then in the next paragraph they stretch it to represent a global temperature record
"Discussion: In Figure 4.5, we have digitized the temperature profile as presented in the IPCC Assessment Report 1990. The early period between 1100 to about 1400 of above average temperatures is known as the Medieval Warm Period and the period from about 1500 to 1900 is known as the Little Ice Age. The 1990 report was not predicated on a global warming scenario. It is clear that at least in 1990, the Medieval Warm Period was thought to have temperatures considerably warmer than the present era."And finally, in the conclusions they bootstrap it to attack Mann Bradley and Hughes
"The cycle of Medieval Warm Period and Little Ice Age thatwas widely recognized in 1990 has disappeared from the MBH98/99 analyses, thus making possible the hottest decade/hottest year claim."
The interchange between Rep. Stupak and Wegman in the Congressional Hearing on this point is one for the ages. It's a bit long, but Eli promises that it will entertain. Under questioning Wegman admits that he did not read the 1990 IPCC report
MR. STUPAK. I think you have it in front of you, your 52-page summary there, you have a figure that you say is a digitized version of the temperature profile in the IPCC assessment report of 1990. I take it you read the 1990 IPCC report?Some byplay about what page of the Wegman Report this was on followed by
MR. STUPAK. Well, then you must have at least discussed this temperature profile.Wegman says that the chart was a) a cartoon and b) based on European temperatures. Next, watch Prof. Wegman pull out the Sgt. Schultz defense.
DR. WEGMAN. The temperature profile that was published in 1990 I believe was related to the European temperatures and was a cartoon--essentially a cartoon. The point of our discussion here was not that we were trying to say that this was what happened in 1990. The point of our discussion was that you could reproduce this shape from the CPF, CFP and the climate plus--whatever--CPS methodology so we are not endorsing that this was the temperature that was thought of in 1990. We are simply using this as an example.
MR. STUPAK. Were you endorsing 1300 as being a real high temperature time? Were you endorsing it in your report?But Bart Stupak has been well briefed.
DR. WEGMAN. No, we have not said that.
MR. STUPAK. What was the 1990 IPCC temperature profile based on? Basically what was this based on? You are a statistician.
DR. WEGMAN. This--
MR. STUPAK. Was this based on data?
DR. WEGMAN. As I just said moments ago, this was a cartoon I believe that was supposed to be representing a consensus opinion of what global temperature was like in 1990 as published by the IPCC.
MR. STUPAK. Well, is this cartoon then--again, I am on page 34, I am reading now from your report, discussion you have underneath this cartoon. Last line: "The 1990 report was not predicated on global warming scenario. It is clear at least in 1990 the medieval warm period was thought to have temperatures considerably warmer than the present era." Is that your discussion?
DR. WEGMAN. Yes.
MR. STUPAK. So we should not believe that statement then?
DR. WEGMAN. No, I said--I didn't say I believed it was. I said they believed it was. The IPCC gave that report in 1990.
MR. STUPAK. All right. This chart--
DR. WEGMAN. I didn't--
MR. STUPAK. This is in your executive summary, right, page
DR. WEGMAN. The cartoon is IPCC's cartoon, not mine.
MR. STUPAK. You relied upon it though in your executive summary. So I am looking at the cartoon. There is no data, is there, to say that around 1300 it warmer than it is in the latter half of--Which is interesting because earlier Prof. Wegman said that he did not have the IPCC report which would have had the data or a citation AND the original figure. The question occurs, who gave Prof. Wegman the figure and at what stage was it distorted. This is one of the things that the GMU investigation should get at.
DR. WEGMAN. I think that is an inaccurate statement. I think there is data. I think the data--
MR. STUPAK. Do you have any of it? Can you show us where any of that is?
DR. WEGMAN. No, I don't have it. I take no responsibility for what IPCC did in 1990. There is no way I could do that. Their data is not available to me. In fact, the reason it was digitized was that I had to go back and construct it from their picture. That doesn't mean no data exist. And in fact, as far as I know, it was based on European and Asian temperature profiles that were available in the 1990s.
MR. STUPAK. Sure, and in that, it was thought--it was still not clear that all the fluctuations indicated were truly global. In fact, I think some of the testimony earlier said that parts of western Europe, China, Japan, and eastern U.S.A. were a few degrees warmer in July than other parts of the world. Parts of Australia, Chile, and I think Greenland were actually cooler, they said, and China was actually cold
The Wegman Report should be withdrawn.
Posts
37 comments:
When Wegman was asked if he had the IPCC report he was also asked if he could show the relevant passage, so it seems to me his "No, I don't have it" refer to him not having it with him at the hearing, not that he had never seen it. He was wrong about what the figure in IPCC-90 was based on, but as I recall the report itself is rather vague on that point, you have to dig deeper to realize it's just temperature in England.
Cartoons? All sounds a bit Mickey Mouse and Goofy to me.
Cymraeg llygoden
As Tommy Steel once said... "Flash, bang, wallop, what a picture!"
Eli Rabett, will you marry me?
Eli, you don't go into the distortion issue much, but isn't that even more damning? Just as plagiarizing could be passed off as merely forgetting some quote marks except that they *changed some of the words*, so here the use of the IPCC 1990 figure can be passed off as no big deal except that *they changed the figure*. Note in particular the year 1300 that Stupak asked about. In the original, 1300 was well after the peak (which looks like about 1150) of the putative MWP. But in Wegman's version the peak is after 1200, and 1300 is pretty much still peak territory. Why would they do that?
Could it have something to do with MBH98 which only goes back to 1400, but for which McIntyre's distorted reproduction found spurious warming in the 1400-1500 timeframe?
Also note Wegman's version distorts the 20th century - the original IPCC graph stopped around 1950, before the modern warming got going...
'old on, Eli, you've cut ou the best bit. Over at my place, macjim quotes from Wegmans testimony as:
MR. STUPAK. Let me ask you this. Page 34 of your report, I
think you have it in front of you, your 52-page summary there,
you have a figure that you say is a digitized version of the
temperature profile in the IPCC assessment report of 1990. I
take it you read the 1990 IPCC report? [clarification of page numbers]
DR. WEGMAN. No, I have not been able to obtain a copy of the 1990 report.
See http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=109_house_hearings&docid=f:31362.wais
Stupak misses his chance to go for the jugular there: you mean, you really ahven't found a copy of the IPCC report? And yet you've used its pictures? Which means the obvious: they copied them from someone else.
WOW! This will be a sensation. John Mashey put in an incredible amount of work, dissecting the Wegman Report. I expect there are some red faces at George Mason U.
Thanks, Eli!
Nevada Ned
Hmmm, the GMU library today does have a copy of the 1990 IPCC AR
http://magik.gmu.edu/cgi-bin/Pwebrecon.cgi?BBID=380706
It would be interesting to know, when it was bought.
It's also readily available on Amazon for $1.46, and for the truly frugal bunnies, downloadable for free at IPCC. Total search time: less than 5 minutes. Maybe they didn't look very hard?
"It's also readily available on Amazon for $1.46, and for the truly frugal bunnies, downloadable for free at IPCC. Total search time: less than 5 minutes. Maybe they didn't look very hard?"
Perhaps they were looking for something they could plagiarise.
Cymraeg llygoden
Thomas: it pays to read the real material. WMC quotes the zinger by Wegman. So do I in SSWR W.4.2. Wegman said he had not seen it.
See SSWR W.8.9. I think the evidence is pretty strong they got it from M&M's May 2005 presentation at George Marshall along with much else. that's why I gave one vaguely-referenced paper a section for itself. The WR in many ways is an greatly-expanded version of that paper.
I especially liked:
"In fact, the reason it was digitized was that I had to go back and construct it from their picture. That doesn't mean no data exist. "
That's from a senior statistician...
"It's also readily available on Amazon for $1.46, and for the truly frugal bunnies, downloadable for free at IPCC. Total search time: less than 5 minutes. Maybe they didn't look very hard? "
In a weak defense of Wegman on this particular point, the IPCC didn't put it on-line until recently (I think I found it about a month ago? And promptly e-mailed all my climate change buddies in my excitement, since I've been wanting to get my hands on this thing for years). I don't know about Amazon (I'd never thought of looking, or I would totally have bought a copy myself), but I wouldn't be surprised if the on-line availability means that people who were previously hoarding their copies are now willing to part with them...
-M
Whether the IPCC only recently put it online or not and whether GMU only recently obtained it or not is immaterial really.
The Wegman coterie consisted of professors and postgrads. Each and every one of them could have ordered the FAR from their university library.
They could have obtained it from CUP (UK or USA) or the IPCC direct and billed it as necessary materials.
They could have loaned it out from the Library of Congress, who AFAICT have had it since 1990.
If they didn't consult the primary document (and "the figure" is suggestive that they didn't at that time), then this is just a further example of poor scholarship.
Cymraeg llygoden
Cymraeg:
To be really precise, this was:
1) Wegman, a professor.
2) Said, who was a Lecturer @ JHU 2005-2006.
3) with help acknowledged from John Rigsby (MS 2005), whose are of contribution is fairly clear (SNA analysis), and Denise Reeves (PhD 2009, whose involvement is nonobvious.)
[Scott was uninvolved in the plagiarism.]
Again, I would strongly suggest to everybody the extent to which the various Memes of MM05x, described in SSWR's W.8.9, permeate the WR.
Here's a fun exercise:
1) Get a copy of MM905x
2) Read SSWR Appendix W.8.9, 2 pages. Imya list:
02, 03, 18, 21, 32, 56 (MWP, the biggie), 107 [Skeptical Science]
a, b, c, d, e, f, g, h [oops, I see typo].
Now check 2) vs 1), and strike off the list any you think are not justified or too marginal to worry about.
3) Then take the remaining list to the Index and see how often those memes appear. Red-coded numeric Memes are found in MM05x, as are memes a-g.
Meme-56 (big MWP) appears often, as of several others. Anyone who wants check every mention to see if the assessment is reasonable, arguable, or unreasonable.
BUT, it is unfortunate that non one asked Wegman "If you couldn't get a copy, where did yoyu get this chart from?"
(An answer like: "it came in a plain brown wrapper with no return address", or "we got it from McI/McK" would not have been well received. Recall that MM05x is just listed in the Bibliography, not even cited...)
You have Curry's attention. She is using quotes and cites in her latest take down of the IPCC to placate the "plagarism police".
Paul
Cymraeg the Library of Congress only loans stuff to congressional offices. Hmm, Barton could have gotten it for them
Eli: delicious. My favorite part is that the 1990 IPCC report isn't even referenced in Wegman.
Or is it that the McIntyre and McKitrick 2005 talk (thanks John Mashey!) is referenced in Wegman, but is conspicuously (or is it sloppily?) missing the presentation location: George C. Marshall Institute.
Ah, I see now it was a Senate Committee, but ...
The library is open to the general public for academic research and tourists. Only those who are issued a Reader Identification Card may enter the reading rooms and access the collection. The Reader Identification Card is available in the Madison building to persons who are at least 16 years of age upon presentation of a government issued picture identification (e.g. driver's license, state ID card or passport).[18] However, only members of Congress, Supreme Court Justices, their staff, Library of Congress staff and certain other government officials can actually remove items from the library buildings. Members of the general public with Reader Identification Cards must use items from the library collection inside the reading rooms only; they cannot remove library items from the reading rooms or the library buildings.
Wikipedia
Considering the amateurish scholarship on display then, perhaps the members of the Wegman coterie were of insufficient age to gain a Reader Identification Card.
Cymraeg llygoden
Mr. Rabbett,
Suppose you can abstract away the fact that Wegman’s recommendations seem to follow his conclusions. If you can, imagine that these recommendations were not coming from a Congress House hearings, but coming from the spirit of Richard Feynman after a Ouija session. Here is my question:
> Would you agree that these recommendations would improve science?
The “should” there is not a “must”. The “would” is not a “will” either. I would never want you to make you prophecize anything.
Just go with your gut feeling.
If you don't agree, what are the specific words in these recommendations that are ticking you off? I want to know what you and I can agree about these recommendations.
Please note that I am not asking you if they are necessary conditions to do science. So please read the “should” as a “should”, not a polite way to say “must.”
I have been told to be inclined to galimatias, non sekwitchurs and red tree rings in the past, so if you need me to rephrase, feel free to ask.
Feel free to say if anwering this question could have an impact on some of your work, and if revealing your identity could make your publications the subject of a similar Congress House hearings.
She [Curry] is using quotes and cites in her latest take down of the IPCC to placate the "plagarism police".
Horatio was under the impression that using quotes and properly citing the work of others was standard practice in academia.
Then again, Horatio went to college back in the old days (late '70's/early 80's) so it's very possible that things may have changed significantly since then.
And he's the first to admit that all those old school values like honesty and hard work are so out of sync with today's high paced "(re)publish or perish" world of academia.
Not worrying about attribution does make it a lot easier to get "new" stuff published, that's for sure, so that's prolly all for the good.
Horatio is currently working on a "new' idea of his own: E=mc^2
And after that, he's going to write up his thoughts on a truly revolutionary (and novel) idea: how matter bends spacetime.
Willard, Eli has learned not to take poisoned pawns. As he wrote to AMac, it is not necessary to pay any serious attention to the poisonous Wegman Report because the NRC report from the North Committee covers much of the same ground.
As North points out in his testimony, the statistics in Mann, Bradley and Hughes 1998 and 1999 had some problems, but captured the trends and have been supported at least qualitatively and one could argue quantitatively by subsequent work with improved statistical methods.
Wegman's first conclusion that academic climate science has been politicized ranks right up there with patricide claiming mercy because she is an orphan. It goes downhill from there. It's not even worth a fast giggle.
Wegman's third conclusion is also of the don't make me shoot my mom category, claiming falsely as noted by North and by others in the congressional testimony that climate scientists were isolated from statisticians. Based on what Eli asks? Wegman by his own testimony had no contact, either in the writing of the report or in his professional life with climate scientists. Read the testimony of North and Crowley among others which falsifies the conclusion.
Wegman's fourth conclusion as to the worth of paleoclimate studies, is also dealt with by North. see the link to RR above. Wegman's description is best described as an ignorant bleat of the Hal Lewis cruiser class.
Which brings us to the second one for making everything public. Beyond the usual motherhood issues, this is, as we have all seen not so simple given commercial interests of information generators. A good example is Gaussian, an ab initio chemistry program created by John Pople @ Pittsburgh way back and taken commercial. After Pople broke with the firm, he was forbidden to use his own software but even more to the point, no collaborator of his at any other place on Earth was allowed to use Gaussian.
Mr. Rabbett,
I can understand your retiscence to play the thought experiment. It's very though to consider Wegman's recommendations if you can't think of a context where it would not be poisonous. Reading the actual first conclusion and recommendation can easily show that separating Wegman's conclusions from his recommendations is no piece of cake, be it sliced to celebrate the publication of a paper or for another reason.
So I believe what you are saying is:
> I disagree about Wegman's recommendations because I disagree about Wegman's conclusions and both go hand in hand.
Am I correct? That's the impression I have, since I asked to consider the possibility that Richard Feynman's own spirit hushed you these recommendations, not Wegman, so to try to consider them independently from the conclusions. And you still talk about Wegman's conclusions.
If I am correct, I believe it underlines the nature of disagreement. The disagreement pertains to a debate that is not deliberative, but judiciary. We are not seeking truth or falsity, but praise or blame.
That said, please consider the possibility that if you disagree about Wegman's recommendations, people might be tempted to say that you are against Science betterment. So it's important that you clearly state that you are for better science. Nobody can afford to be against better science. Who's against better science anyway?
Sometimes, it's good to take a poisoned pawn. It's a variation of the Sicilian Najdorf that is being played by many top players.
Willard, what we are seeking is neither truth nor falsehood, nor praise nor blame. What scientists are seeking in all this is to allow the process of science to run its course in the same way that has yielded reliable understanding for 400 years. What we do not need is Attorneys General launching witch hunts or Congressional commitees putting their damned thumbs on the scale.
If Wegman had a serious issue with Mann et al. '98, the proper course of action would have been to publish his own study in a peer-reviewed journal and let the experts slug it out. If he himself did not have the expertise to do so, then he should have teamed with a paleoclimate researcher who did.
Instead, we have this botched abortion of a Congressional Report, full of errors, inconsistencies and just flat ignorance. Scientists do not need Dr. Wegman to tell us how to do science. We'd be more than happy to listen to what he has to say in his own limited field of expertise--statistics.
If Micheal Mann or any other scientist should publish a seriously flawed study, their colleagues are there waiting to back them up--with sharpened claws and long knives. Science is competitive as well as cooperative. That's why it works. And that's why Wegman utterly fails to understand it.
Go ask Said
Horatio:
Do note: Yasmin H. Said is "her" not "his".
Noted.
Thanks, John.
Horatio strives for accuracy in his goofy poetry. He has a reputation to keep up, you know.
a_ray_in_dilbert_space,
Whatever scientists are seeking to do in their day, evening, or night jobs, your very comment is not seeking that. Correct me if I am wrong, but your very comment seems to serve the main purpose of blaming Wegman.
I am not blaming you for blaming Wegman. I just want you to understand and acknowledge that it's what you do. If "blaming" seems too harsh for you, let's blame Aristotle ;-)
Willard, as Eli said before, Wegman's recommendations have poisonous roots, and, as North and others point out, they really have little connection with actually possible policy changes. Someone, Eli thinks it was Crowley, who pointed out that climate scientists already have strong links with statisticians, but these are not unidirectional ones, but a scientific interchange.
From Prof. Crowley;s testimony to the Barton inquisition points this out.
"Bullet three - the researchers do not seem to be interacting with the statistical community. This statement is based on a small subsample of paleoclimate papers. Overall, there is increasingly strong incorporation of statistical methodologies in the climate sciences, including increased interactions with statisticians. For example, the National Center for Atmospheric Research has had a postdoctoral program for statisticians for thirteen years. A key project jointly funded by DOE and NOAA for detection and attribution of climate change involves not only several statistical climatologists but also explicitly seeks out input from statisticians. The present (and key) IPCC Fourth assessment chapter on detection and attribution of climate change has a statistician and statistical climatologist (with a training in applied mathematics) as co-lead authors. Statisticians are welcome to respond to any of the chapters in the review process. From these statements it is clear that the Wegman Report is somewhat uninformed with respect to the effort to include statisticians in the IPCC review process.
I might add that interactions between geoscientists and statisticians have long been hampered by what can only be described by some as a condescending attitude from some statisticians that geoscientists were not employing the most recent, state of the art statistical methods. Such attitudes almost guarantee subsequent poor communication and fail to recognize the unusual nature of "field laboratory" geoscience data, which are very different than "closed laboratories" where the conditions of an experiment are well controlled. The latter types of data require an intimate understanding of the raw data and simpler, more robust statistical methodologies that recognize the limitations of such data."
Wanna go on.
Willard, I said that scientists would gladly listen to what Wegman had to say in his field of expertise. Where Wegman comes in for criticism--and justly--is for pontificating well outside of that narrow expertise. Moreover, there is a long history in science of disdain for those who publish their results "IN PRESS"--that is in a forum for non-experts, and Congress certainly qualifies there. Such ex-cathedra publication is exactly where Pons and Fleischman went wrong.
Is is really too much to ask that scientific debate take place among real experts and in a deliberative forum?
Eli,
Crowley makes an interesting point regarding one of W's recommendations. Let's focus on this one and consider this question:
> Should climate scientists interact with statisticians for science betterment?
Here is one answer:
> Yes, considering that I believe the related W's conclusion #3 to be correct.
That's not what you think. But that does not mean you need to answer "no". For instance:
> Yes, considering that we already do it.
Interestingly, one can also reply:
> No, because we already do it.
This is caused because we don't know how to interpret the question. This quandary is not unlike the paradox of implication in logic.
My own suggestion would be to only consider the recommendation, ignore what it might presuppose and focus on Science betterment. In that case, we should see that:
> Whatever W's conclusions and under normal circumstances, consulting a statistician to validate studies that contain stats would not be a really bad idea.
Hence I might be tempted to answer "yes" to this recommendation.
I tested this kind of reasoning with around ten persons for now, and I have yet to find one person who disagrees with me. It would be very tough to disagree with having one's stats verified by a statistician. I don't mean that it isn't done in climate science. I don't care about any specific case: I only want better Science.
That does not mean it is a truism. For the life of me, I would never say it is a truism. I have yet to find anyone against this principle. I suppose that those who say "we already do it" agree with me more than they disagree. It would be interesting what reason can be opposed to it.
Please note that this is only a general principle, void of any enforcing teeth. That is to say, we can only agree on principle. This kind of answer can't lead to a formal agreement.
Do W's recommendations have any teeth as they are? What do they mean exactly?
a_ray_in_dilbert_space,
I really like your idea to have a deliberative forum of experts. This should be implemented for Science betterment.
Willard, read what Crowley wrote, you need to understand the source and the nature of the data to pick the right statistics to treat it. Dialing up a statistician is not useful, you need to find one who appreciates the nature of the observations. Wegman is not a useful guide.
Eli,
I am not sure how Crowley responds to the recommendation.
There is this sentence that looks like a "yes":
> Statisticians are welcome to respond to any of the chapters in the review process.
There is this criticism that looks like a "no":
> [T]he Wegman Report is somewhat uninformed with respect to the effort to include statisticians in the IPCC review process.
Here is one or two desiderata:
> [I]nteractions between geoscientists and statisticians have long been hampered by what can only be described by some as a condescending attitude [...].
> [T]he unusual nature of "field laboratory" geoscience data [...] require an intimate understanding of the raw data and simpler, more robust statistical methodologies that recognize the limitations of such data.
Taking all this into account, I believe we could say that Crowley's testimony coheres with this:
> For Science betterment, Climate scientists shall work with statisticians to validate the statistical methods to suit their needs and requirements.
I rephrased the "should" by a "shall" to stay neutral about what is actually being the case.
I am tempted that this counts as a "yes" to my question.
***
I always appreciate when you quote from the Barton hearings. I know there are even more interesting quotes there. I sincerely believe it provides material for a Mametian play.
So please continue,
w
OK, need the Dummies guide to your concerns here:
The "doctoring" of the graphic was a change in y axis value or more?
Graphics seem similar and the point about MWP is a reasonable one, so what's the big deal here?
As it says on pp 137 (see link above)
----------------------
The IPCC curve is copied as the dashed red at left. To match the WR black line, the red must be shifted~50 years later, shrunk slightly horizontally and expanded vertically ~30%. That is closer to MM03‘s 1400AD, but still different in shape.
------------------------------
In other words, Wegman, et al. claimed that the endpoint was 2000, but the endpoint was 1950. Since 1950, there has been considerable warming. They also exaggerated the temperature anomaly from the IPCC first report for 1200 by ~30% making the peak at 1200 higher than it was in the IPCC report. Seems a bit of an. . . adventure.
Post a Comment