Backwards
In 2000, NOAA changed some of its reporting for the US Historical Climate Network Data. In 2007, Steve McIntyre discovered an inconsistency about how GISSTEMP processed the information. Obviously this error could only affect post 2000 data, but mysteriously many reports claimed that this had forced GISS to re-rank 1934 as the warmest US year instead of 1998 and the change had arisen from the correction found (Eli will resist the temptation to snark) by Blogger McIntyre (sounds Canadian, eh?). Even Gavin Schmidt on Real Climate appears to have gotten this wrong on 10 August 2007, which was very early days in the controversy
Frankly Eli paid it no never mind at the time, because the difference between 1934 and 1998 was not even fractional either way (0.01 C or so). There was a significant effect on the post 2000 years which affected their rankings for the US data. Still there were lots of such reports
We find the solution in an Email from Makiko Sato to James Hanen 24 Aug 2007:
"Of course Reto thinks the ranking that shows which year was warmer by 0.01 deg is stupid. But as long as I give the table of US mean temperature on our web site, people can make rankings themselves. What Reto wanted to tell you was from Jan 7 - Aug 7, 2007 we had 1998 warmer than 1934 by 0.01 deg in the table I showed on our web page. (The reason was those numbers keep changing by such a small amount by adding station data, as probably as Reto pointed out, we processed data in January before a lot of data came in. These recent data can change numbers in old time by small amounts.) From next time I will update the US mean table every month. I was doing it only once a year because I didn't think people would make such a mess out of 0.01 deg difference in US. 0.01 deg is negligible globally but even 2% of that for the US."Reto Ruedy lays out the details of the thing on 23 August 2007
The US temperature graph in our 1999 paper, based on GHCN data, shows 1934 0.5 C warmer than 1998; 1998 was in 5th place behind 1921, 1931, 1938, 1953.Remember this is for the US, something a lot of people disremembered sort of on purpose
In the corresponding graph in our 2001 paper, now based on the carefully corrected USHCN data, 1934 and 1998 are in first, 1921 in third place (NOAA who provided the USHCN data had 1998 slightly ahead of 1934).The final summary on GISSTEMP for 2007 states
The US table we had posted during all of 2006 showed 1998 and 1934 even at 1.24 C (I got a copy from a journalist in Brazil, we don't save these data).
As far as I know, the US table on our site from Jan to Aug 2007 was the first and only one with 1998 ahead of 1934, some US stations must have still been missing in the GHCN file we downloaded on January 8, 2007. (Each month GHCN regererates the whole file over a period of a few days; In previous years we had to wait til mid January for the US stations to be added in again.)
Finally, we note that a minor data processing error found in the GISS temperature analysis in early 2007 does not affect the present analysis. The data processing flaw was failure to apply NOAA adjustments to United States Historical Climatology Network stations in 2000-2006, as the records for those years were taken from a different data base (Global Historical Climatology Network). This flaw affected only 1.6% of the Earth's surface (contiguous 48 states) and only the several years in the 21st century. As shown in Figure 4 and discussed elsewhere, the effect of this flaw was immeasurable globally (~0.003°C) and small even in its limited area. Contrary to reports in certain portions of the media, the data processing flaw did not alter the ordering of the warmest years on record. Obviously the global ranks were unaffected. In the contiguous 48 states the statistical tie among 1934, 1998 and 2005 as the warmest year(s) was unchanged. In the current analysis, in the flawed analysis, and in the published GISS analysis (Hansen et al. 2001), 1934 is the warmest year in the contiguous states (not globally) but by an amount (magnitude of the order of 0.01°C) that is an order of magnitude smaller than the uncertainty.Never mind although Eli assumes some will proceed to have the vapors.
Posts
26 comments:
The TOB adjustments made to the USHCN caused a pretty good shift in the US numbers when they were made. After that point, all the tinkering hasn't really amounted to much change.
An actual measurement, probably a sonde. Grabbed this off a meteo class page. Take a look at the wiki to see the differences between the wet and dry adiabats
Recently, with the release of Giss emails we found out that the difference between 34 and 98 was 1.4 to 0.9 -- much greater than 0.01.
http://noconsensus.wordpress.com/2010/01/15/1934-1998-gissmatic/#more-7549
It was during those 10 years of very active adjustments that the value dropped to 0.01. NicL just posted at the Air Vent an analysis of GHCN which includes a US analysis which by using all long record stations has it back at the original 1999 ratio.
http://noconsensus.wordpress.com/2010/01/19/long-record-ghcn-analysis/
The US is only 2% of the earth but contains the vast majority of temperature stations. I don't claim this makes any difference to global trend but the modifications to the US record are suspect.
The global picture doesn't come into play until we work out why the raw vs adjusted versions of the US have so much difference.
Jeff Id
"The global picture doesn't come into play until we work out why the raw vs adjusted versions of the US have so much difference."
Instead of trolling around, you could actually read the f'ing papers.
The adjustments in the USHCN are perfectly well described in the literature. The big one is time of observation bias TOBS. Read the papers. There were some big adjustments when these methods were first introduced; revisions since then have been fairly minor.
This isn't something you need internal GISS emails for; it's something you'd have known for years if you followed the literature. The NOAA/NCDC hardly hides the adjustments it makes for the USHCN; they're published left and right.
Why would you assume I haven't read the papers?
Since I have, do you think it's possible to question what's in them?
Let's see, perhaps because your comment shows absolutely zero sign of you having read the papers?
In your comment, I see no discussion of the methods behind the TOB adjustment.
What I do see is "Recently, with the release of Giss emails we found out that the difference between 34 and 98 was 1.4 to 0.9 -- much greater than 0.01."
So apparently you only recently, with the GISS emails, found out that something changed in the GISS numbers around 2000. If you had read the papers and knew what you were talking about, you'd know that's when GISS began using USHCN data adjusted for TOB and inhomogeneities.
You go on then to declare that "but the modifications to the US record are suspect."
Right. You seem to reach that conclusion based on the mere fact that adjustments exist, not based on any analysis of the methodology.
Then you close with "until we work out why the raw vs adjusted versions of the US have so much difference."
You'd know why there is a difference there if you actually read the papers which discuss this very thing.
So it's actually quite justified to assume that you haven't read them, or at least, didn't understand or remember anything about them. Your comment told me as much.
The US is only 2% of the earth but contains the vast majority of temperature stations. I don't claim this makes any difference to global trend but the modifications to the US record are suspect.
Which modifications exactly Jeff? And how? Not with a link somewhere, but just briefly, in basic terms.
Eater,
"So apparently you only recently, with the GISS emails, found out that something changed in the GISS numbers around 2000. "
Not so my new friend, in fact what I found out was that the changes were so severe that 1934 dropped in relation to 1998 by 50%.
"Right. You seem to reach that conclusion based on the mere fact that adjustments exist, not based on any analysis of the methodology."
Well, I didn't feel a discussion of satellite image based UHI corrections deserved any time here.
What do you think after reading the paper, suspect or no? I think if you're honest with yourself, you'll agree with my assesment.
"You'd know why there is a difference there if you actually read the papers which discuss this very thing."
Well the metadata from GHCN doesn't do a great job with tobs (to say the least). There are a large number of steps and a huge amount of station starts and stops, all of which affect trend.
"So it's actually quite justified to assume that you haven't read them"
I think it's more likely it just makes you feel good to yell at the id and you felt you had an easy target.
BTW, as anexpert, have you ever downloaded the dataset yourself?
What did you find?
------
Jim, check out the link to Nic's post. It's just the data no magic.
"Not so my new friend, in fact what I found out was that the changes were so severe that 1934 dropped in relation to 1998 by 50%."
That's an odd way of phrasing it. Stick to saying the difference fell by about 0.5 C, which is plenty big. I didn't offhand know what that magnitude was, so I can give you a pass if your surprise is the exact magnitude of the change. But it should be no surprise that there was a major change in analysis at that time.
"Well, I didn't feel a discussion of satellite image based UHI corrections deserved any time here."
GISS adjusted their UHI correction between 1999 and 2001, but I think the bigger change then was due to their adopting the USHCN TOB and homogeneity adjustments. Anyway, the UHI adjustment would be pushing things the other way. So it's TOB and homogeneity you should be discussing.
"Well the metadata from GHCN doesn't do a great job with tobs (to say the least)."
To say the least, I'm sure as heck not going to take your simple statement on TOB with any regard. Do some work and show how/why TOB in particular is not being handled well. Then we'll talk. And you made this statement specifically about TOB, so isolate on TOB.
"you felt you had an easy target. "
I did have an easy target. Somebody making blanket statements that adjustments are suspicious, without showing any sign of having read the material describing the adjustments.
Ball's in your court. Back up your assertions. What is so wrong with the NOAA's TOB adjustment?
Jim, check out the link to Nic's post. It's just the data no magic.
C'mon Jeff, I said no links please. Surely you can give us a brief synopsis of why TOB or other bias corrections to USHCN data are so "suspect", given the amount of time you seem to have spent on this topic.
Carrot,
I don't stop by here often, I thought the readers might be interested in some of the GHCN data. Nic did a nice job.
Are you Eli, or does your moniker represent someone else.
Now with respect ot your unfounded statements on reading papers. You assumed I hadn't read the papers, I assumed you would recognize my name. Whatever.
Why wouldn't we discuss UHI no matter which direction you assume it's going? You assume further that I would only look at cooling vs warming and ignore the other side.. Now why the hell would I do that?!
You are far to quick to judge which way a change might go.
I made no claim about TOB being right or wrong, just thought I'd point out some interesting detail in the data. The corrections are very heavy to say the least.
Again, I wonder with your expertise, have you ever downloaded or directly looked at the GHCN data or metadata yourself?
"Never mind although Eli assumes some will proceed to have the vapors."
We have a winner.
> Blogger Jeff said...
... Recently, with the release of Giss emails we found out ...
... noconsensus.wordpress.com
... NicL just posted at the Air Vent ...
"we"?
What you mean we, Kemo Sabe?
Not getting enough clicks over there at the mail pirates' blog?
Good luck recruiting.
Hit's are doing well, hadn't cheked in a while. Thanks for asking.
I'll leave you boys alone now.
I am not Eli.
"Why wouldn't we discuss UHI no matter which direction you assume it's going?"
The issue you raised was the relative difference between 1934 and 1998 in the US data as produced by GISS. To the extent that this has changed over time, the biggest change in analysis occurred around 2000. And the biggest part of that story is TOB and homogeneity, as adopted from USHCN (before 2000 these were not used by GISS). Yes, the GISS UHI algorithm was also updated at the same time, as discussed in the joint NCDC-GISS paper, Hansen et al 2001. But in terms of what happened to the analysis around 2000 to bring 1934 and 1998 closer together, the change in UHI method is not the first place to look.
"I made no claim about TOB being right or wrong,"
You said the following:
"Well the metadata from GHCN doesn't do a great job with tobs (to say the least)."
Certainly sounds like there is something about TOB that you don't like. Care to clarify the statement?
Or is it just that you don't like the results of TOB, and have nothing substantive to say about the TOB method?
"The corrections are very heavy to say the least. "
In USHCN, the sum of all adjustments is not insignificant. That's never been hidden. Observing that, in itself, is not a contribution.
"Again, I wonder with your expertise, have you ever downloaded or directly looked at the GHCN data or metadata yourself?"
I've messed around with it some; not much. But don't bore me with a lecture about your other hobbyhorses. The specific question I'm asking you is what you don't like about the change in adjustments made around 2000, and TOB in particular.
"The issue you raised was the relative difference between 1934 and 1998 in the US data as produced by GISS."
I am interested in the quality of surface temperature data but it's understandable that you would assume I was only interested in a monolithic aspect of it.
"Well the metadata from GHCN doesn't do a great job with tobs (to say the least)."
This was a test of your understanding, there are a lot of pretenders on the web, as you probably realize. There is no TOBS metadata in GHCN. It must be out there somewhere, but GHCN doesn't seem to have any.
I wasn't complaining about TOBS corrections but the lack of metadata with the slim hope that someone helpful would provide direction.
I've only been digging into this data for a few weeks although I've read up on and off for about 2 years. It appears that my two weeks of knowledge has already exceeded your own, at least on this matter, yet you have chosen to attack. It's an odd personality trait considering you aren't familiar with the data yourself.
If you're interested, Dr. Loehle has a post on tAV which goes into this in more detail.
Apologies if this has already made it's way to the net, but I'm currently at the AMS National Meeting and there was an interesting presentation about the USHCN and Watt's surfacestation.org by people from NCDC. To make a long story short, NCDC did an analysis of warming based on the station's "exposure" classification by Watts's merry crew.
The punch line: "poorly" situated stations show less warming than "well-sited" situations. The probable reason for this: "poorly-situated" stations are predominantly MMTS (aspirated thermistor) sites which have a known cool bias compared to the Liquid in Glass stations (which tend to make up the "well-sited" sites.) Also, to paraphrase the presenter, "Watts said he was going to work with us on this, but then he stopped talking to us". Geez, what a surprise...
I don't know what happened with Watts and NCDC.
As I understand it now, instead of working with him on his research project, they have twice went ahead and published incorrect results without consultation or credit using his hard work.
What would you do if you put together years of work, including time and money, share it with the pros', only to have the work used incorrectly and without credit or consultation.
Unprofessional doesn't even begin..
be fair.
Your assertion of "unprofessional" is without merit, the NCDC presenter cited the Heartland Institute paper like it was a serious, peer-reviewed journal article (I sure wouldn't) and he specifically mentioned the surfacestations.org URL repeatedly.
In any event, it doesn't change the findings.
"In any event, it doesn't change the findings."
We'll see.
Jeff is beating around the bush, I see.
"I am interested in the quality of surface temperature data but it's understandable that you would assume I was only interested in a monolithic aspect of it."
You made a very specific comment about what happened to the relative difference between 1934 and 1998 in the GISS US analysis around the year 2000. So, let's address that specific point, shall we?
"This was a test of your understanding, there are a lot of pretenders on the web, as you probably realize. There is no TOBS metadata in GHCN. It must be out there somewhere, but GHCN doesn't seem to have any."
Stop beating around the bush. We were talking about the USHCN, which has TOB metadata and makes use of it. The GHCN has nothing to do with the current topic, which is the USHCN and what GISS does with it.
"I wasn't complaining about TOBS corrections but the lack of metadata with the slim hope that someone helpful would provide direction."
The NOAA/NCDC makes the US TOB adjustments using the metadata relevant for the TOB.
"I've only been digging into this data for a few weeks although I've read up on and off for about 2 years. It appears that my two weeks of knowledge has already exceeded your own, at least on this matter, yet you have chosen to attack. It's an odd personality trait considering you aren't familiar with the data yourself."
Wow, that's special.
"As I understand it now, instead of working with him on his research project, they have twice went ahead and published incorrect results without consultation or credit using his hard work."
Incorrect because you don't like the results? Or are you just going to say some vague thing about homogenisation? Ooh, there's adjustments, we can't ever trust adjustments.
Anyway, Watts has been cited. So the initial article didn't have the citation quite the way Watts wanted it. Get over it.
jknapp
You state "Obviously this error could only affect post 2000 data" but I believe that GISS uses current data to re-analyze previous estimates and infill data. Thus corrections post 2000 can and do change the averages for preceeding years.
Mphysopt:
The corresponding paper is in press, and I recommend everybody to have a look.
For the whiners, Watts is prominently cited, and there are repeated mentions to surfacestations.org. So get over it.
http://www1.ncdc.noaa.gov/pub/data/ushcn/v2/monthly/menne-etal2010.pdf
Teapot teempest.
"What would you do if you put together years of work, including time and money, share it with the pros', only to have the work used incorrectly and without credit or consultation."
Since I am a scientist, I adhere to the universal scientific convention that any other investigator may freely use any information that I make publicly available, for whatever purpose they choose and without the need for my approval, and that all they owe me is a citation. If I disagree with their conclusions, I'll say so in my own papers.
Post a Comment