Saturday, January 26, 2008

On the straight and not so narrow

A day or so ago Eli pointed out that there is a remarkable correlation between all of the global temperature measurements, GISS, HadCRUT, RSS and UAH. Over at Tamino's place some of the unusual suspects are claiming that the GISS temperature record is diverging from the satellite records. The data says no, but it also says some other interesting things. First, let us look at the annual differences between the two microwave sounder reconstructions, RSS-UAH

The diagram shows the difference between the RSS and UAH series, after the different zeroing for the two series has been corrected for by subtracting the average difference over the entire period. The most interesting thing here (at least to your friendly bunny) is that there was a systematically increasing difference between RSS and UAH (shown by the trend line btw 1979 and 2005), but that appears to have decreased starting in 2002 and disappeared in ~2005. That could be a statistical fluctuation or a result of something physical. Perhaps related to the launch of the AMSU on Aqua. Time will tell.
Next, we can look at the difference between the RSS MSU lower troposphere record and the GISS ocean+surface stations record. There is no trend here. It is interesting that in 1998, with a strong El Nino, the lower troposphere (~0.5K) warmed more than the surface (~0.3K)








UPDATE: The abbreviations are getting to the mice. Here is a brief summary:

MSU - Microwave Sounder Unit - First flown on satellites in 1979
AMSU - Advanced Microwave Sounder Unit - Flown first on satellites in the late 1990s
RSS - Remote Sensing Systems - a global temperature reconstruction from the (A)MSUs. Key people are Carl Mears and Frank Wentz
UAH - University of Alabama Huntsville - the original (A)MSU reconstruction, 1993. Key people are John Christy and Roy Spencer, aka Christy and Spencer
GISSTEMP - often called GISS - Goddard Institute for Space Studies surface temperature record. Comes in two flavors, surface stations only, and combined with ocean surface temperatures. Key people are Jim Hansen, Makito Sato and Reto Ruedy
HadCRUT - Hadley Center for Climate Prediction and Research surface temperature record. Comes in several flavors. See the link. Key person Phil Jones

32 comments:

Anonymous said...

As I said at Tamino's, thank you for matching the zeros.

Could you do a quick compare to HadCRU and GISS?

It's my understanding that they both use the same stations, but GISS uses an extraploation for the Arctic.

If I'm correct, there should be a small, continuous pos difference between them, showing that GISS is properly picking up the Arctic warming.

John Mashey said...

RSS(3.1) - UAH

- I assume these are the current numbers with the various corrections that have occurred.

- If in 1995, 2000, and 2005 (or some other relevant set of dates, especially just before changes happened), you had plotted the chart for the years up to that point, how would those charts look?

Put another way, was there less or more divergence earlier?

Anonymous said...

I assume that in the second graph you combined the RSS and MSU record and then compared it to GISS. Am I right, this is not clear in the graph, but is stated in the paragraph above the graph.

EliRabett said...

Sam,

RSS IS an MSU record. I first showed RSS vs. UAH (the two MSU records) and then RSS vs. GISS/ocean-surface stations. I suppose I could put up the UAH vs GISS, but I don't think it adds much.

Anonymous said...

Am confused, I am/was assuming that RSS was the weather balloon troposhpere record, and the MSU was the same from the Satellites. and I am assuming GISS was part of the surface temp record. I must get this more straight in my head. gah sry

EliRabett said...

Sam,

I've put up an update which explains the abbreviations and provides some links.

Anonymous said...

@Henry:
It's easy to do as you ask.

If you subtract GISS Met station data to the Hadley Cru data-- which are both over land only, over the full period when GISS data were collected, the Hadley data "warms" compared to the GISS data. The relative temperature increase is about 0.2 C over the period.


In contrast land-ocean GISS data matches HadleyCru extremely closely.

You can see the plots Charts here

I don't know what this means. Possibly, some measurement bias was either eliminated or introduced by someone, somewhere. Or maybe this is just spurious. (Hopefully, a previously existing bias was eliminated, but just subtracting and plotting can't tell us that.)

However, since these are data, the 0.2C shift could, hypothetically, make some differences in any trend analyses requiring someone to examine the full period. For that reason, it's worth being aware it exists.

Since this seems to be discussed in terms of supporting or refuting warming (as opposed to a simple data quality issue), I'll add this:

Yes, if we look at the various temperature records super-imposed, the central tendency shows distinct warming. This difference between the Hadley and GISS data doesn't affect that. The 0.2C difference over 127 years does not dominate the warming signal.

Anonymous said...

lucia says: "The 0.2C difference over 127 years does not dominate the warming signal."

What is the confidence interval (at the 95% confidence level) associated with the regression coefficient for the difference (Hadley- GISS) trend?

Is that 0.2C over 127 years significantly different from a null result?

I would note that the uncertainty associated with the warming over the last century is given by IPCC as 0.18C (or roughly the same as the "warming" of one data set relative to the other)

Anonymous said...

Thanks Eli,

so how do Radiosonde balloons figure into this if at all? Wikipedia wasn't helpful in the climate context.

Anonymous said...

Yes, UAH, RSS, GISS and CRU are consistent in showing that their has been no warming over the last few years.
Bert Taile

Anonymous said...

Bert Taile:
"Yes, UAH, RSS, GISS and CRU are consistent in showing that their has been no warming over the last few years."

If you mean "a few" as in three (or even 5), that time span is too short to conclude that "there has been no warming". Noise dominates the signal over such a short period.

If you mean "a few" as in "ten" (since 1998 or even 2000), that is simply false as you can see from the Tamino's analysis.

Anonymous said...

I'm not sure what you mean by Tamino's 'analysis'. Just look at the graphs. If you want precision, by few I mean 6, since the alarmists howl with indignation if you go back to the El Nino in 1998. All the data show no warming since 2001. Six years that don't fit the alarmist claim. When temperatures rise, it's 'further evidence of catastrophic man-made warming'. When they don't, it's 'short-term noise'. Ha ha.
Bert Taile

Anonymous said...

You clearly did not understand Tamino's analysis.

Perhaps you do not understand statistics, but you can not "just look at the graphs".

You have to use statistics to make a decision whether a trend ("warming", "cooling" or "flat") is significant at some level of confidence (95% is the standard level for most scientific purposes)

Over short periods, the uncertainty due to noise dominates as this analysis shows, but you are simply mistaken when you say that "All the data show no warming since 2001."

You might want to educate yourself a bit about statistics before commenting further.

here's another good post on the subject

Anonymous said...

Although Eli's HadCRUT link is correct, this HadCRUT link (which can be got to from the original link, via Data/Temperature) might be better.

Cymraeg llygoden

Anonymous said...

Luciea says "If you subtract GISS Met station data to the Hadley Cru data-- which are both over land only, over the full period when GISS data were collected, the Hadley data "warms" compared to the GISS data. The relative temperature increase is about 0.2 C over the period."

0.2 C over 127 years is equivalent to a regression coefficient of 0.0016C/yr.

IPCC provides standard errors for their calculated trends over the past 100 and 150 years of 0.0018C/yr and 0.0012C/yr. (fig 3.1 in latest IPCC assessment report)

If we assume that the standard error for your regression coefficient for 127 years falls roughly in the middle of those two numbers, the standard error for your regression would be about 0.0015C/yr (Note: one would actually expect the standard error for the trend for the difference to be greater than this, since we are talking about the uncertainty associated with a difference between data values, but I will use 0.0015C/yr in my estimate below).

If one uses your regression coeff of 0.0016C/yr and the 0.0015C/yr as a standard error for that coefficient to calculate the confidence interval for that regression coefficient, here is what you get at the 95% confidence level

-0.001369 <= B <= 0.004569

where B is the regression coefficient in degC/yr

Translated to 127 years that means the confidence interval (at 95% level) for the difference (delta) between the data sets is

- 0.17 <= delta <= 0.58

So, the conclusion that "hadley 'warmed' relative to GISS over the 127 year period" may not be warranted based on the statistics.

Anonymous said...

You don't need any fancy statistics Bert Taile. You just need to be able to read and understand when one number is bigger than another one.

HadCRUT3 global temp. anomalies (°C) are as follows:

1998 0.546
1999 0.296
2000 0.270
2001 0.409
2002 0.464
2003 0.473
2004 0.447
2005 0.482
2006 0.422
2007 0.403

Which puts 2002, 2003, 2004, 2005 and 2006 all warmer than 2001!

And the GISS equivalent global temp. anomalies (°C) are as follows:

1998 0.57
1999 0.33
2000 0.33
2001 0.48
2002 0.56
2003 0.55
2004 0.49
2005 0.62
2006 0.54
2007 0.57

on which all years since 2001 are warmer!

So in fact, Bert Taile, you couldn't be much more wrong when you say "All the data show no warming since 2001"!

I daresay most people here can work out where you've gone wrong. Have fun working out why.

Cymraeg llygoden

Anonymous said...

and, Bert, if you don't like numbers, just have a look at this image: http://tamino.files.wordpress.com/2008/01/75-08.jpg
Then tell us what you see.

-Flori

Anonymous said...

You don't need any fancy statistics Bert Taile. You just need to be able to read and understand when one number is bigger than another one.'

The problem with just comparing the numbers is that, even if the temperatures increase from one year to the next over the period in question, one is faced with the question, how much change is really meaningful?

For example, is the GISS temp anomaly in 2007 -- 0.57deg C -- really different from the anomaly in 2001 -- 0.33 C?

Many people would probably say "sure", but it's not that simple, since the fluctuation from one year to the next can be 0.3 deg C (or even bigger) due to noise (eg, due to El Nino).

Furthermore, if the temperature progression over some period of time is not increasing from beginning to end -- ie, has ups and downs -- it is even harder to say what is going on by comparing individual temperatures.

The most meaningful way to address these issues is with statistics -- and pretty basic statistics at that: linear regression.

In other words, you fit a straight line to the data, attempting to minimize the effect of the ups and downs so that they cancel out. Then you estimate the change over the period based on the slope of the line (deg C per year) and the number of years.

Anonymous said...

Anonymous 9:54 am said many things of which I am fully cognisant, but which seemed to matter nought to Bert. That was the point of my earlier post. Now if Bert had said roughly what you'd said Anon 9:54 am, then we would know that Bert is nought but a troll in all likelihood.

But now you've gone and spoiled that little game ;-)

Cymraeg llygoden

Anonymous said...

recent trends are not statistically significant, so you can see what you like in the numbers, according to your prejudice - for example 06 and 07 are cooler than any of the 4 previous years according to HADCRU.

John Mashey said...

I normally keep "anonymous" KILLFILEd, but with 11 straight postings, I turned it off, only to realize there were several people all posting as anonymous arguing with each other. Sigh. Waste of time and disk space.

EliRabett said...

Sam,

UAH calibrates it's readings to sonde's at places where they occur simultaneously (the sonde record has its own problems, it is not a gold standard). Tamino has put up a better list of links to the data. What you probably need is one of the UAH papers which describes the process in detail.

EliRabett said...

GENTLEBUNNIES,

Eli shouted above the anonymouse roar. PLEASE take a number when you post!!!!

EliRabett said...

Lucia,

Try that the oceans are about 75% of the surface and that they are both using the same ocean data:)

Anonymous said...

john Mashey said; "I normally keep "anonymous" KILLFILEd"

Do what you want but why inform the the rest of us about how clever you are?

John Mashey said...

I'm not trying to be clever or impress anyone.

I'm trying to help the quality of discourse, and the S/N ratio, because I hate to see good blogs damaged and eventually destroyed in the same way that happened to many once-good USENET newsgroups.

Anonymous said...

John,

I see your killfile program has a bug in it. :)

Anonymous said...

Eli--
First: Darn I hate blogger comments! Did the one I entered get muched? Am I repeating myself?!

Second: Cute brief answer which explains why the GMST measurement by GISS and Hadley should match. The problem is: The GMST measured over land only don't. match.

So, you see, the fact that there is a lot of ocean doesn't explain the mismatch over land. :)

Anon 1 and Anon 2--
This is a measurement issue-- not a global climate change issue. It has nothing to do with IPCC"s predictions for the rate of change in the actual measurements. I added a graph to show someone the answer to a question. That actual change iss shown by the yellow and blue dots in the figure here

The rapid increase since 1980 on in those data are what is supposed to increase as suggested by IPCC, and have the uncertainty intervals that IPCC suggests.

But that rate of expected rate of increase in GMST has nothing to do with whether or not the two groups measurements of GMST should disagree with each other.

The difference in GMST (CRUT-MET) over time, as reported by HADLEY and GISS is plotted by the red symbols. While we would expect some scatter about a horizontal line, it's the slope in the trend line through that data that should be there.

Whether or not the globe warms (and both sets of data say yes), the two measurements of GMST over land should stay on track with each other.


As for statistical significance, it turns out, based on the Fdistribution, the trend I described is significant to the 99.9999999997% level.

(I have to admit, I didn't adjust for for autocorrelation. The residuals for adjacent years have a correlation coefficient of of 0.13, which is fairly uncorrelated. But, you could correct for this if you like.)


As I said: I don't know what caused this, or what it means.

I only discovered it because I wanted to look at whether the land/sea measurements lagged the land only measurements, and noticed that I got quite different answers depending on which data set I used.

EliRabett said...

Lucia,

Eli can only answer the questions that are asked. True it was both obvious and short, but what the heck I had to think about it for a while.

I would suggest that if you want to make a serious attempt you look not at the global numbers but at regions.

Anonymous said...

Eli--I know you can only answer what is asked.

The problem wasn't that your answer was obvious or short. The problem was that it was entirely wrong.

The fact that GISS and Hadley use the same data, cannot explain why the trends for warming over land disagree by 0.2C since 1880.

And of course, to make your answer odder still, the fact that many measurements are made over oceans, doesn't explain why the land only (i.e. no ocean) measurements disagree.

And so, the mystery of the disagreeing GISS/Hadley values reported over land only remains entirely untouched by your brief quip. :)

Savvy?

Anonymous said...

Lucia said: "This is a measurement issue-- not a global climate change issue. It has nothing to do with IPCC"s predictions for the rate of change in the actual measurements."

Understood. I only quoted the IPCC trends because they give an idea of the standard error for the regression coeff of a trend for a series of about 100 annual mean temperature anomaly values, when the uncertainty attached to each value is of order 0.1C.

If you are taking the difference between the temperature values in two data sets (from which you then do your regression) the associated error is basically twice what it would be for either one (ie, 0.2C) assuming the error for individual values in each data set is 0.1C.

This increased error in individual measurements propagates to the standard error in the regression coefficient for the difference trend.

Lucia also said:
"the trend I described is significant to the 99.9999999997% level."

I don't believe it. My goodness, "six sigma" confidence would not even be significant to that level, but IF we assumed six sigma confidence, that would mean that the standard error associated with the regression coeff for your difference trend (0.0016deg C/yr) would have to be less than 0.0016/6, or less than about 0.0003 deg C/yr

As a comparison, the standard error associated with the the 150 year temp trend given by IPCC that I quoted above is 0.0012C/yr (or 4X as large as 0.0003). (And you are claiming higher confidence than 6 sigma?)

As i indicated above, the latter IPCC trend is for annual mean temperature anomaly values, not a trend for the difference between such values -- which should yield a LARGER associated standard error for the regression coeff.

Even assuming 2 sigma confidence would mean that your standard error would have to be less than 0.0008 deg C/yr, which is STILL less than the standard error for the 150 year trend given by IPCC.

Something is fishy.

So, what is the standard error associated with your regression coefficient?

Anonymous said...

IF we assumed six sigma confidence, that would mean that the standard error associated with the regression coeff for your difference trend (0.0016deg C/yr) would have to be less than 0.0016/6, or less than about 0.0003 deg C/yr,

Thank you for bringing my attention to your reply over on the other thread. And the answer is, the uncertainty in the slope is lower than 0.0003 C/yr! :)

I'm not sure why you are trying to relate the uncertainty in the the historic instrument measurement recort to IPCC predictions for future warming. They are entirely different things.


My main advice to you is, rather than resorting to wordy arguments about the statistics, you download the GISS LAND/Ocean data and Hadcrut data, and perform both the trend fit and the uncertainty analysis yourself.

It's fairly easy using any number of statistics packages.