Thursday, January 14, 2010

Numberology

Remember Darwin Dearest? You know, how Willis Escherbach manipulated the data from Darwin Australia to make a warming trend into a cooling. Tim Lambert dealt with this in detail and Willis of course has a number of priors.



What Willis forgot to correct for according to the Australian Bureau of Meteorology was

A change in the type of thermometer shelter used at many Australian observation sites in the early 20th century resulted in a sudden drop in recorded temperatures which is entirely spurious. It is for this reason that these early data are currently not used for monitoring climate change. Other common changes at Australian sites over time include location moves, construction of buildings or growth of vegetation around the observation site and, more recently, the introduction of Automatic Weather Stations.

The impacts of these changes on the data are often comparable in size to real climate variations, so they need to be removed before long-term trends are investigated. Procedures to identify and adjust for non-climatic changes in historical climate data generally involve a combination of:

  • investigating historical information (metadata) about the observation site,
  • using statistical tests to compare records from nearby locations, and
  • using comparison data recorded simultaneously at old and new locations, or with old and new instrument types.
Well guess where the tortured numbers pop up again. Three points to the guy in the corner who said, gee, that's just the sort of thing that Joe D'Aleo would love and five to the lady in the black hat who said, that's so good it should be featured on the new KUSI blockbuster show we set our TIVO to capture when we went to the tea party. It would be nice to see a libel suit dropped on these clowns, but it's time to saddle up and point out that the icecap man is melting. Oh yeah, here is the "programming expert", please help undress him

There is more of the usual, the number of stations has fallen over the past thirty years, etc. Pushback will be needed.

Comments?

17 comments:

Chad said...

The 'programming expert' stepped on a rake with claims of false precision in the surface temperature record. I looked into the issue here as did Lucia here and here.

Anonymous said...

MarkeyMouse says: I take it the equipment change adjustment was in 1940. Still leaves a substantial wrong upward adjustment to the data, given that the other factors you mention, in summary, the UHI effect, ususally militate in the direction of further false temperature increase readings.

Martin said...

Oops.


integer incount(12), nncount(12), itmptot(12), ntmptot(12)

...

tmpavg(m) = (itmptot(m)/incount(m))/10.

...

ttmpavg(m)=(ntmptot(m)/nncount(m))/10.

Oops.

...and precisely what is he averaging? Station values?

Is this for real?

carrot eater said...

I wasted a bit of time on this one over at WUWT. By the way, you don't show the plot that Willis was basing most of his screaming on.

In the original post, you can tell Willis read at least one relevant paper from Peterson at the NCDC that described the adjustment process. He in fact quoted it at length. In the post itself (but not the later comments), he just then went on as if he'd never read it.

Now, keep in mind that the Della Marta paper you quote describes how the ABoM does their own homogenisation. This is independent of the GHCN. The ABoM uses historical metadata (first bullet point), whereas the GHCN adjustments Willis is looking at do not. The GHCN relies on statistical tests using neighboring stations (the second bullet point).

Last I looked, Willis had refined his argument a bit. See those tiny little adjustments before 1930? Willis says he's having trouble finding the sufficiently correlated neighboring stations that would let the GHCN make those adjustments, and that therefore they did something outside of the published procedure.

I lost interest because by this point, others had showed that in the big picture Darwin was an outlier, and also because Willis's spectacular claims are appealing to the larger adjustments after 1940, but what little substance he had was looking at the tiny blips in the 1920s.

To the extent that tiny adjustments at a single station that don't affect the overall trend (even at that station, certainly not to the global) are interesting, somebody could try to reproduce the GHCN method there. But I haven't had time or interest to deal for the last few weeks.

carrot eater said...

And I didn't think I'd ever defend Willis in this stupidity, but I'd object to your opening. He didn't manipulate any data. He plotted it pretty much as obtained; he just nudged the series up and down so that they coincide in the beginning, instead of at the end. That doesn't really change anything, but it gives the visual impact he was looking for.

Dano said...

I think we should open an InTrade for when someone like NRDC or EDF ponies up a junior staff or two to take on a libel suit or FOIA campaign.

Best,

D

word verif concurs: amitia. Yup. I miss ya sending some growling lawyers too.

guthrie said...

I trust people are keeping copies of the pages in question in case they get disappeared?

guthrie said...

I trust people are keeping copies of the pages in question in case they get disappeared?

Deech56 said...

Crikey, watt's with all the fawning over @ WTF?

From Mr. Ball:

Proud to be able to say I post on the same blog as E.M. Smith.

From mkurbo:

Also (as others say) proud to be able to say I post on the same blog as E.M. Smith.

It's like they're basking in the presence of the great pumpkin.

carrot eater said...

Who the dickens is E. M. Smith?

And what, do they think they have to pass some sort of aptitude test to post comments there?

CapitalClimate said...

It's an aptitude test, all right; aptitude for delusional thinking.

bluegrue said...

@martin:
thumbs up on the find

@guthrie:
webcite is your friend, here's an archived copy of chiefio's page, which Eli referenced above.

Deech56 said...

I'm putting all of gavin's in-line responses to me in my scrapbook.

Even Steve replied to me once. *sigh*

NikFromNYC said...

Do you really find the hand waving argument about a *possible* shelter-change works here? A shelter change is a single day event which could only result in a step jump in a single year. No single-year step change is found in the raw data. Only a 3-4 year long ramping down is seen. Willis and commenters fully discussed adjustment types, and none of them were plausible explanations for either the step adjustments or the ramp adjustments.

Your argument style is has a tabloid style ring to it, like you really think mere sound bites one-up your opposition, as if they are blundering idiots who "miss" huge holes in their arguments. But you should stop using the "well, if I'm wrong, it doesn't matter any way..." endings since you really can't have it both ways and hope to end on such a triumphant note. If you really want to throw single-station speculation posts into question, there are much better debating strategies, and you must know that (the recent demonstration that GHCN adjustments are nearly equal in positive vs. negative alterations in individual station slopes pretty much cinches it).

This blindness to real debate exposes your motivation. It's not reasonable and convincing debate you are interested in. It's self and public image maintenance, namely the feeling that you are some genius dealing with clownish buffoons, preaching to a choir of other geniuses. If that gets you off, fine, but this exact attitude mints many a skeptic when they early on start commenting on pro and anti AGW blogs.

In your spare time, feel free to sound-bite debunk the biggest glaring hole in AGW science of all: actual very long running SINGLE SITE thermometer records spread over two continents, steadfastly refuse to show any trend resembling a Hockey Stick. I've plotted all of the the longest record cities of Europe and America that most Americans have heard of that lack big gaps in the data (and added Copenhagen for fun even though it's not very long). My emphasis on single site records prevents any debate about statistical methods. Many more plots can be found at ClimateReason.com (who I have no association with). There is NO HOCKEY STICK in actual thermometer records, only an occasionally wavy blade that continues back to 1659 (!). I thus grandly refute AGW theory.

The NOAA's own web site sheds great doubt on AGW theory, based on a true global average. They must resort to bizarre data plotting formats to hide this fact. I unravel their subterfuge using their own plots here and here.

The only remaining argument is that there would have been a negative break in trend that AGW has prevented. That barely passes the laugh test, I'm afraid.

Dano said...

Nik is independently wealthy from the proceeds and residuals from his perpetual motion machine, thus his wide-ranging and independent study into physics and climate to ekspoze th' SCAY-UM..

Best,

D

carrot eater said...

Nik,
Willis did no such thing (meaning, he never did a thorough job of thinking about what all could have happened there, to require adjustments). Somebody dug out the site metadata from the Australian web page, and there were all sorts of issues with the Darwin site. Not just multiple site moves, but other factors that would degrade the quality of the data at different times (one can imagine that maintaining the weather station was not a high priority while the Japanese were bombing the place).

In any case, the GHCN doesn't know about any of that. All it sees is the raw data, and the raw data of the surrounding stations. From that, it makes adjustments. So the GHCN adjustments won't match up exactly with the adjustments you would make if you had the site metadata at hand, but they'll roughly correspond. This shouldn't be difficult to grasp, if you review the algorithm.

guthrie said...

NikfromNYC- have you disproven the urbani island heat effect? MarkyR was complaining in another thread that McIntyre had shown that the UHI contaminated the temperature record. So have you taken it out or what?