In the spirit of Deep Climate, Anonymous, as he wrote to Eli, has penned an analysis of how Richard Tol cooked and indeed, it is McIntyre class, although from the nature of the mistake, it may very well have been hubris and not malice. Dickie's misbegotten analysis starts from Cook et al's description of their procedure, and just to show that Eli is an honest card shark, why not start with Tol's description first of what he did, and then of what Cook et al did. At the end of this it should become clear that, well, Eli will leave that to the bunnies. From Tol's blog
According to Cook et al., each abstract was assessed by at least 2 and at most 3 raters. In fact, 33 abstracts were seen by only one rater, 167 by four raters, and 5 by five.Eli will not get into why at this time that of the 12,000 odd abstracts 33 were seen by one rater, but bunnies might ask if those 33 had something about them and whether they were included in the published ratings. Tol rants on
If the initial ratings disagreed, as they did in 33% of cases, abstracts were revisited by the original raters. In 15.9% of cases, this led to agreement. In 17.1% of cases, a third rater broke the tie.
A reported error rate of 33%, with 2 ratings and 7 categories, implies that 18.5% of ratings were incorrect. 0.6% of abstracts received two identical but wrong ratings. 2.9% of ratings are still wrong after reconciliation. 3.2% of ratings are wrong after re-rating. In total, 6.7% of reported data are in error.At this point Dickie goes totally off the rails. That there were disagreements in the ratings is clear, such disagreement is to be expected when one imposes an ordinal value on a continuous one. For example, if bunnies measure a continuous value of 3.5 + 0.1 imposing an ordinal value will in half the cases result in an ordinal value of 3 and in the other half of the cases in an ordinal value of 4. In this case it is not surprising that in one third of the cases there was a disagreement amongst the raters.
Where the separation between ratings is only one unit these are not disagreements in any meaningful sense and anyone who claims that they are is fooling himself or attempting to deceive the reader. Those perhaps interested in cross tabbing can look at those from Brian's survey of 666 (yes, Eli reads the Old Testimony) abstracts selected at random from those published between 2002 and 2007 (more about the prequel here).
Tol shows a figure indicating how in the reconciliation and re-rating processes, ratings changed. For 92% of the cases the ratings changed by one unit.
Anonymouse describes Tol's mathturbation
Tol (2014) argues reasonably that from the number of disagreements, the error rate in the initial abstract ratings was about 18.5%. He further argues that if the same error rate applies during the reconciliation process then 6.7% of ratings will still be in error after reconciliation, implying that 11.8% were corrected during reconciliation. He assumes that the remaining errors are equally distributed among categories, a claim which is problematic but which will be assumed for the remainder of this analysis.Eli disagrees with this on two grounds. First, as mentioned previously, these differences, and certainly the differences of one unit are in no way errors. Second, the assumption that the remaining differences are equally distributed among categories is more than problematic. It is WRONG and the data showing that it is WRONG have been made public. Anonymouse continues
In Tol’s analysis the 6.7% of error ratings are redistributed to other categories in proportion to the corresponding proportion of shifts in the histogram. Shifts which would move the abstract rating outside the 1...7 range leave the abstract in the most extreme category.
This operation on the error ratings may be represented by the matrix S
T = (1-0.067)I - .067S using the 6.7% disagreement rate.
UPDATE: Everyone, inc. Eli got gremlins. See comments by Neal King
As Tol points out in his blogspot, AS does get the wrong matrix S. However, his calculations have been redone with the correct matrix S, and yield essentially the same results:
- If one were to expect Tol's approach to be applicable to the only solid evidence there is (the records of the reconciliation), the approach requires that the initial distribution have negative numbers in one category, prior to reconciliation; and
- the results of Tol's projection bear no similarity to the statistics they seek to model: How does a 2:98 split turn into a 55:45 split? via the power of Tol's assumptions! and this is for the category that comprises 2/3 of the papers that were studied.and Mark R
- For those gentlepeople of leisure who don't want to muck around in excel, here is Tol's 'S' matrix: http://textuploader.com/0gj6
Elements 1,1 and 7,7 should be zero (otherwise it's not an error), so to make the columns sum to unity each of the remaining values are scaled by a constant to normalise.
During reconciliation, the consensus went from 96.73% to 97.06%.
The Cook response method predicts that reconciliation changes consensus from 96.73% to 97.06% (versus observed 97.06%).
The Tol comment method predicts that it changes from 96.73% to 87.44% (versus observed 97.06%). Working backwards, Tol's correction requires that the pre-reconciliation consensus should be >100% (as pointed out by Anonymous).
I'm sure that Professor Tol will realise and retract his paper or, at the very least, submit a quick correction.
| 0.0 | 0.429017350 | 0.030959032 | 0.001167679 | 0.000468604 | 0.000000000 | 0.000000000 |
| 0.916281027 | 0.0 | 0.415735568 | 0.031060252 | 0.001171509 | 0.000487329 | 0.000000000 |
| 0.068994531 | 0.523180399 | 0.0 | 0.417094816 | 0.031162137 | 0.001218324 | 0.001038422 |
| 0.006731174 | 0.039394667 | 0.506983240 | 0.0 | 0.418462980 | 0.032407407 | 0.002596054 |
| 0.007993269 | 0.003843382 | 0.038175047 | 0.508640822 | 0.0 | 0.435185185 | 0.069055036 |
| 0.000000000 | 0.004564016 | 0.003724395 | 0.038299860 | 0.510309278 | 0.0 | 0.927310488 |
| 0.000000000 | 0.000000000 | 0.004422719 | 0.003736572 | 0.038425492 | 0.530701754 | 0.0 |
Which agrees with Tol's. This is somewhat hidden in the URL that Neal gave. Bunnies have to download the Excel spreadsheet http://www.sussex.ac.uk/Users/rt220/Consensus.xlsx and also http://www.sussex.ac.uk/Users/rt220/tcp_allratings.xlsx which are under the headings Data and Graphs on Abstract Ratings and Individual Ratings
--------------------------------------------
The problem being that if you use the inverse, the percentage agreeing with the IPCC consensus is 116% and the number of papers rejecting the consensus at level 5/7, 7 being outright Dragon Slayer territory, is -555 which is the new number of the beast. The two errors in the S matrix will not change that much
What to say, what to say. . . First of all shifts that would move the abstract out of the ordinal range should have a probability of zero. This unjustifiably inflates the number of abstracts at the extremes. Second, Skeptical Science has just published it's full response, and bunnies can see how Tol's assumptions lead to Tol's conclusions
Which are wrong
As Tol points out in his blogspot, AS does get the wrong matrix S. However, his calculations have been redone with the correct matrix S, and yield essentially the same results:
ReplyDelete- If one were to expect Tol's approach to be applicable to the only solid evidence there is (the records of the reconciliation), the approach requires that the initial distribution have negative numbers in one category, prior to reconciliation; and
- the results of Tol's projection bear no similarity to the statistics they seek to model: How does a 2:98 split turn into a 55:45 split? via the power of Tol's assumptions! and this is for the category that comprises 2/3 of the papers that were studied.
Neal, where is the correct S matrix? Tol is not very helpful on pointing to things
ReplyDelete" Tol's assumptions lead to Tol's conclusions"
ReplyDeleteWell, he is a practitioner of the 'dismal science', after all.
Anonymous Etc.
Don't miss SkS's "24 Errors" PDF: http://sks.to/24errors
ReplyDeleteIt's been a rough year for the Good Professor.
Hey, weird problems that don't really need to be solved are hard to find. Sometimes you just have to take what jobs you can get, lol.
ReplyDeleteThe most entertatining aprt of the papere maybe be:
ReplyDelete"Acknowledgements
I had useful discussions with Andrew K, Brandon Shollenberger, Carrick, Dana Nuccitelli, John Ray, Joshua Halpern, Kenneth Fritsch, Lance Wallace, Lucia Liljegren, Max Auffhammer, mt, Shub Niggurath, Willard and wottsupwiththatblog.
Two anonymous referees and Lorna Greening made helpful comments.
I am unaware of any previous peer-reviewed paper that Ack'd a >a Lovecraft entity, but there must be a first for everything.
I suppose Anthony and his crew are the Crawling Chaos?
ReplyDeleteThe correct version of the matrix S has been provided at: http://www.sussex.ac.uk/Users/rt220/consensus.html .
ReplyDeleteThe matrix elements are described on the "Bias" spreadsheet in columns P through V.
"I am unaware of any previous peer-reviewed paper that Ack'd a >a Lovecraft entity, but there must be a first for everything."
ReplyDeleteSo maybe it wasn't gremlins in this paper. Maybe it was Shoggoths.
For those gentlepeople of leisure who don't want to muck around in excel, here is Tol's 'S' matrix:
ReplyDeletehttp://textuploader.com/0gj6
Elements 1,1 and 7,7 should be zero (otherwise it's not an error), so to make the columns sum to unity each of the remaining values are scaled by a constant to normalise.
During reconciliation, the consensus went from 96.73% to 97.06%.
The Cook response method predicts that reconciliation changes consensus from 96.73% to 97.06% (versus observed 97.06%).
The Tol comment method predicts that it changes from 96.73% to 87.44% (versus observed 97.06%). Working backwards, Tol's correction requires that the pre-reconciliation consensus should be >100% (as pointed out by Anonymous).
I'm sure that Professor Tol will realise and retract his paper or, at the very least, submit a quick correction.
Thanks to Neal and Mark R
ReplyDeleteMarkR,
ReplyDeleteI'm sure that Professor Tol will realise and retract his paper or, at the very least, submit a quick correction.
That's very funny, but you really should put a smiley face or some kind of emoticon at the end of such statements, or else people might think you're being serious.
ATTP: I have much more hope than you! Everyone makes mistakes and it's embarrassing when it happens, but it's much more embarrassing in academia to deny them forever rather than slip out a quick correction.
ReplyDeleteThe climate debate is loud, tribal and aggressive sometimes and I imagine Professor Tol could be feeling defensive at the moment. Snide comments are fun and we all give into the temptation, but they could trigger stronger emotive responses that get in the way of fixing the damn problems with your maths.
Tol has shown that he can overcome tribal feelings by his repeated statements that there really is a consensus, despite his 'sympathisers' (GWPF, US Republicans) not wanting to hear anything about it. So I stick by that statement and don't think it needs an emoticon at the end. :)
MarkR
MarkR,
ReplyDeleteYou are, of course, quite right and my comment was indeed snarky. At the moment, however, I'm finding it hard not to make such comments, but I would happily apologies and feel duly embarrassed if my comments were shown to be unfair. I shall do my best to follow your much better example - although I may not always succeed :-)
> That's very funny, but you really should put a smiley face or some kind of emoticon at the end of such statements, or else people might think you're being serious.
ReplyDeleteI would advise against the smiley face or some kind of emoticon for the same reason.
I would also advise against emoticons, for other reasons, or no reason at all.
The climate debate is loud, tribal and aggressive
ReplyDeleteDude, can you point out this debate to this informed and numerate individual, because, well, I just don't see it. What is pathetic is claiming there is still a debate.
Go ahead, make my day, debate all you want, and I'll show you some snark that will shut you down in a second. That you are even debating Tol's masturbation is pretty pathetic as well. Methinks you need a much thicker skin.
I imagine Professor Tol could be feeling defensive at the moment
ReplyDeletesure, but maybe if he wasn't so offensive before on several occasions he wouldn't have painted such a huge target over himself.
You can see that as a kind of karmic instant retribution.
> What is pathetic is claiming there is still a debate.
ReplyDeleteOf course there are debates, perhaps even more than ever. Debates are everywhere, in the climate blogosphere and elsewhere. Even if these debates are not justified on scientific ground, they still exist.
But debates over what? That is the question. What debates should we have instead? That is another question.
Maybe it's just a vocabulary thing.
But debates over what? That is the question. What debates should we have instead? That is another question.
ReplyDeleteOh really. You still have to ask yourself that question. I'm here to help, I'm from the private sector.
• Epidemic Diseases
• Political Corruption
• Human Overpopulation
• Irrational Religious Beliefs
• Global Financial Instabilities
• Tribalism, Gangsterism, Fascism
• Conventional and Nuclear Weapons – War
• Industrial Contamination of the Holocene Biosphere – Air, Land and Water
• Carbon Dioxide and Emissions Induced Global Warming and Climate Change
• Agricultural Droughts and Blight, Insect, Weather and Flood Related Agricultural Losses
• Glaciers, Arctic Sea Ice and Antarctic Ice Sheet Losses and Sea Level Rise
• Ocean, Sea and Lake Pollution and Ocean Chemistry Changes
• Deforestation, Soil Loss and Adverse Land Use Changes
• Groundwater Contamination and Aquifer Loss
• Biodiversity and Species Loss – Extinction
• Terrestrial Hazards – Floods, Tornadoes, Hurricanes, Volcanoes, Earthquakes and Tsunamis
• Extraterrestrial Hazards – Orbital Debris, Asteroid Impacts, Solar Flares and Cosmic Radiation
I've published this BTW, so if there is anything you'd like to add to that I'd love to hear about it. The only obvious one I forgot to mention is : debt.
Richard Tol, not so much. Most of these takedowns to unnoticed and this is a very obscure academic takedown. He will survive, lol. It's more the fault of the press for pimping this kind of stuff.
> I've published this BTW,
ReplyDeleteCitation needed.
Do you realize, dear Thomas, that your argument is of the same form as Bjorn's?
Incidentally, this is the same as Judy's:
I find the topic of abrupt climate change to be more more interesting scientifically than AGW, and potentially of greater societal significance (although one key question is the potential role of AGW in triggering an abrupt event). IMO much more scientific effort should be focused on the topic of abrupt climate change.
http://judithcurry.com/2014/06/04/explaining-abrupt-climate-change
Commenters from the private sector should beware equivocation over the word "debate".
Citation needed.
ReplyDeletehttp://rabett.blogspot.com/2014/06/in-spirit-of-deep-climate-anonymous-as.html
lol. Feel free to cite that.
Do you realize, dear Thomas, that your argument is of the same form as Bjorn's?
Do you realize that using dear makes you sound like an ass?
I don't recall making any arguments here on this thread and I certainly won't be 'doing Judy' or Bjorn. Good luck with that.
I would argue you have serious problems you seem unwilling to address let alone argue or debate.
"Most of these takedowns go unnoticed and this is a very obscure academic takedown"
ReplyDeleteRabbett Run in a nutshell: "Some Tol is wrong on the internets"
> http://rabett.blogspot.com/2014/06/in-spirit-of-deep-climate-anonymous-as.html
ReplyDeleteWait. Are you suggesting you wrote that blog post, dear Thomas?
***
> I don't recall making any arguments here on this thread [...]
Well, for starters, there was "What is pathetic is claiming there is still a debate." Then there was "That you are even debating Tol's masturbation is pretty pathetic as well." This claim was followed by a list starting with "Epidemic Diseases," which was a response to what we should debate instead of Tol.
In other words, the argument goes like this, paraphrasing:
(1) Debating Tol is stupid;
(2) There are more important things to debate;
(3) Here are some more important things;
(4) There is no debate anyway;
(5) We ought not debate Tol.
This argument is of the form "we should rather do B instead of A (i.e. what we're doing right now) because B is less important than A.
Compare and contrast:
We'd rather fight poverty instead of debating over mitigation.
We'd rather study emergent properties of natural variability than debating over AGW.
We'd rather debate just about any important societal topic than to debate Tol.
See?
Not complicated. Snark all you want, dear Thomas. I'm all bunny ears.
Hope this helps,
w
Erratum:
ReplyDelete> This argument is of the form "we should rather do B instead of A (i.e. what we're doing right now) because B is more important than A.
***
Oh, and since I'm here, dear Thomas, let's add this. If I'm correct about what I think you're arguing, you're not even supposed to be here.
In case you reply "I'm here for the snark," then my next move would be: why, of course, dear Thomas, you're at Eli's. Welcome to the club.
The first rule of ClimateClub (tm) is: you need to talk about ClimateClub (tm).
"Willard", in a nutshell:
ReplyDelete"Erratum:
This argument is of the form "we should rather do B instead of A (i.e. what we're doing right now) because B is [replace "less" with "more"] important than A."
Details, details, from the fellow who has taken his pseudonym (and until recently, avatar) from the great logician and language expert Willard Quine.
Why do we care? There are a very few smart guys out there that get things correct, and a lot of dumb guys out there get stuff wrong, frequently.
ReplyDeleteI would rather have the considered thoughts of 3 really smart guys, than the ramblings of 97 not so smart guys. Science does not work by vote. The smartest guy in the room may not have all the facts, but if you have the 3 smartest guys come to a consensus, you likely have the right answer. If you wait for a consensus of 97%, then you can be sure that you have the right answer, but it is too late - your prey has moved on or the monsoon has passed and it is too late to plant rice. The species has traded certainty for time to act.
Mother Nature is running the program and she is only rewards species that are able to tolerate very high levels of uncertainty. Mother Nature punishes species that demand certainty.
No, the monsoon has stalled: http://robertscribbler.wordpress.com/2014/06/05/monsoon-at-dead-halt-chances-for-summer-el-nino-increase-to-70-as-india-swelters/
DeleteAnd Eli would rather have the agreement of 97 really smart bunnies. You have a problem with that?
ReplyDeleteAaron,
ReplyDeleteTrue, scientific truth is not a matter of polling--but then scientific consensus is not either. If you want a better metric for measuring scientific consensus, look at publications or better yet, citations. At some point and idea, technique or theory becomes so indispensable to an area of study that you cease to publish meaningful work if you don't embrace it. Then you have consensus.
Wait. Are you suggesting you wrote that blog post, dear Thomas?
ReplyDeleteNo, it's because I can't index blogspot comments without looking a the HTML source for comment indexing tags if they exist. And guess what, I don't feel like wasting my time on you more than I already have. It is ... unproductive.
If you think you are anybody's peer around here, review it. Can't find it? Look for it. If not, I posted a whole set of problems for your PhD thesis. Take your pick and get back to me. I will be quizzing you.
Please forgive me for not bothering with the rest of your spewage. It's ... too ... lame. I give anything you say as much credence as I do Richard To, Judy or Roger Pielke (Jr or Sr.). That kind of nonsense might work for corrupt and ignorant congressmen and woman, but hey, I've got just enough science and math to see through any bullshit you or any of your other cronies throw my way. So good luck with the time wasting. You are a waste of time.
Try clicking on the timestamp of the comment you wish to cite, Thomas. The URL should react accordingly.
ReplyDeleteThank you for your concerns,
w
Try clicking on the timestamp of the comment you wish to cite
ReplyDeleteBecause I'm using the popup window and I couldn't care less about your cites and citations. That's soooo ... Tolish.
Got any other problem avoiding contributions to make here?
> Got any other problem avoiding contributions to make here?
ReplyDeleteIndeed I do, dear Thomas, but why would you care?
Here goes. You assume that Eli's is a place to solve first, second, and third world problems. That this thread about Tol is exactly where we should be talking about that. That "debating" Tol's ideas somehow "avoids" to contribute on these problems.
These concerns follows the same kind of logic as Bjorn's argument against mitigation.
Please own it.
***
Here would be a comment to help you solve first, second, third problems: please curb your enthusiasm regarding Eli's. This is not the place where this will happen. Not the way you envision it. You can do better than commenting here.
Here would be another one: to think that everyone should always try to solve first, second, and third problems the way you envision it is just silly. There are lots of people who tackle these problems already. There are many ways to solve these problems, and paying due diligence to what Tol writes may even have an impact, however indirect or remote it might be.
Thank you again for yours concerns,
w
Your inability to recognize important problems is quite ordinary. You are ... unremarkable.
ReplyDeleteThe dear thing again. That's so ... dignified. Distinguished. You'll do fine with the status quo, they love that kind of shit. It's bankable.
Lindzen, Curry and Christy walk into a bar.
ReplyDelete"Ouch."
Three smart people!
More from the fellow who has taken his pseudonym from the great logician and language expert Willard Quine:
ReplyDeleteThat "debating" Tol's ideas somehow "avoids" to contribute [sic] on these problems.
These concerns follows [sic] the same kind of logic...
Thank you again for yours [sic] concerns,
Thank you for yours concerns.
Tol is a waste of time.
ReplyDeleteTol's main argument has been that below about 2C, the impacts of warming would be positive. Even Tol has agreed that above that,impacts will most likely go negative.
But the likelihood that humans will keep warming below 2C is slim to none at this late date.
Anyone who believes otherwise (that the measures by the US and other countries to address climate change are even close to what is required ) has not been paying attention.
> Tol's main argument has been that below about 2C, the impacts of warming would be positive.
ReplyDeleteHere's the short summary of the Copenhagen Consensus:
Global warming is a major global challenge, which could increase food and water insecurity for many millions of people in the developing world and cause trillions of dollars worth of damage to the world economy. An affordable, cost-effective package of measures would not only start the process of reducing greenhouse gas emissions, but would also enable other Copenhagen Consensus challenges to be properly addressed.
Many people believe this is the world’s biggest problem and solving it should be the top priority. This research shows that even the most extreme mitigation efforts (reducing carbon emissions through taxes) would not guarantee the worst damage of warming. This is in line with recent IPCC findings – but the paper goes further and fills in the gaps about the mixture of spending that would provide value for money. One solution it proposes is mixing adaptation and mitigation approaches with greater research and development into greener technology resulting in high returns.
http://www.copenhagenconsensus.com/expert/richard-s-j-tol
Richard may very well be a waste of time to bunnies, but dismissing him from one's armchair won't prevent him to waste more time in the world.
Eli invests time so that otters won't have to:
http://discourseontheotter.tumblr.com/post/84323473615/p-david-marshall
The least bunnies can do is to stop whining about that choice.
"I've published this BTW, so if there is anything you'd like to add to that I'd love to hear about it. The only obvious one I forgot to mention is : debt."
ReplyDeleteI have no desire to be involved in the discussion above, but another one for the list is antibiotic use, both medically and industrially.
That's going to be a big oopsie one day i the not too distant future, and there's been inadequate movement on the matter to date.
Antibiotic Resistance.
ReplyDeleteThank you. If anyone else has something to add to the list I would love to hear it, I submitted the essay early and so I know there were things that I missed on it.
Debt was the biggie, but Antibiotic resistance is right up there as well. Again, my sincere thanks.
It was a NASA RFI BTW.
Acronyms!
Another item I left off the list purposely was plain old weather. This of course led to homes and automobiles and thus the problems, which of course are related to unprotected sex, overpopulation and is intimately related to irrational religious beliefs. lol. Certainly one of the best things to ever happen to the human race is latex. It's just too bad two of the major irrational religions (Catholicism and Islam) aren't wise to this.
ReplyDeleteAnyone interested in a list of things that need fixing could do worse than look out a copy of 'The Little Earth Book' by James Bruges.
ReplyDeleteFrom the top of my hat, I'd add to that list malaria (which may have killed half of the human species to date), eternal corporations, inheritance (in law; perhaps in OOP too, but I'm tempted to add OOP in general), and populist rhetoric.
ReplyDeleteI'm not looking for anything that specific, and it falls under the phrase 'epidemic diseases'. Antibiotic resistance is a category by itself since it concerns bacterial response (although not viral response). Nowadays one can hardly claim ignorance unless living wild in the jungles.
ReplyDeleteWillard claims "Eli invests time so that otters won't have to"
ReplyDeleteYou mean so Bob Ward, John Cook, Andrew Gelman, NeilKing, MarkR,"Anonymous" and others won't have to?
How's that going?
> You mean so Bob Ward, John Cook, Andrew Gelman, NeilKing, MarkR,"Anonymous" and others won't have to?
ReplyDeleteOf course not, as if Eli had to this alone. I mean all those who, like Thomas, who would try to solve all the most important problems in the world.
Speaking of which, here is Daniel Botkin's list of nine environmental issues that need our attention now:
Energy
Fresh water
Phosphorus and other essential minerals
Habitat destruction
Invasive-species control
Endangered species
Pollution by directly toxic substances
Fisheries
Forests
http://www.climatedepot.com/2014/05/29/scientist-dr-daniel-botkin-tells-congress-why-he-reversed-his-belief-in-global-warming-to-become-a-skeptic-there-are-several-lines-of-evidence-suggesting-that-it-agw-is-a-weaker-case-today-not/
Botkin's testimony contains a very similar argument than Thomas'. It does not mention that Richard is a waste of time.
But sure, one can certainly argue that Richard, who was a lead author to the WGII, who testified to Congress not long ago, who is ranked among the Top 100 economists in the world by IDEAS/RePEc1, who is among the 25 most cited climate researchers according to Google Scholar, and who is the most cited scholar in the Stern Report, is a waste of time.
"Of course not"
ReplyDeleteAnyone (except you?) can see that "Eli invests time so that otters won't have to:"
followed immediately after and was clearly a reference to "Richard may very well be a waste of time to bunnies, but dismissing him from one's armchair won't prevent him to waste more time in the world."
You should really change your pseudonym because you are making a mockery of logic and language.
Willard Quine must be rolling in his grave.
Your list of 'problems' is so vague as to be useless. It's ... tolish. If that's all you can up with then you are indeed in very serious trouble. Good luck with that.
ReplyDeleteHere's a couple of solutions to wrap your naive and illucid mind around - resonant ZT enhancement and topological degree of freedom reduction. You are just so special. Tol is just too funny, a caricature of all that is wrong with academia, joining the ranks of Judy, Bjorn and the Rogers, and of course, then there is you, the most eminent pseudoscientist. Show me your license and your CV!
> Anyone (except you?) can see that "Eli invests time so that otters won't have to:"
ReplyDeleteAnyone can see that "otters" does not imply all otters, Anonymous. It did not imply Thomas, for instance, as I already told you and which you failed to acknowledge. Grammar suits you better than quantifiers.
Grammar suits you better than quantifiers.
ReplyDeleteBut not by very much.
> Your list of 'problems' is so vague as to be useless [...]
ReplyDeleteYou mean Daniel Botkin's, Thomas? I thought my own was too specific.
***
Here's a list of list of issues:
http://www.globalissues.org/issue
Would that be useful to you?
Perhaps not, as you may not need any list at all:
> The problem I see with transitioning to a completely carbon free or carbon neutral energy economy is that, as wonderful as that may be, it is only one of a myriad of horrible problems that are self evident at the extinction level. The major problem is the use of energy, Thus I have come to the conclusion that complete evacuation of the human species from the planet is the only solution that will yield anything even vaguely resembling an early Holocene era planetary ecological and biological diversity. And indeed, I have worked out the necessary global technological and engineering actions required to produce that desired result. Your opinion on the subject of future evolution of humanity may diverge from my own, of course, but it is a subject I have studied deeply.
http://www.realclimate.org/index.php/archives/2014/01/unforced-variations-jan-2014/comment-page-6/#comment-446571
Please tell bunnies more about your deep studies.
q: How strong does the DK effect have to be for someone to adopt "Willard" (aka Willard Quine) as their nom de plume?
ReplyDeletea:pretty damned strong.
Here's a list of list of issues:
ReplyDeleteSure if you are an idiotic policy wonk. My issues are solar irradiance, composted maple leaves and where my food comes from. After my air and water, of course.
Please tell bunnies more about your deep studies.
Why bother when it's free for anybody to read. The bunnies know all this stuff. Why are you so concerned about the bunnies?
They eat grass, they don't even eat maple seeds. Or maybe they do but I've never seen them eat any.
You get your food from coal, oil and gas. Very bad for you. Very bad for the land, water and air.
Dude, that's all you need to know. Solar irradiance. Atoms and molecules. Get it? Do I have to spell it out for you in SI units, or what? It's not my problem third world people live in third world slums and third world conditions, I've got my own hood and concrete jungle to deal with. And farmer joe and his boys, lol. You do it your way, and I do it my way. I just happen to prefer it the abundant solar irradiance, water and deciduous leaf soil way. Mostly because it works.
Bitch.
Did you get that? It comes with a cite, just click on the timestamp.
> My issues are solar irradiance, composted maple leaves and where my food comes from. After my air and water, of course.
ReplyDeleteWouldn't such a list be "too vague as to be useless"? Perhaps we need to distinguish between an issue and a problem. Another vague list:
- Economic Collapse
- Peak Oil
- Global Water Crisis
- Species Extinction
- Rapid Climate Change
http://www.arlingtoninstitute.org/wbp
Nope, Richard ain't there.
Bad, bad bunny.
Global Water Crisis
ReplyDeleteNot that's funny. You live on a terrestrial planet with an active hydrogeological cycle and ice caps.
If you have a problem with water, it's your own damn fault. What you have is a problem with the price of water, and the price of transporting it. There is no crisis. What you are doing is screwing up the weather, as an effect of screwing up the climate, and you are utterly destroying your aquifers. Any crisis is of your own making.
Read the list again. The natural problems are way at the bottom. And I added a postdated addition to the top of the list (antibiotic resistance), and put debt at the end of the global financial instabilities entry.
If anybody has anything else to add, I will be listening in.
The following update has been left at the page of Richard Tol's dismissal of A.Scientist's brief argument against the validity of Tol's calculation of reassignments:
ReplyDelete================================
Neal J. King7 June 2014 05:15
[Below, I denote the set of abstract counts per category published in Cook et al. 2013 as '(C13)'. I refer to the set of abstract counts, prior to the reconciliation procedure, as the 'ur_count'; this is:
(128, 1844, 5820, 15,940, 108, 30, 18)
Richard:
- It is true that AS got the wrong matrix S; however his approach of defining the matrix T(eps) = (1 - eps)*I + (eps)*S seems to be an accurate description of your calculation. Some of us have taken your invitation to use the correct matrix S', and have substantially reproduced his conclusions.
- To verify the proper S: According to our calculations, your projected correction of C13 is T(6.7)*(C13)
= (92.76, 961.65, 2970.25, 7540.56, 329.10, 37.77, 11.92).
(Unfortunately, there will be a slight difference from your calculated numbers: In your spreadsheet calculation, we believe that bias!B20-B26 should refer to the normalized column O, not the raw histogram in column N.)
- Based on our version of matrix S, we calculate the inverse of T(11.8) = T_inv(11.8). In accordance with AS's argument, if it makes sense to apply T(6.7) to (C13) to project the corrected counts, it should also make sense to apply T(11.8) to the ur_count to obtain (C13):
T(11.8)*(ur_count) = (C13).
But that implies:
(ur_count) = T_inv(11.8)*(C13)
= (10.7, 854.3, 2745.9, 8879.1, -559.5, 7.0, 6.5).
- Comparison with the (ur_count) stated above shows that this is wrong; but with a negative number of papers in category 5, it is clearly absurd. This means that T(11.8) was not the process applied to the (ur_count); but if T(11.8) is not applicable to the ur_count to obtain (C13), why should T(6.7), which is what you have used, be applicable to (C13) to produce the projected correction? For T(11.8) is constructed on the same lines as T(6.7): they only differ in the values of correction strength, which you have estimated as 11.8% and 6.7%.
- Or in other words: If for some reason you don't believe the S matrix approach applies to the ur_count-to-(C13) transition, please explain why you apply it to the (C13)-to-correction transition. Particularly since the ur_count-to-(C13) transition is the only one that is possessed of actual data, as opposed to speculation.
- Or yet in other words: How do you justify a model that concludes that a 55/45 split is an appropriate representation of an actual 2/98 split - and for a category that comprises 2/3 of all articles?
The linear algebra, matrix manipulation and statistical transformations aside, the take away I'm getting from this lesson, besides the fundamental issues of peer review, is that one must or should choose your problems wisely (or in the cases under discussion, urgently) and just do the best job you can on them, and take your knocks when they come. Peer review via crowd sourcing and enthusiast forums works, I've seen this just recently with that satellite Cowing and Wingo have been working on. This is most definitely a new paradigm.
ReplyDeleteTol's problem isn't urgent, but correcting it was, since it was rather late in the process and had been corrupted by the media.
Neal's comment is very interesting, but he still assumes an error rate from ur_count to C13 of 11.8%. Tol in fact assumes a constant 18.5% error count per rating process. With an initial disagreement rate of 33% between first and second ratings, the error rate from ur_count to C13 is 18.5% of 33%, or 6.1% with the remaining 0.57% of errors being undetected errors from initial rating when both initial raters provide the same erroneous value.
ReplyDeleteI am asking for a recalculation with the correct values. Before doing so, however, I will note that Tol's calculation of the base error rate is incorrect. He assumes equal probabilities for all rating values in the calculation, so that the 2 times the individual error rate minus (1/6 times the individual error rate squared) equals the disagreement rate after initial ratings. However, not all ratings have equal probability. My best estimate is that there is a 52.6% chance that two erroneous ratings will coincide, from which it follows the individual error rate falls to around 17.3%. That increases the rate of initial errors escaping attention to 1.57% of all abstracts, the error rate during reconciliation to 5.71% of all abstracts, and the total error rate to 7.28%. This assumes the average error distribution. Using the correct error distribution by rating may adjust these figures further. In any event, would anybody care to repeat anonymous scientist's analysis with these corrected values.
Tom, do not fall into Tol's (and Lucia, etc.) trap. The disagreements on first rating ARE NOT ERRORS. As SkS points out the disagreements are almost all the result of imposing an ordinal value on the continuum of positions.
ReplyDeleteConsider what the disagreement rate would be if the value of a set of papers was 3.5 +/- 0.1? 50%.
Eli, we can call the "errors" disagreements and the "undetected errors" accidental agreements if you prefer. However, it makes no difference to the analysis or the final consensus rate found at the end of that analysis. Therefore I do not think the terminological issue is substantive.
ReplyDeleteMore importantly, SkS points out the problem is treating ordinals as a continuum rather than the reverse (See error 14 of the 24 errors). The endorsement classification is achieved by responding to a set of binary oppositions. Does the abstract endorse the consensus, or not? If so, does it endorse it explicitly or implicitly? If explicitly, does it quantify the contribution of anthropogenic factors to recent warming? If it does not endorse, does it reject, or is it neutral? And so on. Partial values have no meaning in this context. An endorsement rating of 3.5, is nonsense. It literally has no meaning in the classification scheme.
What is true is that the determination of the answer to the binary oppositions is often difficult. In such circumstances, many people will get the answer wrong, and indeed whether the same person gets the answer right or wrong may vary based on arbitrary factors (time of day, tiredness, endorsement rating of other recent ratings etc). Further, what determines that the answer is right or wrong may be somewhat arbitrary. It may be resolved, for example, how it would be assessed by the majority of climate scientists, or by the authors of the paper, etc. That means we may not have a final determination as to which answer it genuinely right or wrong, but by the logic of the classification system, it is presumed that a unique endorsement rating is correct for each abstract, and that all other endorsement ratings are incorrect for that abstract. (This is not the same issue as to whether the error of rating a level 1 paper as a level 2 paper is as consequential as rating a level 3 paper as a level 4, or worse a level 5 paper. They are not, as SkS again, correctly, points out.)
Finally, regardless of the truth of the matter, the "errors" discussed in Tol's faulty analysis are internal errors. They are estimate disagreements with the limit of classification from the actual raters as the number of agreements on classification becomes to finalize a rating arbitrarily large. For most initial ratings, the raters got that right, but there were some errors (or disagreements if you prefer), and the approximate rate of those disagreements, and their effect can be estimated. Just not using Tol's S matrix, which rates more as a blunder than as an error.
Tom, in this as much else it is important to take the Cheshire Cat position wrt word choice. Error is simply wrong and to an outsider indicates that there were problems with the methodology.
ReplyDeleteWhat the reconciliation process did was try and find the best, as in most appropriate rating on the ordinal scale. However, as Eli's example shows, there often would be no unique choice
"Eli invests time so that otters won't have to"
ReplyDeleteThe math was all done by otters and Eli merely posted their results.
Weasels? No damn otters here.
ReplyDeleteWeasels do math?
ReplyDeleteWho knew?
I thought they just ranted about anti-vaxers.
...and rowed.
> The math was all done by otters and Eli merely posted their results.
ReplyDeleteIndeed, and the other Anonymous should simply have had to post his results on ArXiV, instead of bothering Eli with it. Blogging is so useless. We should leave everything to interpreters of interpretations.
***
Some more problems:
> Sustainable energy, Internet, and making life multi-planetary.
http://www.inc.com/elon-musk/elon-musk-tesla-spacex-solve-world-problems.html
He also mentions AI and cryogenics, but he admits there are still question marks on these two.
Bunnies should never underestimate the power of a good shower.
"Bunnies should never underestimate the power of a good shower."
ReplyDeleteI'm not sure what Musk was doing in the shower, but at least some of it was an excursion into fantasy.
I've said that I've said it before and that I'll say it again, so I'll say it again... Interplanetary colonisalism in any population-effective form is a pipe-dream of the highest order, and we'll have Quiddich on Firebolts as an Olympic sport before we see interplanetary settlement beyond a few shivering, starving and suffocating suicides on Mars.
Thermodynamics says "no", and no business plan has addressed the inescapable energy budget, let alone the ecosystem transplant/compatibility issues involved or the ethics of colonising habitable planets if they happen to already support life. And as evidence of the infeasibility of interplanetary (and especially interstellar) colonialism, I tender the fact that our planet is still absent any little green aliens.
We have one blue marble in the whole universe. It's our alpha and omega, and anyone who imagines otherwise is simply getting in the way of making sure that omega is as far away as possible.
Charlie Stross does the math here:
ReplyDeletehttp://www.antipope.org/charlie/blog-static/2007/06/the_high_frontier_redux.html
He quotes Bruce Sterling:
"I'll believe in people settling Mars at about the same time I see people settling the Gobi Desert. The Gobi Desert is about a thousand times as hospitable as Mars and five hundred times cheaper and easier to reach. Nobody ever writes "Gobi Desert Opera" because, well, it's just kind of plonkingly obvious that there's no good reason to go there and live. It's ugly, it's inhospitable and there's no way to make it pay. Mars is just the same, really. We just romanticize it because it's so hard to reach."
Interplanetary colonisalism in any population-effective form is a pipe-dream of the highest order, and we'll have Quiddich on Firebolts as an Olympic sport before we see interplanetary settlement beyond a few shivering, starving and suffocating suicides on Mars.
ReplyDeleteI'm pretty confident I can file that with numerous other statements such as 'spaceflight is utter drivel', 'if man were meant to fly' and 'who needs more than 640K'. The only thing that will stop this is the collapse of civilization. That is the alternative that you propose.
Good luck with that, fool. Please, by all means, feel free to make a fool of yourself here. Everybody knows superconductivity above 22.5 K is impossible. Now tell me, what is the average yearly number of airline passengers. Quickly now, billions of dollars of revenue are at stake.
Charlie Stross does the math here:
ReplyDeleteTommy does his own math. Elon Musk is a multi-billionaire. Charlie Stoss is not. QED.
Tommy asks, who is Charlie Stoss?
Tommy neatly sidesteps the question. So, we ask once again, why go to Mars, what is the point, and who will benefit from it?
ReplyDeleteI didn't say anything about going to Mars, that was presumptions from you idiots. Let me explain it to you, Charlie Stross is a science fiction author. Is that where you get your freakin science? That explains a lot about the state of the biosphere. Elon Musk built his own rocket engine and rocket. Get over it. You can't stop it now.
ReplyDeleteYour logic is laughably tragic. Normally I would not even bother to debate someone so naive, someone so behind the curve both technologically and scientifically. It's just a waste of time for me. All I can say is get out there and educate yourself. Start here. I regularly blog titles that I feel are relevant to your problems. I regularly publish essays on these subjects as well. Even better, get yourself an education, and do try not to quantify that with degrees.
Speaking of shit Elon says:
ReplyDelete> You know, life has to be about more than solving problems. If all that life is about is solving problems then why bother getting up in the morning?
http://shitelonsays.com/transcript/npc-luncheon-with-elon-musk-2011-09-29
Right, so invest all those millions and billions in mutual funds, so you don't have to worry about all the environmental and ecological damage your investments have wrought. It's not your fault!
ReplyDeleteIt's death by proxy. I don't mind discussing these things, but not with a bunch of dismissivists, lol.
I point at this:
ReplyDelete> I don't mind discussing these things, but not with a bunch of dismissivists, lol.
And I point at Thomas' overall contribution in this thread.
That is all.
I find it very easy to dismiss something long after it has been demonstrated to be both irrelevant and wrong. Richard Tol, in a nutshell. Maybe his next paper will be an improvement, lol (as in relevant, not fraught with errors and not fundamentally wrong). That part I am not dismissive of in the least.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteI'm guessing at least Musk and Stross both understand that the moon is a long way and uphill from low earth orbit.
ReplyDeleteMusk's genius is manifest not so much in replicating what others have done for a half-century, nor in profiting from the publicly funded R&D involved in that, but in gaining the admiration for the same from the people who wrote the checks.
Right, but you could be living on a super Earth with no moon, so count your blessings you happen to be on a relatively small terrestrial planet barely able to support plate tectonics. And uphill is quantified we have numeric and computational symplectic integrators now that can easily handle anything you throw at it. And now get this, the technology we need to do this is pretty much the same as what we need to live sustainably on the Earth. Cryogenic production, storage and distribution, fuel cells, cooling, heat exchangers, solar electric, batteries, propulsion, life support, you name it, we need it cheaper and better. I guess you just missed that I suppose. Besides, you don't seem to understand. You have no choice anymore. It's a done deal. So get on with it. Hint : SLS and Orion in their current form are not 'it'. It seems to me you have a real NASA problem on your hands as well.
ReplyDeleteThomas does not always dismiss, but when he does, it's because it's irrelevant and wrong.
ReplyDeleteThe most important bunny at Eli's is thus a true Scot.
***
More important problems:
1. War, terrorism, organized crime
2. Violation of human rights, poverty, famine, water
3. Destruction of nature, climate and the planet as a whole; deforestation, extinction of species, loss of biodiversity
http://www.independencyproject.org/en/global-solutions-to-worlds-major-problems.html
It even comes with solutions:
1. Nature First!
2. Full eradication of Tax Havens (bank secrecy, offshore "banking"
3. 100% VAT on TV commercials
4. From Corporations to Cooperatives
5. Introduce MAXIMUM wage
6. Educate and raise for society, not for profit.
7. Use hemp as sustainable alternative material on many levels.
Bunnies love hemp, although some may prefer Soylent Green.
Thomas does not always dismiss, but when he does, it's because it's irrelevant and wrong.
ReplyDeleteYou missed 'after it was demonstrated' to be irrelevant and wrong. This is where the smell test comes in. That saves a lot of time. Generally I give crackpots more of my time than people like you. The end result is generally the same,, but you never know, they might be giants.
I don't think you have a very firm grasp of what constitutes credible solutions to biological problems on a terrestrial planet.
> You missed 'after it was demonstrated' [...]
ReplyDeleteI said "it is irrelevant and wrong," which it what a demonstration usually implies. I could have said "after Thomas declares it irrelevant and wrong," to underline the question begged. In either case, Thomas dismissed the topic of this post not because it was wrong, but because he considers it irrelevant.
***
Another list, this time they're called challenges:
1. How can sustainable development be achieved for all while addressing global climate change?
2. How can everyone have sufficient clean water without conflict?
3. How can population growth and resources be brought into balance?
4. How can genuine democracy emerge from authoritarian regimes?
5. How can policymaking be made more sensitive to global long-term perspectives?
6. How can the global convergence of information and communications technologies work for everyone?
7. How can ethical market economies be encouraged to help reduce the gap between rich and poor?
8. How can the threat of new and reemerging diseases and immune micro-organisms be reduced?
9. How can the capacity to decide be improved as the nature of work and institutions change?
10. How can shared values and new security strategies reduce ethnic conflicts, terrorism, and the use of weapons of mass destruction?
11. How can the changing status of women help improve the human condition?
12. How can transnational organized crime networks be stopped from becoming more powerful and sophisticated global enterprises?
13. How can growing energy demands be met safely and efficiently?
14. How can scientific and technological breakthroughs be accelerated to improve the human condition?
15. How can ethical considerations become more routinely incorporated into global decisions?
http://www.millennium-project.org/millennium/challeng.html/
I think you should worry more about backing up away from the agricultural cliff you've created for yourselves, and into something a little more resistant to all those known agricultural disruption mechanisms and probabilities. Nothing makes humans more ornery and desperate than outright hunger.
ReplyDeleteTyler Cowen seems to agree, while finding not unrelated headlines:
ReplyDelete> The bottom line is this: right now agriculture is a laggard sector — in part due to state interventions — and this is not totally unrelated to recent headlines about unrest in the Middle East.
http://marginalrevolution.com/marginalrevolution/2012/09/world-hunger-the-problem-left-behind.html
Tom Curtis:
ReplyDeleteThe way I calculate it, the probability of a disagreement between two evaluations when the likelihood of "error" is x is: D = 2x(1-x) + 6*(x/6)^2
= 2x(1-x) + (x^2)/6
This breaks down into:
a) the likelihood that one rater gets right while the other gets it wrong; plus
b) the likelihood that both get it wrong - but agree on the answer (1 out of 6). Probabilities for erring in the direction of each category are assumed equal.
c) Tol set this expression, which I will denote as D, to be:
D = 0.33. Solving this with the quadratic formula, one infers that:
x = 18.5%
d) I believe you have an erroneous minus sign in the equation for D, leading you to the anti-intuitive conclusion that a greater likelihood of two raters agreeing in error reduces the remaining-error rate.
e) I need to think a bit more about how unequal probabilities affect things. My sense is that it gets ugly.
Eli (8 June, 1:42 am), just over 9% of disagreements between initial ratings and final ratings were larger than 1 point. These, at least, must be considered genuine errors by either the initial rater or the adjudicator. Further, given that large percentage, a significant percentage of those ratings disagreeing by just one point must be considered to be disagreements over papers that were near central values. That means the number of disagreements to which your sorites style argument applies is unknown. In consequence it is not possible to divide discrepancies into those which are genuine errors (even by your standard) and those which are mere discrepancies due to indeterminancy of classification. Further, that sub-analysis is irrelevant except at most for rhetorical points. Consequently it is better to have a single name for both.
ReplyDeleteFurther, as regards the rhetoric, among people interested in the arcane enough to even follow this debate, those with an open mind will happily accept that humans err, and that therefore there are a small number of errors in Cook et al. They will even accept, if they follow the arguments, that classification is difficult so that there will be variation in classification at the margins, and that considering that Cook et al achieved a surprisingly low error rate. Before they get to that point, however, they may well be put of by people insisting that what they can clearly see are errors are not.
Neal, if x is an error rate of 18.5%, than 2x = 1.63, and hence is not a probability. Adding a positive number only makes the problem worse, so your formula must be wrong.
ReplyDeleteIt is best to start by considering the ratings to be binary choices so that if it is not correct, then it is an error. Assume, further that all errors are independent so that no two raters make an error on the same abstract. Then, as any error creates a discrepancy, there will be 2x discrepancies where x is the error rate. Errors are not independent, however, so we must subtract the number of occasions when both raters make an error, ie, x^2, so the base formula is:
D = 2x - x^2
This formula can be obtained algebraicly from the rate of agreements (=1-D = (1-x)^2).
The situation is not binary, however, so two ratings both being incorrect do not necessarily agree with each other. With six equally possible errors, that gives you an error rate of 2x - (x^2)/6.
I had another look, and the discrepancy is because Tol uses yet another formula. Using my formula, and equal probabilities the error rate estimate is approx 16.7%
Tom C writes: "...those with an open mind will happily accept that humans err, and that therefore there are a small number of errors in Cook et al. They will even accept, if they follow the arguments, that classification is difficult so that there will be variation in classification at the margins, and that considering that Cook et al achieved a surprisingly low error rate....
ReplyDeleteTom, well said.
I think the rhetorical vocabulary is what causes many of the points of contention. Since we are dealing not just with an ordinal scale, but a subjective scale, the disagreement/variation between raters/adjudicators can be either viewed as errors or simply as data.
The implicit assumption is that discrepancies > 1 are 'genuine errors'. Why?
If we knew the true value for each abstract then we could say that any deviation is rating error. Not knowing the true value, each rating can be viewed as an estimation of the true value. And rather than a constant measurement (rating) uncertainty for all abstracts it could just as easily (and more likely to my mind) be that some abstracts have a greater measurement uncertainty than others.
By this latter understanding no rating can be erroneous unless we know the true value and the uncertainty of measurement (rating).
The notion that a disagreement between ratings indicates an error on the part of the raters is full of assumptions that I don't think can be justified.
Of course you prefaced the quote I used to start this comment with, "...among people interested in the arcane enough to even follow this debate..." and now I've let the cat out of the bag -- that I'm still following it :)
Kevin O'Neill writes:"The notion that a disagreement between ratings indicates an error on the part of the raters is full of assumptions that I don't think can be justified."
ReplyDeleteUmm ... I think he should have added. "with the data we have now."
If we were truly interested in the rating uncertainty of each, or even a small subset, of the abstracts we could always run rating tests on them to see what the results from 100 raters might tell us.
The fact that no one has done this means our collective laziness is probably greater than our collective need to know :)
Tom:
ReplyDeleteI misread my spreadsheet. Let's try again:
a) The probability of error by one reader is x.
If two independent readers, with the same error probability of x, disagree, that has probability of 2x*(1-x), because:
- one reader must make an error: probability x;
- the other reader must not make an error: probability (1-x);
- the readers can switch right/wrong roles.
Put it all together, and the probability of one reader (out of 2) getting it right and the other getting it wrong is 2 * x * (1-x) = 2x(1-x) which is never greater than 1/2.
b) Another way in which two readers can disagree: they both get it wrong, but they judge different categories. Assuming for simplicity that the error would be equally in any of 6 directions, the probability would be:
x for one error;
x for the 2nd error;
5/6 that they don't agree on the category;
giving 5x^2/6 probability that they both get it wrong and disagree anyway.
Thus, the probability that the two readers disagree is:
2x(1-x) + 5x^2/6 = D.
For D = 0.33, this gives x = .185
Correspondingly:
c) the probability they both get it right is (1-x)*(1-x);
d) the probability they both get it wrong, but agree on the evaluation is:
(1/6) * x * x = x^2/6.
So the probability of agreement is:
A = (1-x)^2 + x^2/6
Note that:
A + D =
[(1-x)^2 + x^/6] + [5x^2/6 + 2x(1-x)
= 1 - 2x + x^2 + x^2 + 2x - 2x^2]
= 1 identically.