Comment to the InterAcademy Council Review of the Intergovernmental Panel on Climate Change.
1. Summary
The IPCC procedures should be amended to differentiate between levels of reviewers and deal with an anticipated higher number of comments. The proposed amendments would clarify the roles of reviewers at different levels of the process and expedite the review.
2. The Problem
IPCC reports contribute to global public policy debates and processes, which may have major effects on the daily lives of every person in the world. Every government and large enterprise has already been affected. As the century continues, the effects of policies based on IPCC work will increase in their scope and impact: they will create whole new industrial sectors, thousands of businesses, and many ways of life.
For this reason, the IPCC reports and the processes which create them have been under increasing scrutiny. Members of the public have gained access to draft reports and submitted a large number of comments in ways that were not anticipated when the procedures were originally written. This development is to be applauded.
Yet, this has created problems. The large number of comments on the AR4 created a significantly increased amount of work for the Review Editors and Lead Authors. Some who contributed comments used the IPCC designation of "Expert Reviewer" as a affirmation of expertise, to the point that the term has become a bit of a joke. The IPCC itself contributed to this problem in its press releases dealing with the number of scientists who contributed to the report.
3. Proposed solution
The InterAcademy Council should consider how the inclusion of comments from the broad public should be better implemented. We wish to make three suggestions
1. A new category should be created for members of the public who provide comments in the second stage of the review. Expert Reviewer should be reserved for those described in section 4.2.4.1 First Review (by Experts) of the Procedures. Others whose first contribution comes in the second stage could be called Reviewers.
2. The IPCC should maintain separate lists of Expert Reviewers and Reviewers. Both should be allowed, if they wish, to provide a single link to materials establishing their standing such as a web page, a publication, etc.
3. If the number of comments exceeds the capacity of the Convening Lead Authors and Lead Authors to deal with they should be allowed, with the approval of the IPCC Secretariat, to group comments together for responding.
4. Recommendations
This is a series of concrete recommendations for amendments to the document “Principles Governing IPCC Work, Appendix A – Procedures for the preparation, review, acceptance, adoption, approval and publication of IPCC Reports”, with the effect of implementing the solutions described above. Changes are shown in bold
4.2 Reports Accepted by Working Groups and Reports prepared by the Task Force on National Greenhouse Gas Inventories. . . .
To ensure proper preparation and review, the following steps should be undertaken:
1. Compilation of lists of Coordinating Lead Authors, Lead Authors, Contributing Authors, Expert Reviewers, Reviewers, Review Editors and Government Focal Points.
4.2.4.1 First Review (by Experts)and
First draft Reports should be circulated by Working Group/Task Force Bureau Co-Chairs for review by experts selected by the Working Group/Task Force Bureaux and, in addition, those on the lists provided by governments and participating organisations, noting the need to aim for a range of views, expertise, and geographical representation. The review circulation should include:
• Experts who have significant expertise and/or publications in particular areas covered by the Report.
• Experts nominated by governments as Coordinating Lead Authors, Lead Authors, contributing authors or expert reviewers as included in lists maintained by the IPCC Secretariat.
• Expert reviewers nominated by appropriate organisations.
The designation Expert Reviewer should be reserved to those in these categories.
The first draft Reports should be sent to Government Focal Points, for information, along with a list of those to whom the Report has been sent for review in that country.
The Working Group/Task Force Bureau Co-Chairs should make available to reviewers on request during the review process specific material referenced in the document being reviewed, which is not available in the international published literature. (Nick might want to add something here)
4.2.4.2 Second Review (by Governments, Experts and the Public)5. Conclusion
A revised draft should be distributed by the appropriate Working Group/Task Force Bureau Co-chairs or through the IPCC Secretariat to governments through the designated Government Focal Points, and to all the coordinating lead authors, lead authors and contributing authors and expert reviewers.
Governments, at their discretion, are invited to open the second review to the public. Governments should only forward comments which discuss the accuracy and completeness of the scientific/technical/socio economic content and the overall scientific/technical/socio-economic balance of the drafts. Members of the public who comment shall be designated as IPCC Reviewers.
Governments should send one integrated set of comments for each Report to the appropriate Working Group/Task Force Bureau Co-chairs through their Government Focal Points.
Non-government reviewers should send their further comments to the appropriate Working Group/Task Force Bureau Co-Chairs with a copy to their appropriate Government Focal Point.
Addition to Annex I
4b. REVIEWERS
Function:
To comment on the accuracy and completeness of the scientific/technical/socio economic content and the overall scientific/technical/socio-economic balance of the drafts.
Comment:
Reviewers are members of the public who comment on the text in the second stage of the review process according to their own knowledge and experience. They may be nominated by Governments, national and international organisations, Working Group/Task Force Bureaux, Lead Authors and Contributing Authors.
Annex 2
3. Responsibilities of the Working Group/Task Force Bureau Co-Chairs
The Working Group/Task Force Bureau Co-Chairs coordinating the Report will (a) collect and index the sources received from authors, as well as the accompanying information received about each source and (b) send copies of unpublished sources to expert reviewers who request them during the review process. Reviewers shall have access to these materials by electronic means if possible.
4. Responsibilities of the IPCC Secretariat
The IPCC Secretariat will (a) store the complete sets of indexed, non-published sources for each IPCC Report not prepared by a working group/the Task Force on National Greenhouse Gas Inventories (b) send copies of non-published sources to expert reviewers who request them. Reviewers shall have access to these materials by electronic means if possible.
We believe that explicitly opening the second stage review to the public through their governments would be an important improvement on the transparency of the current process. Governments would benefit from the involvement of their citizens.
Open for comments/suggestions changes. Eli will ask for signatures in a short while
Eli: Amazing things are happening. I again agree with you. Don't let this set a trend.
ReplyDeleteNote that "the public" will not be allowed to comment on draft IPCC reports because this would take away from the big media bang when the final report is released. I would prefer to a well-vetted to a high-publicized report, and draft reports are leaked to the press anyway, but the IPCC is determined to keep this illusion alive. The latest instruction for authors even contains a "thou shalt not blog" clause.
Very nice. I don't have anything substantive to add, but want to voice my support for this.
ReplyDeleteSounds like a variation on my idea from your last post! By the way, this one seems to be missing a title?
ReplyDeleteI think some venue for an open public review at some stage by all interested parties really would be useful, I don't think it would necessarily take away from the "big media bang" on final publication, actually. It might even enhance it by making a greater public awareness of what's coming - you would get a build-up of interest in the final version and all the back-and-forth, when it comes out. Could well be a good thing.
Yep
ReplyDeleteSince Fred has already volunteered, will Mickey , Minnie and Mighty join Viscount Monckton as reviewers of the next report of the International Panel of Cartoon Climatologists?
ReplyDeleteMarc Morano normally disdains peer reviewed journals, but the current issue of Pediatrics reports" 73 to 88 percent of 4-to-6-year-olds offered "3 pairs of identical foods (graham crackers, gummy fruit snacks, and carrots) ... prefer the package with a sticker showing a cartoon character (Shrek, Scooby Doo, or Dora the Explorer)."
As the kids were inclined to say the cartoon-associated products tasted better, the authors conclude that” licensed characters can influence children's eating habits.’
As what's true of advertising goes double for disinformation, a quick licensing deal might help Fred Singer save Heartland's 800 page vanity press alternative to the IPCC report from the nation’s remainder tables and compost heaps.
To whet the appetite of the target demographic-home schoolers with the scientific literacy of 4 to 6 year olds, Singer need only provide unsold copies with a dust jacket offering Shrek, Minnie and Dora in place of himself, Willie and Pat.
Heartland could further raise the tone of the debate with TV spots featuring the amiable ogre brandishing a tree-sized hockey stick as he bellows “Donkey! Donkey!” at passing climatologists.
I think there's a general problem with the IPCC review process, but I don't really feel familiar enough with it to make detailed suggestions for improvement. This seems to point in the right direction: "Expert Reviewer" should be a label awarded by someone else, not one which anybody can claim for himself or herself. And grouping comments is a little like the EPA work which seems to have been scrupulous and effective. It might at least reduce absurdities like Monckton's claim to be a Nobel laureate.
ReplyDeleteOh, and thank you for the bigger comment box!