Finding reliable information about climate science

In brief

  • There is a lot of information on climate change science available in the media and on the internet. It is therefore important to ensure that what you are reading is accurate and not influenced by personal, social or political agendas.
  • As with all scientific fields, climate change science relies on the continued questioning and challenging of ideas. The peer-review process provides a mechanism to quality control scientific discourse and therefore peer-reviewed papers provide a reliable and quality assured source of information on climate change science.
  • Once an article is published in a peer-reviewed journal, its ideas can be challenged or supported by other scientists with peer-reviewed articles of their own. Peer-review ensures that published findings are objective and conform to accepted scientific standards. Without the peer-review system, publication of research findings would be arbitrary, and possibly influenced by personal, social or political agendas.
  • Major reports are normally peer-reviewed. For example, the Intergovernmental Panel on Climate Change (IPCC) assesses the peer-reviewed literature on climate change every five to six years. The reports are subject to an intense peer-review process involving hundreds of scientific experts and government reviewers. This unprecedented level of peer and government review makes this compendium of climate change science one of the most scrutinised documents in the history of science.
  • The Fourth Assessment Report, produced by the IPCC in 2007, states global warming is 'unequivocal' and 'most of the observed increase in globally-averaged temperatures since the mid-20th century is very likely due to the observed increase in greenhouse gas concentrations'.
  • The findings of the IPCC have been strongly supported by more recent publications that summarise the peer-reviewed literature including, the American National Academies, the Royal Society in the United Kingdom and the Australian Academy of Science.

In detail

The purpose of peer review

Scientific findings appear in a wide range of sources, some more credible than others (Brown, 2006). A common source of information is the media. While science and environment reporters are usually familiar with scientific publishing, many other journalists do not have the expertise to interrogate press releases or give suitable weight to different claims (Brown, 2006).

Peer review is the process of quality control of science to ensure published findings are objective, unbiased and conform to accepted scientific standards. Relying on peer-reviewed information is one way non-scientists can avoid being mislead by people who use scientific findings selectively or dishonestly to support a particular agenda. However, one problem non-scientists face when interpreting and judging scientific issues is that the peer-reviewed literature is not generally accessible (Moore, 2006) and it is often too technical.

Science relies on continued questioning and challenging of ideas. The research community understands that scientific information that has not been peer reviewed should not be taken seriously and scientists draw a sharp distinction between material in blogs or opinion pieces and peer-reviewed scientific literature (Moore, 2006).

Reliable information evolves in the following ways:

  • Research findings or hypotheses are published in peer-reviewed journals.
  • Other scientists then challenge or support these results with peer-reviewed articles of their own.
  • Eventually a consensus builds around the idea or set of ideas that explain the observations most successfully.
  • Sometimes, the prevailing ideas may be challenged by conflicting data or new theories. These new ideas may eventually be rejected or may replace the old ideas as a new consensus builds because the new ideas provide a more convincing explanation of observations than the old ones.
  • Science moves forward through the building of mounting consensus, not through an isolated paper that adopts a 'maverick' position, even if it has been peer reviewed (Moore, 2006).

In the past it has sometimes taken a long time before an accepted position was replaced by a new consensus. However, the pace of scientific communication today and the intense focus on areas like climate change means that new ideas are tested and rejected or accepted more quickly. In contrast, vested interests often cherry-pick the 'facts' they find most convenient to their agenda. Ignoring every paper bar the one that most conveniently suits a preconceived position, especially if that paper has subsequently been shown to be incorrect, could be considered scientific misconduct (Moore, 2006).

Scientists must be able to back up their views with results published in peer-reviewed literature. Without a peer-review system, publication of research findings would be arbitrary and more easily influenced by personal, social or political agendas (Brown, 2004). Moreover, the sheer volume of scientific information which scientists would have to deal with if no quality control were applied would make it impossible to operate the 'marketplace of ideas' in a rational way.

Scientists give different weight to ideas that are published in different parts of the scientific media because different forms of peer review apply for scientific journals, reports, books and conference proceedings.

Scientific journals

When a manuscript is submitted to a scientific journal, the editorial staff refer the manuscript to experts in the same field of work, who are qualified to impartially judge the competence, significance and originality of the research. Reviewers are anonymous and independent, and they do not communicate with each other. Comments from at least two reviewers are normally required (Wager, 2006). They identify strengths and weaknesses of the work with suggestions for improvement. These comments are passed to the authors of the manuscript with a covering note from the editor, indicating whether changes need to be made before the manuscript is acceptable for publication.

The final decision about whether the manuscript should be published lies with the editor. When an editor receives very positive and very negative reviews for the same manuscript, he/she may seek comments from additional reviewers, or ask authors to reply to the criticisms before he/she makes a judgement. Publication of a controversial paper can lead to publication of brief comments from other authors, which in turn are dealt with by the authors of the original paper, for example Mann et al. (2009) and Rahmstorf (2007). This type of debate is part of the peer-review system.

Access to data and analyses that are not fully reported in a submitted manuscript may be necessary for thorough peer review (Lee and Bero, 2006). The Council of Science Editors recommends that journals establish data-access policies for editorial evaluation and peer review before and after publication so that the validity of the work can be verified or errors corrected (Lee and Bero, 2006).

It is generally understood among scientists that there is a hierarchy of journals (Lawrence, 2003). At the top of the pyramid stand the most prestigious multidisciplinary journals, such as Nature and Science; below them is a group of good discipline-specific journals with varying degrees of selectivity and specialisation; and at the base lies a large and diverse collection of journals whose foci are narrow, regional or merely unselective (Jennings, 2006).

The ways peer review is put into practice vary across journals and disciplines. The highest quality scientific journals often have the highest rejection rates for submitted manuscripts because of their rigorous standards and the number of manuscripts they receive. For example, Nature receives about 10 000 papers every year, the editors reject about 60 per cent of them without review, and about seven per cent of submissions are published (Greaves et al. 2006). Rejection rates at The Lancet and the New England Journal of Medicine are well over 90 per cent (Wager, 2006). In contrast, some electronic journals accept anything that meets minimum scientific and ethical standards, and between these two extremes are the many specialty journals with rejection rates of around 50 per cent (Wager, 2006). All use peer review, and may even use the same reviewers, yet the outcomes are different so the weight that other scientists give to the findings is very dependent on the journal in which they are published (Wager, 2006).

Journals are evaluated using impact factors that are based on how often their papers are cited in other journals, up to five years post-publication. Journal Citation Reports summarise citations from more than 10 000 journals and proceedings in the sciences indexed in the Web of Science database. For example, Nature has an impact factor of around 30, while Australian Journal of Botany has an impact factor of one.

Books and conference proceedings

Some university presses undertake peer review of books. After positive review by two or three independent referees, a university press sends the manuscript to an editorial board for final approval. Not all scientific books are peer reviewed, even though the help of many scientists may be acknowledged—the expertise and impartiality of these scientists cannot be guaranteed.

Some scientific conference proceedings include peer-reviewed papers. These papers are reviewed by a program committee (the equivalent of an editorial board), which generally requests inputs from referees. The rejection rate is usually low.

Major reports

Major reports are normally peer reviewed. Good examples include the Millennium Ecosystem Assessment and the IPCC Fourth Assessment Report.

Every five to six years, the IPCC produces reports on climate change science that underpin policy developments within the UNFCCC framework. The reports consider results which have been published for at least a year in peer-reviewed journals so that the scientific community has had time to respond to them and, if necessary, challenge them. Because the reports potentially have such a significant impact on the progress of climate change science and on policy developments, they are subject to an extraordinarily intense, virtually 'line-by-line', peer-review process.

Four drafts of the IPCC (2007 a,b,c) report were produced over four years. More than 450 lead authors and more than 800 contributing authors were involved.

Comments were sought from over 2500 scientific expert reviewers for the first three drafts, and from government reviewers for the last three drafts. Thousands of comments were received on each chapter, and the relevant authors were required to address every comment in a standard spreadsheet. Review Editors were appointed to independently ensure that comments were handled properly. For each report, the Summary for Policymakers was approved by officials from 113 governments, which represents their understanding and ownership of the report.

This unprecedented level of peer and government review makes this compendium of climate change science one of the most scrutinised documents in the history of science.

References

Brown, T. (2004). 'Peer review and the acceptance of new scientific ideas'. Published by Sense about science, London, 49 pp, www.senseaboutscience.org/

Brown, T. (2006). 'I don't know what to believe'. Nature, doi:10.1038/nature04998

Greaves, S., Scott, J., Clarke, M., Miller, L., Hannay, T., Thomas, A. and Campbell, P. (2006). 'Overview: Nature's peer review trial'. Nature, doi:10.1038/nature05535

Groves, T. (2006). 'Quality and value: How can we get the best out of peer review?' Nature doi:10.1038/nature04995

IPCC (2007a). 'Climate change 2007: The physical science basis'. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change [Solomon, S., D. Qin, M. Manning, Z. Chen, M. Marquis, K.B. Averyt, M.Tignor and H.L. Miller (eds.)]. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA. http:www.ipcc.ch

IPCC (2007b). 'Climate change 2007: Impacts, adaptation and vulnerability'. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, M.L. Parry, O.F. Canziani, J.P. Palutikof, P.J. van der Linden and C.E. Hanson, Eds., Cambridge University Press, Cambridge, UK. http:www.ipcc.ch

IPCC (2007c). 'Climate change 2007: Mitigation'. Contribution of Working group III to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change [B. Metz, O. R. Davidson, P.R. Bosch, R. Dave, L. A. Meyer (eds)], Cambridge University Press, Cambridge, United Kingdom and New York, USA. http:www.ipcc.ch

Jennings, C. (2006). 'Quality and value: The true purpose of peer review'. Nature, doi:10.1038/nature05032

Lawrence, P.A. (2003). 'The politics of publication'. Nature, 422, 259–261 pp.

Lee, K. and Bero, L. (2006). 'Increasing accountability. What authors, editors and reviewers should do to improve peer review' Nature, doi:10.1038/nature05007

Mann, M.E., Bradley, R.S. and Hughes, M.K. (2009). Reply to McIntyre and McKitrick: 'Proxy-based temperature reconstructions are robust'. Proceedings of the National Academy of Sciences of the United States of America, http:www.pnas.org/content/106/6/E11.extract

Moore, J. (2006). 'Does peer review mean the same to the public as it does to scientists?' Nature, doi:10.1038/nature05009

Rahmstorf, S. (2007). Response to Comments on 'A semi-empirical approach to projecting future sea-level rise'. Science, 317. doi: 10.1126/science.1141283

Wager, E. (2006). 'What is it for? Analysing the purpose of peer review'. Nature, doi:10.1038/nature04990