Recent journal articles on Post Normal Science and Knowledge Quality Assessment
|Posted by Jeroen on Tuesday, October 02 2012 @ 05:47:15 CEST
Post-normal reflections on the Science-Policy Interface
Methods for knowledge quality assessment
- S. Funtowicz, R. Strand. (2011) Change and commitment: beyond risk and responsibility. Journal of Risk Research, 14 (8) 995-1003, DOI:10.1080/13669877.2011.571784
- J.P. van der Sluijs, R. van Est, M. Riphagen (2010) Beyond consensus: reflections from a democratic perspective on the interaction between climate politics and science, Current Opinion in Environmental Sustainability 2, 409-415, DOI: 10.1016/j.cosust.2010.10.003
- D.J. Dankel, R. Aps, G. Padda, C. Rockmann, J.P. van der Sluijs, D.C. Wilson, and P. Degnbol (2012) Advice under uncertainty in the marine system, ICES Journal of Marine Science, 69 (1) 3-7. doi:10.1093/icesjms/fsr179
- J.A. Curry and P.J. Webster, (2011) Climate Science and the Uncertainty Monster. Bulletin of the American Meteorological Society
- Turnpenny, J.R. (2012) Lessons from post-normal science for climate science-sceptic debates, Wiley Interdisciplinary Reviews: Climate Change 3 (5) 397-407.
- Röckmann, C., Ulrich, C., Dreyer, M., Bell, E., Borodzicz, E., Haapasaari, P., Hauge, K.H., (...), Pastoors, M. (2012) The added value of participatory modelling in fisheries management - what has been learnt?, Marine Policy 36 (5) 1072-1085.
Case studies applying tools for knowledge quality assessment
- L. Maxim and J.P. van der Sluijs (2011). Quality in environmental science for policy: Assessing uncertainty as a component of policy analysis, Environmental Science and Policy, 14, (4) 482-492. doi:10.1016/j.envsci.2011.01.003
- P. Kloprogge, J.P. van der Sluijs and A.C. Petersen (2011). A method for the analysis of assumptions in model-based environmental assessments, Environmental Modelling & Software 26, 289-301, Doi: 10.1016/j.envsoft.2009.06.009
- J. de Boer, J.A. Wardekker, J.P. van der Sluijs (2010), Frame-based guide to situated decision-making on climate change, Global Environmental Change 20 (3) 502-510.
- A.B. Knol, P. Slottje, J.P. van der Sluijs, E. Lebret (2010), The use of expert elicitation in environmental health impact assessment: a seven step procedure, Environmental Health 9:19.
- J.A. Wardekker, A. de Jong, L. van Bree, W.C. Turkenburg, J.P. van der Sluijs (2012) Health risks of climate change: An assessment of uncertainties and its implications for adaptation policies. Environmental Health, 11 (1) 67 doi:10.1186/1476-069X-11-67
- A. de Jong, J.A. Wardekker, J.P. van der Sluijs (2012) Assumptions in quantitative analyses of health risks of overhead power lines. Environmental science & policy, 16: 114-121. doi:10.1016/j.envsci.2011.11.012
- E. Laes, G. Meskens and J.P. van der Sluijs (2011). On the contribution of external cost calculations to energy system governance: The case of a potential large-scale nuclear accident. Energy Policy, 39 (9) 5664-5673. doi:10.1016/j.enpol.2011.04.016
- L. Maxim and J.P. van der Sluijs (2010) Expert explanations of honeybee losses in areas of extensive agriculture in France: Gaucho® compared with other supposed causal factors. Environmental Research Letters, 5 (1) 014006 (12pp)
Post-normal reflections on the Science Policy Interface
S. Funtowicz, R. Strand. (2011) Change and commitment: beyond risk and responsibility. Journal of Risk Research, 14 (8) 995-1003, DOI:10.1080/13669877.2011.571784
Abstract Risk and responsibility have always been linked philosophically in the Western tradition. The purpose of this article is to discuss possible alternatives to the centrality of the risk discourse, arguing that such alternatives call for a revision in the concept of responsibility, decoupling it from the aspirations of control over Nature and the future. It implies also a more complex relation between knowledge and action. Rather than believing that contemporary global challenges will be sufficiently met by being responsible under risk, we will explore how to stay committed in times of uncertainty and change.
J.P. van der Sluijs, R. van Est, M. Riphagen (2010) Beyond consensus: reflections from a democratic perspective on the interaction between climate politics and science, Current Opinion in Environmental Sustainability 2, 409-415, DOI: 10.1016/j.cosust.2010.10.003
Abstract The international debate about the Intergovernmental Panel on Climate Change (IPCC) and climate science in the aftermath of ‘Climategate’ gives cause for reflection. While the main emphasis lies on evaluating the procedures of the IPCC during the production of the fourth assessment report, too little attention has been paid to the political role of the IPCC. This article reflects on that political role by distinguishing three strategies to deal with scientific uncertainties in interfacing science and policy: 1) quantify uncertainty, 2) building scientific consensus, and 3) openness about ignorance. Each strategy has strengths and weaknesses. The way the international community has set up the IPCC and its procedures has basically been guided by the consensus approach. The current emphasis on restoring faith in the IPCC by improving its procedures reinforces this strategy. Guaranteeing the scientific reliability of IPCC reports is indeed essential but it does not address the main weakness of the consensus approach: the underexposure of both scientific and political dissent. As a result of this weakness climate science has become politicized over the past decades. Moreover, as we illustrate for the Netherlands, the consensus approach has hindered a full-blown political climate debate. The third policy strategy that aims for more openness and attention for diversity and deep uncertainty in knowledge and views may inspire more democratic ways to organize the interface between climate politics and science.
D.J. Dankel, R. Aps, G. Padda, C. Rockmann, J.P. van der Sluijs, D.C. Wilson, and P. Degnbol (2012) Advice under uncertainty in the marine system, ICES Journal of Marine Science, 69 (1) 3-7. doi:10.1093/icesjms/fsr179
Abstract There is some uncertainty in the fisheries science–policy interface. Although progress has been made towards more transparency and participation in fisheries science in ICES Areas, routine use of state-of-the-art quantitative and qualitative tools to address uncertainty systematically is still lacking. Fisheries science that gives advice to policy-making is plagued by uncertainties; the stakes of the policies are high and value-laden and need therefore to be treated as an example of “post-normal science” (PNS). To achieve robust governance, understanding of the characteristics and implications of the scientific uncertainties for management strategies need to come to the centre of the table. This can be achieved using state-of-the-art tools such as pedigree matrices and uncertainty matrices, as developed by PNS scholars and used in similar science–policy arenas on other complex issues. An explicit extension of the peer community within maritime systems will be required to put these new tools in place. These new competences become even more important as many countries within the ICES Area are now embarking on new policies.
J.A. Curry and P.J. Webster, (2011) Climate Science and the Uncertainty Monster. Bulletin of the American Meteorological Society
Abstract How to understand and reason about uncertainty in climate science is a topic that is receiving increasing attention in both the scientific and philosophical literature. This paper provides a perspective on exploring ways to understand, assess and reason about uncertainty in climate science, including application to the Intergovernmental Panel on Climate Change (IPCC) assessment reports. Uncertainty associated with climate science and the science-policy interface presents unique challenges owing to complexity of the climate system itself, the potential for adverse socioeconomic impacts of climate change, and politicization of proposed policies to reduce societal vulnerability to climate change. The challenges to handling uncertainty at the science-policy interface are framed using the ‘monster’ metaphor, whereby attempts to tame the monster are described. An uncertainty lexicon is provided that describes the natures and levels of uncertainty and ways of representing and reasoning about uncertainty. Uncertainty of climate models is interpreted in the context of model inadequacy, uncertainty in model parameter values, and initial condition uncertainty. We examine the challenges of building confidence in climate models and in particular, the issue of confidence in simulations of the 21st century climate. The treatment of uncertainty in the IPCC assessment reports is examined, including the IPCC 4th Assessment Report conclusion regarding the attribution of climate change in the latter half of the 20th century. Ideas for monster taming strategies are discussed for institutions, individual scientists, and communities.
Turnpenny, J.R. (2012) Lessons from post-normal science for climate science-sceptic debates, Wiley Interdisciplinary Reviews: Climate Change 3 (5) 397-407.
Abstract Climate change is a particularly good example of an issue for which the definition of problems, and attempts to resolve them, contain key roles for scientific research, alongside institutions, ideology, and individual beliefs. This can lead to rich empirical questions as to the relative balance of these influences. However, the most visible discussion about the role of scientific knowledge is often a heated and simplistic debate about the details of the science itself. Post-normal science (PNS) is one way to think about the role of science and expertise more generally in addressing problems like climate change. PNS, it is argued, by extending the group of actors feeding expertise of different types into knowledge-making and decision processes, can provide a richer and more in-depth perspective and an extended function of quality control over the process of inquiry. This review introduces the concept of PNS, and some of the responses to it from various quarters. These responses offer valuable insights into the way that scientific expertise is perceived, such as perceptions about why science is used and what science should do, focusing in particular on 'climate science-sceptic' debates. By introducing insights from science studies, political science, policy analysis, and human geography alongside PNS, the paper offers potential ways out of such debates, and also ways that PNS may develop in the future.
Röckmann, C., Ulrich, C., Dreyer, M., Bell, E., Borodzicz, E., Haapasaari, P., Hauge, K.H., (...), Pastoors, M. (2012) The added value of participatory modelling in fisheries management - what has been learnt?, Marine Policy 36 (5) 1072-1085.
Abstract How can uncertain fisheries science be linked with good governance processes, thereby increasing fisheries management legitimacy and effectiveness? Reducing the uncertainties around scientific models has long been perceived as the cure of the fisheries management problem. There is however increasing recognition that uncertainty in the numbers will remain. A lack of transparency with respect to these uncertainties can damage the credibility of science. The EU Commission's proposal for a reformed Common Fisheries Policy calls for more self-management for the fishing industry by increasing fishers' involvement in the planning and execution of policies and boosting the role of fishers' organisations. One way of higher transparency and improved participation is to include stakeholders in the modelling process itself. The JAKFISH project (Judgment And Knowledge in Fisheries Involving StakeHolders) invited fisheries stakeholders to participate in the process of framing the management problem, and to give input and evaluate the scientific models that are used to provide fisheries management advice. JAKFISH investigated various tools to assess and communicate uncertainty around fish stock assessments and fisheries management. Here, a synthesis is presented of the participatory work carried out in four European fishery case studies (Western Baltic herring, North Sea Nephrops, Central Baltic Herring and Mediterranean swordfish), focussing on the uncertainty tools used, the stakeholders' responses to these, and the lessons learnt. It is concluded that participatory modelling has the potential to facilitate and structure discussions between scientists and stakeholders about uncertainties and the quality of the knowledge base. It can also contribute to collective learning, increase legitimacy, and advance scientific understanding. However, when approaching real-life situations, modelling should not be seen as the priority objective. Rather, the crucial step in a science-stakeholder collaboration is the joint problem framing in an open, transparent way.
Methods for Knowledge Quality Assessment (KQA)
L. Maxim and J.P. van der Sluijs (2011). Quality in environmental science for policy: Assessing uncertainty as a component of policy analysis, Environmental Science and Policy, 14, (4) 482-492. doi:10.1016/j.envsci.2011.01.003
Abstract The sheer number of attempts to define and classify uncertainty reveals an awareness of its importance in environmental science for policy, though the nature of uncertainty is often misunderstood. The interdisciplinary field of uncertainty analysis is unstable; there are currently several incomplete notions of uncertainty leading to different and incompatible uncertainty classifications. One of the most salient shortcomings of present-day practice is that most of these classifications focus on quantifying uncertainty while ignoring the qualitative aspects that tend to be decisive in the interface between science and policy. Consequently, the current practices of uncertainty analysis contribute to increasing the perceived precision of scientific knowledge, but do not adequately address its lack of socio-political relevance. The “positivistic” uncertainty analysis models (like those that dominate the fields of climate change modelling and nuclear or chemical risk assessment) have little social relevance, as they do not influence negotiations between stakeholders. From the perspective of the science-policy interface, the current practices of uncertainty analysis are incomplete and incorrectly focused.
We argue that although scientific knowledge produced and used in a context of political decision-making embodies traditional scientific characteristics, it also holds additional properties linked to its influence on social, political, and economic relations. Therefore, the significance of uncertainty cannot be assessed based on quality criteria that refer to the scientific content only; uncertainty must also include quality criteria specific to the properties and roles of this scientific knowledge within political, social, and economic contexts and processes.
We propose a conceptual framework designed to account for such substantive, contextual, and procedural criteria of knowledge quality. At the same time, the proposed framework includes and synthesizes the various classes of uncertainty highlighted in the literature.
P. Kloprogge, J.P. van der Sluijs and A.C. Petersen (2011). A method for the analysis of assumptions in model-based environmental assessments, Environmental Modelling & Software 26, 289-301, Doi: 10.1016/j.envsoft.2009.06.009
Abstract When analysts do model-based scientific assessments of complex environmental problems, they have to make many assumptions. This inevitably involves – to some degree – subjective judgements by the analysts. Although the potential value-ladenness of model-based assessments has been extensively problematized in literature, this has not so far led to a systematic strategy for analyzing this value-ladenness. In this article, a new method is presented to identify, review, and prioritize assumptions in order to assess the potential value-ladenness of important assumptions and to deal with these potentially value-laden assumptions in an explicit and transparent manner. The potential value-ladenness of the assumptions is analyzed using a so-called pedigree matrix. The matrix addresses epistemic (general and discipline-bound) and non-epistemic (socio-political and practical) values. The method can be applied by the analysts doing the assessment in collaboration with peers and stakeholders or by external reviewers. Here, the method is illustrated for the modelling chain that was used to calculate the indicator ‘death and emergency hospital admittances due to the exposure to ozone’ in the Fifth Dutch Environmental Outlook. The weakest links of the calculation chain were identified through a workshop. This method for the analysis of assumptions enables the analysts to make conscious, well-underpinned, transparent choices, and pinpoints the issues in the chain that are important to communicate to the audience of the assessment report.
J. de Boer, J.A. Wardekker, J.P. van der Sluijs (2010), Frame-based guide to situated decision-making on climate change, Global Environmental Change 20 (3) 502-510.
Abstract The present paper describes a frame-based approach to situated-decision-making on climate change. Building on the multidisciplinary literature on the relationship between frames and decision-making, it argues that decision-makers may gain from making frames more explicit and using them for generating different visions about the central issues. Frames act as organizing principles that shape in a “hidden” and taken-for-granted way how people conceptualize an issue. Science-related issues, such as climate change, are often linked to only a few frames, which consistently appear across different policy areas. Indeed, it appears that there are some very contrasting ways in which climate change may be framed. These frames can be characterized in terms of a simple framework that highlights specific interpretations of climate issues. A second framework clarifies the built-in frames of decision tools. Using Thompson's two basic dimensions of decision, it identifies the main uncertainties that should be considered in developing a decision strategy. The paper characterizes four types of decision strategy, focusing on (1) computation, (2) compromise, (3) judgment, or (4) inspiration, and links each strategy to the appropriate methods and tools, as well as the appropriate social structures. Our experiences show that the frame-based guide can work as an eye-opener for decision-makers, particularly where it demonstrates how to add more perspectives to the decision.
A.B. Knol, P. Slottje, J.P. van der Sluijs, E. Lebret (2010), The use of expert elicitation in environmental health impact assessment: a seven step procedure, Environmental Health 9:19.
BackgroundEnvironmental health impact assessments often have to deal with substantial uncertainties. Typically, the knowledge-base is limited with incomplete, or inconsistent evidence and missing or ambiguous data. Consulting experts can help to identify and address uncertainties.
Methods Formal expert elicitation is a structured approach to systematically consult experts on uncertain issues. It is most often used to quantify ranges for poorly known parameters, but may also be useful to further develop qualitative issues such as definitions, assumptions or conceptual (causal) models. A thorough preparation and systematic design and execution of an expert elicitation process may increase the validity of its outcomes and transparency and trustworthiness of its conclusions. Various expert elicitation protocols and methods exist. However, these are often not universally applicable, and need customization to suite the needs of a specific study. In this paper, we set out to develop a widely applicable method for the use of expert elicitation in environmental health impact assessment.
Results We present a practical yet flexible seven step procedure towards organising expert elicitation in the context of environmental health impact assessment, based on existing protocols. We describe how customization for specific applications is always necessary. In particular, three issues affect the choice of methods for a particular application: the types of uncertainties considered, the intended use of the elicited information, and the available resources. We outline how these three considerations guide choices regarding the design and execution of expert elicitation. We present signposts to sources where the issues are discussed in more depth to give the newcomer the insights needed to make the protocol work. The seven step procedure is illustrated using examples from earlier published elicitations in the field of environmental health research.
Conclusions We conclude that, despite some known criticism on its validity, formal expert elicitation can support environmental health research in various ways. Its main purpose is to provide a temporary summary of the limited available knowledge, which can serve as a provisional basis for policy until further research has been carried out.
Case studies applying tools for knowledge quality assessment
J.A. Wardekker, A. de Jong, L. van Bree, W.C. Turkenburg, J.P. van der Sluijs (2012) Health risks of climate change: An assessment of uncertainties and its implications for adaptation policies. Environmental Health, 11 (1) 67 doi:10.1186/1476-069X-11-67
BackgroundProjections of health risks of climate change are surrounded with uncertainties in knowledge. Understanding of these uncertainties will help the selection of appropriate adaptation policies.
Methods We made an inventory of conceivable health impacts of climate change, explored the type and level of uncertainty for each impact, and discussed its implications for adaptation policy. A questionnaire-based expert elicitation was performed using an ordinal scoring scale. Experts were asked to indicate the level of precision with which health risks can be estimated, given the present state of knowledge. We assessed the individual scores, the expertise-weighted descriptive statistics, and the argumentation given for each score. Suggestions were made for how dealing with uncertainties could be taken into account in climate change adaptation policy strategies.
Results The results showed that the direction of change could be indicated for most anticipated health effects. For several potential effects, too little knowledge exists to indicate whether any impact will occur, or whether the impact will be positive or negative. For several effects, rough 'order-of-magnitude' estimates were considered possible. Factors limiting health impact quantification include: lack of data, multi-causality, unknown impacts considering a high-quality health system, complex cause-effect relations leading to multi-directional impacts, possible changes of present-day response-relations, and difficulties in predicting local climate impacts. Participants considered heat-related mortality and non-endemic vector-borne diseases particularly relevant for climate change adaptation.
ConclusionsFor possible climate related health impacts characterised by ignorance, adaptation policies that focus on enhancing the health system's and society's capability of dealing with possible future changes, uncertainties and surprises (e.g. through resilience, flexibility, and adaptive capacity) are most appropriate. For climate related health effects for which rough risk estimates are available, 'robust decision-making' is recommended. For health effects with limited societal and policy relevance, we recommend focusing on no-regret measures. For highly relevant health effects, precautionary measures can be considered. This study indicated that analysing and characterising uncertainty by means of a typology can be a very useful approach for selection and prioritization of preferred adaptation policies to reduce future climate related health risks.
A. de Jong, J.A. Wardekker, J.P. van der Sluijs (2012) Assumptions in quantitative analyses of health risks of overhead power lines. Environmental science & policy, 16: 114-121. doi:10.1016/j.envsci.2011.11.012
Abstract One of the major issues hampering the formulation of uncontested policy decisions on contemporary risks is the presence of uncertainties in various stages of the policy cycle. In literature, different lines are suggested to address the problem of provisional and uncertain evidence. Reflective approaches such as pedigree analysis can be used to explore the quality of evidence when quantification of uncertainties is at stake. One of the issues where the quality of evidence impedes policy making, is the case of electromagnetic fields. In this case, a (statistical) association was suggested with an increased risk on childhood leukaemia in the vicinity of overhead power lines. A biophysical mechanism that could support this association was not found till date however. The Dutch government bases its policy concerning overhead power lines on the precautionary principle. For The Netherlands, previous studies have assessed the potential number of extra cases of childhood leukaemia due to the presence over overhead power lines. However, such a quantification of the health risk of EMF entails a (large) number of assumptions, both prior to and in the calculation chain. In this study, these assumptions were prioritized and critically appraised in an expert elicitation workshop, using a pedigree matrix for characterization of assumptions in assessments. It appeared that assumptions that were regarded to be important in quantifying the health risks show a high value-ladenness. The results show that, given the present state of knowledge, quantification of the health risks of EMF is premature. We consider the current implementation of the precautionary principle by the Dutch government to be adequate.
E. Laes, G. Meskens and J.P. van der Sluijs (2011). On the contribution of external cost calculations to energy system governance: The case of a potential large-scale nuclear accident. Energy Policy, 39 (9) 5664-5673. doi:10.1016/j.enpol.2011.04.016
Absract The contribution of nuclear power to a sustainable energy future is a contested issue. This paper presents a critical review of an attempt to objectify this debate through the calculation of the external costs of a potential large-scale nuclear accident in the ExternE project. A careful dissection of the ExternE approach resulted in a list of 30 calculation steps and assumptions, from which the 6 most contentious ones were selected through a stakeholder internet survey. The policy robustness and relevance of these key assumptions were then assessed in a workshop using the concept of a ‘pedigree of knowledge’. Overall, the workshop outcomes revealed the stakeholder and expert panel's scepticism about the assumptions made: generally these were considered not very plausible, subjected to disagreement, and to a large extent inspired by contextual factors. Such criticism indicates a limited validity and useability of the calculated nuclear accident externality as a trustworthy sustainability indicator. Furthermore, it is our contention that the ExternE project could benefit greatly – in terms of gaining public trust – from employing highly visible procedures of extended peer review such as the pedigree assessment applied to our specific case of the external costs of a potential large-scale nuclear accident.
L. Maxim and J.P. van der Sluijs (2010) Expert explanations of honeybee losses in areas of extensive agriculture in France: Gaucho® compared with other supposed causal factors. Environmental Research Letters, 5 (1) 014006 (12pp)
Abstract Debates on causality are at the core of controversies as regards environmental changes. The present paper presents a new method for analyzing controversies on causality in a context of social debate and the results of its empirical testing. The case study used is the controversy as regards the role played by the insecticide Gaucho®, compared with other supposed causal factors, in the substantial honeybee (Apis mellifera L.) losses reported to have occurred in France between 1994 and 2004.
The method makes use of expert elicitation of the perceived strength of evidence regarding each of Bradford Hill's causality criteria, as regards the link between each of eight possible causal factors identified in attempts to explain each of five signs observed in honeybee colonies. These judgments are elicited from stakeholders and experts involved in the debate, i.e., representatives of Bayer Cropscience, of the Ministry of Agriculture, of the French Food Safety Authority, of beekeepers and of public scientists.
We show that the intense controversy observed in confused and passionate public discourses is much less salient when the various arguments are structured using causation criteria. The contradictions between the different expert views have a triple origin: (1) the lack of shared definition and quantification of the signs observed in colonies; (2) the lack of specialist knowledge on honeybees; and (3) the strategic discursive practices associated with the lack of trust between experts representing stakeholders having diverging stakes in the case.