Preferences research into flood risk management

About Jacobs and the Environment Agency

Jacobs is a multinational consulting and professional services company headquartered in the USA and represented in more than 40 countries worldwide, including the UK. Jacobs’ services cover a wide variety of sectors, including infrastructure, environment, energy and health.

The Environment Agency is the UK government organization charged with protecting and enhancing the environment in England (also Wales prior to 2013). With almost 11,000 employees and offices across England, the Environment Agency is responsible for flood management, pollution regulations and conservation.

The challenge

The Broads is a low-lying patchwork of wetlands, towns and farmland comprising shallow lakes, waterways, beaches, dunes and historical landmarks located in the English counties of Norfolk and Suffolk. The shallow lakes, known as broads, were formed by the flooding of medieval peat workings (used for fuel).

The Broads is the UK’s largest protected wetland and third largest inland waterway. With the status of a national park, the Broads is also home to some of the UK’s rarest plants and animals. Though fewer than 7000 people live in the Broads, the area receives more than 7 million visitors a year.

This special area is threatened by sea level rise and increased flood risk arising from climate change. Responding to the threat requires input from stakeholders and community representatives. Decisions reached need to reflect community preferences and be environmentally sustainable, as well as being technically feasible and economically affordable.

The BFI’s Elected Members Forum (EMF) – a senior governance body mainly comprising local councilors representing their councils and constituents – drafted 13 objectives to guide the development and evaluation of possible actions for managing flood risks. Because not all objectives can be equally fulfilled, weights need to be established so that the objectives that are seen as more important are prioritized when choosing actions for implementation.

Examples of these objectives include community engagement and participation, resilience of the built environment in the face of flooding, and economic viability, among others.

Critically, the method used to prioritize these objectives needed to be transparent and objective, and meet standards required by UK government regulations, as summarized by The Green Book: Appraisal and Evaluation in Central Government (2022), in particular:

In some cases complex technical trade-offs at the longlist stage, concerning choices of service scope and service solution, may be assisted by the use of expertly facilitated Multi-Criteria Decision Analysis… The inferior form of multi criteria analysis or MCA is not suitable for Green Book appraisal. It involves simple subjective weighting and scoring is not a recognized method due to its lack of transparency and objectivity.

The solution

In contrast to the above-mentioned “inferior form of multi criteria analysis” 1000minds fulfills UK government regulations for decision-making methods by providing an objective and transparent method for establishing criteria and weights representing their relative importance using the PAPRIKA method.

Compared to what are often classed as “non-choice-based” methods – in essence, involving arbitrary weighting of criteria, such as on a 0-100 scale – PAPRIKA elicits decision-maker’s judgements by asking a series of pairwise comparison questions, each requiring the person to make a choice involving a trade-off. Here is an example of such a trade-off question:

Based on decision-makers’ answers to the trade-off questions, PAPRIKA calculates criteria weights and total scores for the alternatives being considered. Through this process, PAPRIKA results in the development of a linear additive (MCDA) model, as recommended in § 4.13 of the Green Book Supplementary Guidance:

…an approach based firstly on establishing the performance matrix in a proper way, leading either to some limited analysis and choice by direct inspection (Chapter 5) and/or to the development of a linear additive (MCDA) model (Chapter 6) is the one recommended.

Jacobs used 1000minds to prioritize the 13 BFI objectives as part of a five-step process:

1. First, members of the EMF were asked to individually undertake a short 1000minds preferences survey online. This was done before attending an in-person workshop to discuss initial findings.
2. At the in-person workshop members were presented with the online survey results and asked to discuss the prioritization of objectives as a group.

3. Following discussions at the workshop, EMF members were invited to complete two new 1000minds preferences surveys:

   • The first survey focused on the BFI objectives in the “near-term”: approximately 25-30 years until 2050.
   • The second survey focused on the same BFI objectives over the “longer-term”: approximately 100 years until 2125.
4. EMF members attended a workshop where results from the “near-term” and “longer-term” surveys were presented and recommendations regarding the prioritization of objectives made.
5. Members agreed to the prioritization of the BFI objectives.

The results

The results of the 1000minds surveys enabled Jacobs to advise the BFI regarding which objectives local representatives found most important, both in the near future and in the long term.

These weighed objectives are to be used to guide the development and evaluation of possible flood risk management actions, ensuring that the preferences of local people are reflected.

Conclusion

1000minds’ PAPRIKA method offered a rigorous approach to Jacobs and their BFI and Environment Agency clients to eliciting the preferences of Broads stakeholders in a way that is transparent and defensible. PAPRIKA proved to be a powerful and user-friendly tool for complex decision-making problems requiring objectivity, and for decisions where fair representation of diverse voices is key.