How health experts developed a prioritization tool for intensive care under COVID19

The coronavirus disease (COVID-19) has claimed nearly 3 million lives globally to date. Hospitals around the world have been overwhelmed by the pandemic, with many running out of beds and ventilators in intensive care units (ICU) to be able to treat all severe cases.

Thanks to its ‘go fast and go early’ approach to lockdown, New Zealand was largely spared from the pandemic’s awful health effects. Nevertheless, early on as the pandemic raged internationally, in preparation for the possibility that NZ hospitals would be overwhelmed, 1000minds was used by health experts to create a transparent and equitable tool for prioritizing COVID-19 patients for ICUs. The result was a scientifically robust and fair prioritization tool consistent with expert clinical and ethical judgment.

Here is how the tool was created:


An expert group was rapidly convened, comprising health professionals from intensive care medicine, infectious diseases and neonatology. As patient prioritization poses important ethical dilemmas, and because Māori and Pacific peoples were likely to be disproportionately affected by the pandemic, representatives from the fields of ethics and Māori health were also included.


In order to eliminate biases inherent in decisions that are made based purely on intuitive judgements, Multi-Criteria Decision-Making (MCDM) was used. MCDM ensures clinically consistent, transparent and fair decisions.

1. Define relevant prioritization criteria

Using a 1000minds rankings survey, each expert was asked to rank 10 patient case ‘vignettes’ representative of people likely to be considered for admission to ICUs.

After individually ranking these vignettes, the experts came together to rank the vignettes as a group and establish a consensus. These individual and group ranking exercises promoted discussion about what the relevant criteria and their levels are for prioritizing COVID-19 patients.

2. Determine weights for the criteria and their levels

The PAPRIKA pairwise comparisons method was used to determine weights for the criteria and their levels.

In a group voting exercise, participants answered a series of simple questions about which of two patients should be prioritized for ICU, where each question involved a trade-off between the criteria. This exercise elicited the experts’ preferences and promoted discussion to even out any disagreements and reach consensus. The experts’ answers to the questions were used by the PAPRIKA method (via 1000minds) to determine the weights on the criteria and their levels. These weights could then be used to rank (prioritize) patients for ICU.

3. Check validity and reliability

To ensure their reliability, some of the trade-off questions were repeated to check for consistency. To evaluate face validity, the experts considered the intuitive plausibility of the relative importance of the criteria and levels implied by the weights. Also, the experts’ consensus ranking of the 10 vignettes created earlier was compared with the ranking produced by the prioritization tools.

Want to create similar decision-support tools for your organization? Check out 1000minds decision-making software’s free 3-week trial, and read more academic publications here about decision-making in the health sector.