The ‘PAPRIKA’ method.

1000Minds is based on our patented PAPRIKA (Potentially All Pairwise RanKings of all possible Alternatives) ‘scoring’ method.

Fundamental to PAPRIKA is the principle that overall rankings of alternatives are uniquely defined by all pairwise rankings of the alternatives vis-à-vis each other (provided they’re consistent) – hence the ‘PAPRIKA’ acronym (as above).

PAPRIKA’s mathematical details are explained in our article:

Notwithstanding its technical sophistication, 1000Minds is very intuitive and user-friendly, as illustrated below.

Depending on what you want to do, 1000Minds supports three Multi-Criteria Decision-Making activities (analogously for Conjoint Analysis):

• Preference Values (Points System)
  This involves discovering decision-makers’ preference values – representing the relative importance (‘weights’) of the criteria entered in 1000Minds. For some applications this includes creating a Points System (also known as a ‘Scoring’ or ‘Point-Count’ System).

• Ranked Alternatives
  As well as discovering decision-makers’ preference values (as above), this second activity involves ranking or prioritising alternatives entered into 1000Minds (and all hypothetically possible alternatives, if desired) – including finding the ‘top’ alternative, etc.

• Your Selections
  As well as discovering decision-makers’ preference values and ranking alternatives (as above), this third activity involves selecting alternatives, subject perhaps to a budget constraint – including, when appropriate, considering their ‘value for money’ and allocating resources.

Having selected your preferred activity mode (at the ‘begin model’ step), all you have to do is perform some or all of the following simple steps – where the first four steps correspond to the first activity above, the first five steps to the second activity, and all six to the third activity.

In addition, customisable processes to include potentially 10s or 100s (even 1000s!) of participants in a variety of decision-making activities are available.

For comprehensiveness, the third activity above (Your selections: all six steps) is explained in detail here, as it, in effect, encompasses the first two activities (the first four or five steps).

Step 1: Your criteria

The first thing to do is enter your criteria for your particular decision-making application. The criteria can be in qualitative or quantitative terms and, within reason, you can enter as many, and as many levels within each criterion, as appropriate – it’s up to you.

As an illustration, let’s consider the example of prioritising projects for a business or government organisation – an application most people can probably relate to (or think of an application of your own involving ranking alternatives or individuals – e.g. see solutions).

For this example the following criteria might be appropriate (all illustrations here are 1000Minds screenshots):

Other considerations that are relevant for prioritising projects, such as 'cost’ and ‘confidence in cost estimates’, etc, are introduced later below. For now the criteria have been chosen exclusively to relate to possible benefits associated with the alternatives (projects) being considered.

Instead of guessing the relative importance of the criteria – i.e. their ‘weights’ or ‘point values’ – 1000Minds determines them in a systematic, transparent and repeatable way, as explained at the Your decisions step later below.

Step 2: Alternatives

You can describe the alternatives you’re considering in terms of your criteria either before or after answering the tradeoff questions at the next step. If you don’t have any particular alternatives in mind yet, that’s fine – you can enter them whenever you like.

Step 3: Your decisions

In order to derive your preference values (i.e. ‘weights’ or ‘point values’) – representing the relative importance of the criteria – all you have to do is answer a series of simple pairwise ranking questions.

As in the example below, each question involves two hypothetical alternatives (‘projects’ ) at a time, described according to just two criteria and involving tradeoffs between the criteria.

Such simple questions are repeated with different pairs of hypothetical alternatives, all involving tradeoffs between the criteria, until enough information about your preferences has been collected to accurately rank the alternatives being considered.

As mentioned earlier, this ‘scoring’ method – which is patented and has won innovation awards – is known by the acronym PAPRIKA (Potentially All Pairwise RanKings of all possible Alternatives).

The number of questions for you to answer is minimised because each time you answer a question like the one above, PAPRIKA eliminates all other potential questions (that could be asked) that are implied as corollaries of those questions already answered.

In essence, PAPRIKA does this by exploiting the ‘transitivity’ property – e.g. if alternative A is ranked above alternative B, and B is ranked above alternative C, then, logically (‘by transitivity’), A is ranked above C.

Approximately 30 decisions would be required for the present example. The number of decisions depends on the number of criteria and levels entered: the more criteria and/or levels, the more decisions.

PAPRIKA / 1000Minds keeps track of all of the potentially millions of pairwise rankings of the hypothetical alternatives implied by decision-makers’ answers (as a result of transitivity). This is the remarkable feat achieved by 1000Minds – and all in ‘real time’ (and very efficiently).

Thus PAPRIKA identifies ‘Potentially All Pairwise RanKings of all possible Alternatives’ representable by the criteria and their levels. Because the pairwise rankings are consistent, a complete overall ranking of alternatives is defined.

This type of question (as above) – involving choosing one hypothetical alternative from two (i.e. pairwise ranking) – is a natural type of decision activity that everyone has experience of in their daily lives.

In contrast, other methods such as the Analytic Hierarchy Process (AHP) ask: e.g. On a scale from 1 to 9, how many times more or less important is “Urgency / importance ...” than “Likely effectiveness ...” for prioritising projects?

Clearly, 1000Minds questions are more natural – because picking one alternative from just two possibilities is the easiest choice in the world. This means you can have greater confidence in your answers and ultimately your final decisions.

Step 4: Preference values

Based on your decisions, your preference values (i.e. ‘weights’ or ‘point values’) – representing the relative importance of the criteria – are derived by 1000Minds. Depending on the application, your ‘Points System’ can be used repeatedly as new alternatives enter the frame and old ones exit.

Step 5: Ranked alternatives

As noted earlier, you can describe the alternatives you’re considering in terms of your criteria whenever you like.

After you’ve answered the pairwise ranking questions (at the Your decisions step above), 1000Minds automatically scores each alternative according to your preference values (as above) and, based on the alternatives’ total scores, ranks them from first to last.

As well as entered alternatives, 1000Minds automatically scores and ranks all hypothetically possible alternatives (all combinations of the criteria and their levels). Sometimes it is useful to consider how such possible alternatives – e.g. that might be considered in the future – will be ranked.

You can also include ‘other considerations’ that are relevant – i.e. in addition to your original criteria entered earlier. As in the table above, examples include 'cost’ and ‘confidence in cost estimates’. Other examples (depending on the application) include ‘strength of evidence’, etc.

Such other considerations (e.g. 'cost’) that have numerical data can be graphed against alternatives’ total scores – representing their benefits – on the axes in the chart below (in the context of the present example representing Value for Money).

The size and colour of the bubbles in the chart can be used to represent additional ‘other considerations – e.g. ‘strength of evidence’, etc (or to highlight a particular criterion of special interest, such as ‘fit with organisation’s strategic priorities’, as below).

Thus, depending on what you want to show, 2 - 4 variables of interest can be represented in the chart, to help you make your selections.

Step 6: Your selections

Assisted by the chart below – in the present context, the ‘Value for Money Chart’ – you can select your preferred alternatives (projects). Depending on the application, this may involve choosing the ‘best’ alternative or multiple alternatives.

In many cases, such selections will be subject to a budget constraint (or resource constraint, more generally). This can also be entered into 1000Minds (see the bottom of the table below – e.g. $8 million), so that you are continually apprised of how much you’ve spent and what’s left respectively.

To appreciate the analytical power of the Value for Money Chart, ask yourself (with reference to the chart): With a budget of, say, $8 million, which projects would you select?

Clearly, the focus of attention in the chart is the top-left quadrant – i.e. high Total Scores (Benefits) and low Costs. Possible acceptable tradeoffs are in a nor-easterly direction, along the blue line representing the ‘Pareto (efficiency) frontier’.

(This is ignoring for now – for simplicity – the size and colour of the bubbles, reflecting other potentially important considerations, such as ‘confidence in cost estimates’, etc.)

In addition, the projects can also be ranked in the table according to their Benefit-Cost Ratios (BCR) (i.e. “y/x ratio” in the table) – or any other variables of interest.

As projects are selected (or rejected) in the table – by clicking the ‘ticks’ (or ‘crosses’) in the table – the chart and the totals at the bottom of the table (i.e. totals spent, rejected, not yet allocated) update automatically.

Reports & accessories

• Everything done with 1000Minds is recorded in a report, which lends transparency and focus to discussions with stakeholders.

• If priorities change, 1000Minds can be fine-tuned easily, further earning the confidence of stakeholders.

• After decision-making you can test yourself (or your group of decision-makers) for consistency.

• You can aggregate the preferences of multiple decision-makers and pin-point areas of disagreement right down to individual decisions.

• You can let other users (in other locations) see the same questions as you – so they can be involved in the decision-making too (via phone or email) or simply observe you.

• You can test how your preferences compare with an existing Points System (perhaps that you’ve used in the past).

• You can compare the rankings of alternatives produced by different Points Systems.

1000Minds decision processes

In addition – depending on what you want to do – you can also use 1000Minds to create your own customised processes involving a variety of decision-making activities.

Each process can include as many participants as you like – e.g. potentially 10s or 100s (even 1000s!) of participants.

For more information, please see 1000Minds decision processes.