1000Minds implements a pairwise comparison method we invented and christened ‘PAPRIKA’.

For reasons that will become clear below, PAPRIKA is an acronym for Potentially All Pairwise RanKings of all possible Alternatives.

PAPRIKA (and 1000Minds) has been recognised in seven innovation awards and is patented in the US, New Zealand and Australia.

The method may be gently explained as follows (technical information is available from the See Also section below).

PAPRIKA method

The PAPRIKA method involves the participant – e.g. you! – answering a series of simple pairwise comparison questions, based on your expert knowledge and subjective judgment.

Each question is based on choosing between two hypothetical ‘concepts’ (i.e. alternatives) defined on just two attributes at a time and involving a tradeoff (in effect, the other attributes are the same). An example of a question – involving designing a car – appears below.

Each time you answer a question, PAPRIKA adapts. Based on your answer, and all preceding ones, PAPRIKA chooses another question for you to answer. Again based on your answer, another question is presented. And then another, and another; and so on. This adaptivity means that PAPRIKA is recognised as a type of adaptive conjoint analysis.

The easiest possible choices!

PAPRIKA’s questions can be thought of as being based on ‘partial profiles’ – beginning with just two attributes at a time – in contrast to alternative ‘full-profile’ methods which involve all attributes together at once (e.g. seven or more). The obvious advantage of such simple questions is that they are relatively easy to think about – so you can have more confidence in your answers!

Indeed, if you think about it ... choosing one alternative (‘concept’) from two, defined on just two attributes at a time, is the easiest possible choice in the universe! In contrast, choosing one alternative from three or more is harder; and so is choosing between alternatives defined on more than two attributes.

Such simple questions are repeated with different pairs of hypothetical concepts – always involving trade-offs between different combinations of attributes, two at a time – until enough information about your preferences has been collected to rank accurately the concepts you’re considering (as described on the attributes).

It’s possible to proceed to the ‘next level of decision-making’, involving three attributes at a time, and then four, five, etc – up to the number of attributes you’ve included – but for most practical purposes this makes no difference to the accuracy of your results. Stopping after having answered questions involving just two attributes at a time is usually sufficient.

From your answers, mathematical methods based on linear programming are used to calculate your ’part-worth utilities’, representing the relative importance (weights) of the attributes to you.

These utilities are used to rank any concepts you’re considering, or, depending on the application, all hypothetically possible alternatives/concepts (all possible combinations of the attributes).

What limits the number of choices, so that PAPRIKA is user-friendly?

Each time you pairwise rank a pair of concepts (‘cars’, in the example above), PAPRIKA immediately identifies all other pairs of hypothetical concepts that can be pairwise ranked and eliminates them. It does this by applying a logical property known as ‘transitivity’.

For example, if you rank concept X ahead of concept Y and also Y ahead of concept Z, then, logically – by transitivity! – X must be ranked ahead of Z. And so PAPRIKA ranks this third pair implicitly, and any others similarly implied by transitivity, and eliminates them, so that you’re not burdened by being asked any redundant questions pertaining to these implied rankings.

This elimination procedure ensures that the number of questions you are asked is minimized. And yet in the process of answering a relatively small number of questions you end up having pairwise ranked all hypothetical concepts differentiated on two attributes at a time, either explicitly or implicitly (by transitivity).

The number of choices depends on the number of attributes and levels you’ve entered for your application: the more attributes and/or levels, the more choices. For example, with four attributes and three or four levels within each attribute, approximately 30 choices are required.

The remarkable feat achieved by PAPRIKA (implemented by 1000Minds) – hence the patents and awards – is that in ‘real time’ and very efficiently, it keeps track of all of the potentially millions of pairwise rankings of the hypothetical concepts implied by decision-makers’ answers (as a result of transitivity).

Thus, the PAPRIKA method manages to identify Potentially All Pairwise RanKings of all possible Alternatives (PAPRIKA) representable by the attributes. Because the pairwise rankings are consistent, a complete overall ranking of concepts is defined, based on your preferences.

Individual-level data

As well as its scientific validity and user-friendliness, a major strength of the PAPRIKA method when it is implemented in surveys – i.e. a 1000Minds conjoint survey is run – is that part-worth utilities (weights) are generated for each individual participant, in contrast to other methods that produce aggregate data only.

Individual-level data enables, for example, cluster analysis to be performed to identify ’clusters’ of participants with similar patterns of preferences.

See also

Wikipedia article about PAPRIKA method

Journal article about PAPRIKA method

What is Conjoint Analysis?