Points systems are used in a wide variety of applications worldwide.
Points systems applications may be characterised with respect to two main features:
- The number of criteria they have, and how many categories within each criterion; and
- Whether the points system is for a ‘one-off’ or a ‘repeated’ application.
With respect to the first dot point, most of the points systems discussed below have a maximum of 10 criteria and seven categories on each criterion. A large proportion have between five and seven criteria each and five or fewer categories.
With respect to the second dot point, ‘one-off’ points system applications involve prioritizing or ranking particular individuals / alternatives that are known to the decision-maker. Common examples include ranking job applicants or assessing employees’ performances (eg. Barclay 2001, Blackham & Smith 1989).
In contrast, ‘repeated’ applications involve prioritizing or ranking alternatives (or individuals) in a pool that is continually changing (eg. prioritizing patients as, over time, new patients present for treatment).
This dynamism means that the points system must be able to rank potentially all hypothetically-possible alternatives representable by the system (ie. all combinations of the categories on the criteria) that might ever be considered.
Common examples of repeated applications are in the areas of immigration, health care, education and employment decision making, etc, as discussed in turn below.
Countries that use points systems for ranking immigration applicants according to their qualifications, age, language ability, etc, include:
- Singapore (Singapore Government 2009)
- Australia (Australian Government 2009, Miller 1999)
- Canada (Citizenship & Immigration Canada 2009, Green & Green 1995)
- New Zealand (Immigration New Zealand 2009)
- United Kingdom (UK Border Agency 2009)
Points systems are also increasingly used to prioritize spending across different health services or ‘health technologies’ (horizontal priority setting), and also to prioritize patients for access to a given service (vertical priority setting).
With respect to horizontal priority setting, Ryan et al. (2001, Appendix 5) identified seven examples of points systems used to prioritize health spending proposals by district health authorities and hospitals in Scotland and England (eg. Farrar et al. 2000, Ham 1993) and for prioritizing Medicaid coverage in the US state of Oregon (Oregon Health Services Commission 1991).
With respect to vertical priority setting, New Zealand and Canada, in particular, have developed points systems at a health system-wide level over the last decade for prioritizing access to a wide range of ‘elective’ (ie. non-urgent) health services.
New Zealand has points systems for cardiology and cardiac surgery (five types), gynaecological and infertility treatments, spinal surgery, cataract surgery, vascular surgery, hip and knee replacements, platic surgery, and paediatric surgery (Ministry of Health 2009).
Since 2004, 1000minds has been used by NZ’s Ministry of Health in collaboration with groups of clinicians and their professional bodies to score points systems for most of these specialities (case study, article).
Canada has points systems for cataract surgery, general surgical procedures, hip and knee replacement, magnetic resonance imaging scanning, and children’s mental health (Noseworthy et al. 2003). Points systems have also been used in the United Kingdom and their system-wide adoption debated (Edwards 1999, Derrett et al. 2002). For a survey of these three countries’ points systems, including a literature review, see MacCormick et al. (2003).
Points systems are also used for diagnosing patients and predicting health outcomes. The Medical Algorithms Project (Svirbely & Iyengar 2013) documents more than 12,500 algorithms from 40 branches of medicine, of which a large proportion are points systems. However, in contrast to the other types of application mentioned here, which are all based on the subjective judgements of decision makers, most points systems for diagnosis/prediction purposes are based on statistical analyzes of the medical and epidemiological relationships between the outcomes of interest and their determinants.
Points systems are also widely used internationally for assessing students for admission to health care education institutions with restricted-entry, such as medical, dental and pharmacy schools (eg. Parry et al. 2006, Latif 2004, Collins et al. 1995, Van Susteren et al. 1999).
Thus, in addition to grade point averages and standardised test scores, most admissions processes include interviews (structured and unstructured) whereby applicants are assessed on points systems comprising criteria such as evidence of their compassion, leadership, etc.
They are also used to allocate scholarships to students. For example, in 2004 University of Otago used 1000minds to score a points system for awarding Masters and PhD scholarships (case study).
Points systems are also used for assessing applicants for jobs (on a repeated rather than one-off basis, as discussed earlier) such as assessing health care professionals for job openings, such as junior doctors for hospital posts (eg. Walzman et al. 2005).
Other applications of repeated points system applications include:
- consumer conjoint (trade-off) analysis (Green, Krieger & Wind 2001)
- allocating public housing (Secretary of State 1997)
- predicting parole violations, business bankruptcies, college graduations, etc (for a survey, see Hastie & Dawes 2001)
Across this variety of applications, a range of methods are available for determining a points system’s point values (sometimes known as ‘scoring’ the points system).
Australian Government, Department of Immigration & Citizenship (2009) What Is The Points Test? Available from www.border.gov.au/Lega/Lega/Form/Immi-FAQs/what-is-the-points-test
JM Barclay (2001) “Improving selection interviews with structure: Organizations’ use of ‘behavioural’ interviews”. Personnel Review 30, 81-101.
RB Blackham & D Smith (1989) “Decision-making in a management assessment centre”. European Journal of Operational Research 40, 953-960.
Citizenship & Immigration Canada (2009) Skilled Workers & Professionals: Who Can Apply? – Six Selection Factors & Pass Mark. Available from www.cic.gc.ca/english/immigrate/skilled/apply-factors.asp
JP Collins, GR White, KJ Petrie & EW Willoughby (1995) “A structured panel interview and group exercise in the selection of medical students”. Medical Education 29, 332-336.
S Derrett, N Devlin & A Harrison (2002) “Waiting in the NHS, Part 2: A change of prescription”. Journal of the Royal Society of Medicine 95, 280-283.
RT Edwards (1999) “Points for pain: waiting list priority scoring systems”. British Medical Journal 318, 412-414.
AG Green & DA Green (1995) “Canadian immigration policy: The effectiveness of the point system and other instruments”. Canadian Journal of Economics 28, 1006-1041.
PE Green, AB Krieger & Y Wind (2001) “Thirty years of conjoint analysis: Reflections and prospects”. Interfaces 31, S56-S73.
C Ham (1993) “Rationing in action: Reports from six districts”. British Medical Journal 307, 435-438.
R Hastie & RM Dawes (2001) Rational Choice in an Uncertain World. The Psychology of Judgement & Decision Making. Sage Publications.
Immigration New Zealand (2009) The Skilled Migrant Category Points Indicator. Available from www.immigration.govt.nz/pointsindicator/
DA Latif (2004) “Using the structured interview for a more reliable assessment of pharmacy student applications”. American Journal of Pharmaceutical Education 68, 1-5.
AD MacCormick, WG Collecutt & BR Parry (2003) “Prioritizing patients for elective surgery: A systematic review”. Australia & New Zealand Journal of Surgery 73, 633-642.
PW Miller (1999) “Immigration policy and immigrant quality: The Australian points system”. American Economic Review 89, 192-197.
Ministry of Health, New Zealand (2009) Elective Services: Guidelines for Criteria for Clinical Priority Assessment.
TW Noseworthy, JJ McGurran, DC Hadorn & The Steering Committee of the Western Canada Waiting List Project (2003) “Waiting for scheduled services in Canada: Development of priority-setting scoring systems”. Journal of Evaluation in Clinical Practice 9, 23-31.
Oregon Health Services Commission (1991) Prioritization of Health Services. A Report to the Governor & Legislature. The Commission, Oregon.
J Parry, J Mathers, A Stevens, A Parsons, R Lilford, P Spurgeon & H Thomas (2006) “Admissions processes for five year medical courses at English schools: Review”. British Medical Journal 332, 1005-1009.
M Ryan, DA Scott, C Reeves, A Bate, ER Van Teijlingen, EM Russell, M Napper & CM Robb (2001) “Eliciting public preferences for healthcare: A systematic review of techniques”. Health Technology Assessment 5(5).
Secretary of State (1997) The Allocation of Housing (Procedure) Regulations. Department of the Environment, London.
Singapore Government, Ministry Of Manpower (2009) S-pass. Available from www.mom.gov.sg
JR Svirbely & MS Iyengar (2013) The Medical Algorithms Project. Available from www.medal.org
UK Border Agency, Home Office, United Kingdom (2009) Working in the UK. Available from www.ukba.homeoffice.gov.uk/visas-immigration/working/
TJ Van Susteren, E Suter, LJ Romrell, L Lanier & RL Hatch (1999) “Do interviews really play an important role in the medical school selection decision?” Teaching & Learning in Medicine 11, 66-74.
M Walzman, A Whitehouse, J Singh & K Marlow (2005) “The junior doctor interview process for hospital posts ― time to change?” British Medical Journal Career Focus 331, 76-77.