University of Otago, New Zealand, 2004-6

Goal: Award student scholarships

Issues:

  • Fairness to students
  • A range of criteria (albeit small)
  • Transparency across the university
  • Lack of common understanding among decision-makers

Outcome: Objective and easily administered tool for awarding scholarships

Background

Around 300 scholarships worth up to $25,000 each for Masters and PhD study are awarded annually, from approximately 1000 applications.

In effect, the ‘winners’ are selected via a university-wide competition by a Scholarships Committee comprising senior academics from the four faculties (Humanities, Sciences, Health Sciences and Commerce).

Each applicant is assessed on three criteria: (1) Grade Point Average (GPA), and an assessment by the Head in the applicant’s nominated department of the student’s (2) past research performance (eg. for an Honours or Masters thesis) and (3) ‘fit’ with the research programme in the department.

Before using 1000minds, the Scholarships Committee, notwithstanding its good intentions, mostly ignored criteria (2) and (3) and did little more than rank applicants by their GPAs. This happened because the ‘hardness’ of the GPA data effectively overwhelmed the ‘softness’ of the data for criteria (2) and (3) – so that they ended up, in effect, being paid ‘lip service’ only.

This generated unease around the university. It was felt that students who had performed exceptionally in their exams (thereby ensuring a high GPA) but who – in the opinions of academics in a position to judge – were less gifted research-wise or who did not fit into the department’s research programme were being awarded scholarships at the expense of other students with slightly lower GPAs but with more promising research careers and who matched their department’s research programme.

A New Approach

To remedy this, in 2004 the Scholarships Committee used 1000minds and created a Points (or Scoring) System for awarding scholarships that properly recognised the importance of all three criteria. The end result was criteria (2) and (3) receiving approximately half of the total points available for assessing students – with the other half dependant on GPA.

These relative weights (derived using 1000minds) were consistent (ex post) with Committee members’ common sense in the sense that the weights had ‘face’ validity.

Moreover, members reached a greater appreciation of what each individual representing their respective faculties and disciplines considered to be important when awarding scholarships.

Finally, the transparency (including an audit trail) of the software’s processes was especially important to the wider University community.

Other Applications

1000minds has also been used at the University of Otago for allocating research grants, for selecting Teaching Fellows in the Department of Design Studies, and for benching-marking the School of Business against its ‘peer’ and ‘aspirant’ Schools internationally.

1000minds is also ideal for assessing students for admission to restricted-entry courses such as medical, dental and pharmacy schools and MBA programmes.