Richard Landers at the University of Minnesota hosted an excellent interdisciplinary workshop on game based assessment this summer featuring the latest international research. As part of the first day‘s content, a number of speakers summarised their findings on using and designing games. This included applications such as psychologically profiling candidates for employment and for assessing skills. It’s an excellent collection of the state of the art on game based assessment practices. Here’s just five of the top key points.
1.Games are as reliable as other tests
Games are as reliable as other forms of candidate psychometric testing, and are often preferred by candidates – as long as they are carefully designed and administered. However, not all game-based assessments are the same, and they have to be designed and administered rigorously within the parameters of their validity. They also have the same limitations.
2. Behaviour in games is indicative of behaviour elsewhere
Research into how players behave in games played solely for pleasure shows that these behaviours reliably crop up in other parts of life, including the workplace. This encompasses behaviour such as teamwork, a positive attitude to learning, competitiveness, as well as negative behaviours such as rudeness and aggression.
3. Computer game design and application can be very time-consuming
In design, implementation and analysis, the application of computer games for valid testing purposes holds huge promise – but is very time consuming. For valid and reliable results against specific criteria, development of new test games can take years. However, if the intended application is a roll out to a mass of candidates in the thousands or millions, such investment may be worthwhile. Mass application of games can also generate valuable datasets to identify performance, though analysis techniques are still in development.
4. Participants feel game based assessment offers fairer methods of recruitment
Candidates often prefer games-based assessments as being less stressful than conventional testing. They also feel that self-assessment games including specific features offer fairer outcomes. Like tests, games offer a scoring system, but participants especially feel that the contextual elements of games including immediate feedback, a progressive level structure, and a business-related narrative contribute to the fairness of the outcomes.
5. Game based assessment can reproduce existing psychometric test biases
Unfortunately, the structural biases evident in existing psychometric testing methods are not avoided when games are designed and implemented to reliably test for the same variables. However, this may suggest that the bias issue is related to existing scientific consensus on the variables themselves rather than the method of testing or a lack of proficiency within disadvantaged groups.
Seriously Learned 