5 things you didn’t know about using games for assessment

Richard Landers at the University of Minnesota hosted an excellent interdisciplinary workshop on game based assessment this summer featuring the latest international research. As part of the first day‘s content, a number of speakers summarised their findings on using and designing games. This included applications such as psychologically profiling candidates for employment and for assessing skills. It’s an excellent collection of the state of the art on game based assessment practices. Here’s just five of the top key points.

1.Games are as reliable as other tests

Games are as reliable as other forms of candidate psychometric testing, and are often preferred by candidates – as long as they are carefully designed and administered. However, not all game-based assessments are the same, and they have to be designed and administered rigorously within the parameters of their validity. They also have the same limitations.

2. Behaviour in games is indicative of behaviour elsewhere

Research into how players behave in games played solely for pleasure shows that these behaviours reliably crop up in other parts of life, including the workplace. This encompasses behaviour such as teamwork, a positive attitude to learning, competitiveness, as well as negative behaviours such as rudeness and aggression.

3. Computer game design and application can be very time-consuming

In design, implementation and analysis, the application of computer games for valid testing purposes holds huge promise – but is very time consuming. For valid and reliable results against specific criteria, development of new test games can take years. However, if the intended application is a roll out to a mass of candidates in the thousands or millions, such investment may be worthwhile. Mass application of games can also generate valuable datasets to identify performance, though analysis techniques are still in development.

4. Participants feel game based assessment offers fairer methods of recruitment

Candidates often prefer games-based assessments as being less stressful than conventional testing. They also feel that self-assessment games including specific features offer fairer outcomes. Like tests, games offer a scoring system, but participants especially feel that the contextual elements of games including immediate feedback, a progressive level structure, and a business-related narrative contribute to the fairness of the outcomes.

5. Game based assessment can reproduce existing psychometric test biases

Unfortunately, the structural biases evident in existing psychometric testing methods are not avoided when games are designed and implemented to reliably test for the same variables. However, this may suggest that the bias issue is related to existing scientific consensus on the variables themselves rather than the method of testing or a lack of proficiency within disadvantaged groups.

Should LARP be not-for-profit?

If there is one oft-cited rule that almost all LARP organisers face it is this: LARP does not make money. In fact, scraping together the cash to ensure you have enough funds to run a future event, keep the group website registered and online, or to pay for prop storage when that convenient friend’s garage becomes unavailable is the constant worry of anyone trying to keep a LARPing group together.

There is evidence to the contrary (of course!). The professional ‘LARPwrights’ of Nordic LARP, the large festival systems that at least make enough money to pay their employees, the adept entrepreneurs who transform LARP into a training activity or even the savvy LARPers who run the same game twice to save on props.

But here’s the interesting question; is there something about LARP that would be ‘lost’ if events were run on a for-profit basis?

Patching together a technicolour reality

One of the very first worries any player is likely to face, whatever their gender, is “what should I wear?” Costume is perhaps not the most important part of entering into the LARP world, but it does play a significant role. In this post, I thought I would share the story of one of my earliest costume creations for a LARP character, the emotions and the practicalities that surrounded that item, and how I think such items are important symbolic artefacts in the reality of the game.

Keeping Dry

Starting my experience of LARP as I did in the north-west UK, the unpredictable nature of the British weather played a large part in my experience of the game. LARP was fun, but it was also interspersed with clammy goosebumps and shivering, which (if you’ll pardon the pun) put rather a dampener on things. My early experiences of ‘fest’ LARP also made me keenly aware of the large proportion of the time that was not running screaming into battle, but instead standing around in the drizzle, huddling together with other players over a small campfire and passing a small hipflask around to keep warm. Cloaks were available for sale, but they tended to be very pricey, and I was on a student budget. Someone at university made blanket cloaks (rectangular woollen things with a clasp), which were better than nothing but didn’t quite keep the rain out or the warmth in. Most importantly, they didn’t convey the dramatic swirling atmosphere of highwayman-come-fantasy romance that the game seemed to aspire to.

Taking all of the above into consideration, and having acquired a budget sewing machine, I began to hatch my plans. Cloaks require a lot of fabric, and wool was expensive. I thought about how I had seen people playing wood-elves in leaf-patterned leather armour, and how I intended my next character to be a mildly aggressive wood-fairy, inspired by woodland, thorns and branches. I wanted something that reflected forest colours in all seasons but which could be used in different situations to blend in as well as to stand out. With that in mind I decided to make a reversible patchwork cloak. I would be able to use scraps to assemble the quantity of fabric required as well as to get the “dappled woodland” feeling I wanted. The gap between the patchwork and cover layer would create an insulating barrier and thus the cloak would be warm even if made from synthetic fabrics.

I bought a pattern, and made free with the sample section of a local curtain shop. I also kept an eye out for bargains in the offcut bin, and bought a large amount of green suede-like fabric for the backing. At a tiny tea-table I sewed square after square together into long strips, then sewed these to each other until I could barely see. I toiled like this for three days, barely eating or sleeping, then trimmed the fabric to the pattern shapes on an ironing board. The more I worked on the fabric, the more I thought about the character. My frustration at a broken machine needle became yet another obstacle in my way to the character’s goals, and why she had left the forest to meet the other players in the game. The fusing of the pattern pieces was fundamentally part of developing the story I intended to play, since as the patches came together, so did the personality of the fictional role.

After the frenzy of stitching, the cloak had to be washed, rinsed in waterproofing solution and dried. Waterproofing solution has a particular waxy scent to it, a scent that to this day I associate with adopting the belligerence and stubborn prejudices of that character. When a close friend of mine became tainted in the game world I refused to associate with her during game time for around three years. We are, of course, still marvellous friends the rest of the time.

Getting kitted up

The cloak, once made, was (and still is) enormous. It would not conveniently fit inside my rucksack, but instead had to be rolled and strapped to the outside of the bag, exposed to the elements. Once arrived at the event in the game world I found that the cloak, although successful at keeping the rain off and some warmth in, was so long as to make standing up once having sat down a perilous process. Other players/characters often stood on the hem and accidentally pinned me to the spot. I feel that this difficulty only encouraged my stubborn nature as the character.    The only way to go into combat elements of the game was to throw the garment over my head and hope to return to it later. I did begin to adopt a rather particular attitude towards objects as a result of this, trusting an item only for as long as it was within my grasp, for who knew what purpose another would find for it? Other player characters held strong views against the written word because of the way it could be twisted when taken away from its originator, and my character fell in line with this argument. My interactions with other players, rather than encouraging segregation due to my performed stand-offish personality, actually encouraged development of the fictional community, establishing a different group morality that was distinct from other types of character. However, it was still an easy task to cast off the persona at meal times along with the cloak, transforming it into a warm wrap or picnic blanket on which to chat and socialise with friends from all parts of the game around a hot drink and bread roll.

Reflections

My intentions in constructing this piece of costume were based on many requirements, but I was determined to create something specific to the identity and presentation of the character I would be playing. Although many parts of the cloak were scavenged from materials aimed at a mass market (in curtains), the outcome was in many ways unique, and the experience of producing the cloak distinctly coloured the wearing of it and behaviour while doing so. Making this item was only the beginning in my own journey of making various costumes for characters, as well as dressmaking more generally, and I have found this causes me to look at items of clothing for sale in a very different light. It is nearly impossible to purchase clothing that fits personal requirements of taste as well as practicality and uniqueness in everyday commercial settings, unless custom made. What effect does that have on my ‘performance’ of identity, or of my character (in the non-LARP sense) in the everyday? Does the limited range of options make it easier to identify those in the same social grouping as myself? Are we all likely to share the same frustrations with  ‘disposable’ mass produced fashion? And do the ‘unique’ items of clothing we adopt declare us idols and/or outcasts by default?

Reflections and comments welcome as always.