What My Comp Votes Mean

There are no formal scales for voting in the IF Comp, Spring Thing, and other interactive fiction competitions with 1-10 voting systems. A lot of people post scores with their reviews, and sometimes authors – particularly if they have expectations formed in other communities (or from the professional world, where an average score less than a high 8 is often considered failure) – will read things into those scores that aren’t intended. I’ve seen authors interpret a 5/10 rating as ‘you hate my game’, which is a reasonable misunderstanding if you’re more used to contexts wherein 10/10s are routine, and any rating under 8/10 means ‘don’t play this.’ But really, if I had hated your game you wouldn’t be getting more than a 3.

For some context, the Comp-winning game generally gets a vote average somewhere between 7.5 and 8.4. (The mean, 1999-2015, is 7.94.) Other reviewers, such as Jacqueline Lott, have also published personal rating scales – generally these have somewhat different criteria but a similar distribution.

10. A near-impossible goal. The game is technically strong, highly engaging to play, and inspires at least a little bit of awe. It at least touches on important, difficult, human-condition subject-matter, and its forms of interaction closely involve player action with the narrative. Years from now it will still be a favourite. It’s very unlikely that more than one game in a comp will earn this score: in most years, none will.

8-9. A very good work, at a level of technical and artistic competence strong enough that criticism becomes a more delicate process, a lit-crit exercise rather than a listing of unambiguous failures of craft. Ambitious, in the sense that it’s tackling something difficult in either design or subject-matter. I would happily recommend this. (And since the comp-winning game generally has an average score around 8, any score at or above an 8 implicitly says ‘I would be OK with this getting first place.’)

6-7. Pretty good, but either flawed in some major ways, or unambitious in its goals. I am glad I played this, but I would not recommend it without some caveats.

5. The most important question for me in the whole voting process is: did I get something out of this game? If I didn’t respond in some significant positive way – I didn’t learn anything, didn’t have fun, and wasn’t moved to any emotion more beautiful than annoyance – then the absolute maximum score I’ll assign to a game is 5, no matter how worthy it is in other respects. If a game does satisfy one of those conditions, then 5 is the absolute minimum score it can get, regardless of its other failings. A totally shambolic game that I had a fun time with is a 5.

4. A respectable effort: if I don’t like a game overall, but still think that it has some things going for it, a 4 is a likely score. Maybe it represents decent technical and craft skills but the content is really boring. Maybe there’s a cool idea that wasn’t fully realised. Maybe it’s a noble failure that took on a really difficult goal. Perhaps it’s just a game for which I am very, very much not the intended audience.

3. A bad game with few redeeming qualities. I got little or nothing out of this, and would recommend other players to avoid it.

2. A very bad game; something has gone seriously wrong here. Either the author has put in very little effort, or they presently lack some of the basic skills required to make an decent game.

1.  The score reserved for games which should not have been entered into the Comp. Either they’re troll entries, or they’re so broken as to be unplayable, or they demonstrate a fundamental misunderstanding of what a comp entry ought to aim at.

An increasing number of reviewers don’t post their scores. They’ve got decent reasons, so I should probably lay out why I do:

• I’m going to be giving out a 1-10 vote regardless – that’s the nature of the Comp – so you may as well know what it was.
• In a review I often focus on things that are unusual, strange, broken or disappointing, because that’s interesting; but a good experience of a game often depends just as much on solid, boring, unexceptional, capable work that’s no fun to write about. The score forces me to think about, and acknowledge, the game as a whole.

I make no pretense at rating games objectively, because that is a silly thing to do. I’ll make some attempt to consider games on their own terms, but ultimately the critic’s job is to unpack their own response to the work, not speak on behalf of some imagined ideal audience. To say the same thing from a different angle: while I’m somewhat interested in whether a work succeeds at the goals the author set for it, I don’t think that that should be my primary consideration.

I prefer works that are strong both as stories and as games, but if forced to choose, I’m generally more sympathetic to works with strong prose/story/characterisation and weak gameplay than the reverse -I enjoyed Deadline Enchanter more than I did Shuffling Around. In particular, I’m willing to forgive a game an awful lot if its writing is on point.

The score I post with a review is usually, but not always my final vote; I often tweak things a little at the comp’s end to get scores working better relative to one another (e.g. ‘if this deserves a 6, that sure as hell does.’)

While we’re on the topic: here are my thoughts on how authors should approach reviews.

Two concepts that I refer to in Introcomp entries too often to repeat every time:

The Great Evil is when you use an Introcomp entry as a public alpha; it may not have been tested at all, and the player finds themselves wandering among Under Construction signs. In short, the intro isn’t presented as a cleanly limited preview of what the final, polished play experience will be like, but as the half-built shell of an entire work in progress.

The Little Evil is when your Introcomp entry fails to present a very strong idea about what the heart of the game is going to look like, thus making it difficult for a judge to answer the question ‘how much would I like to continue playing this?’ An Introcomp entry should provide the player with enough content that they feel confident about their answer – most importantly, samples of typical gameplay. Really short intros are often guilty of the Little Evil; so are games where the intro ends on a dramatic change, such as when a mundane protagonist stumbles into a fantasy world.