Measuring the User Experience by Tom Tullis and Bill Albert
At first, usability testing seems fairly straightforward; watch someone using your system and infer something helpful from it. However, once you start to look into the details of doing a proper test, all sorts of questions begin to appear. What type of data are you going to collect? Are you going to have a post-test feedback form? If so, how will that be structured? How will you analyse the results?
I haven’t done any formal usability testing yet, but it will be a focal point of my final year project at university, so I’ve been looking for books that explain the process. Measuring the User Experience initially seems like a textbook full of boring statistics, but on further inspection the information inside is very practical. Tullis and Albert’s book sets out to cover the whole process of doing a usability study, from planning and collecting data all the way through to analysing and presenting it.
It begins by explaining why you’d use metrics in a usability study and introduces the various types of data that you might measure. It’s heavy going to begin at first, with nominal and ordinal data, measures of central tendency and nonparametric tests, but once you reach the third chapter, it starts to make sense. Here there’s a section which outlines ten common usability studies and which metrics you should pick for each of them, including comparing alternative designs and evaluating frequent use of the same product.
The chapters that follow which make up the core of the book cover each type of metric in further detail, including performance, issues-based, self-reported, behavioural, physiological, combined and comparative metrics. Within this mass of information are some great discussions of usability testing issues and how to tackle them.
One of the most interesting of these is “What constitutes an error?”, which is something that I probably wouldn’t consider until I’ve started a study. You’d think that it would be fairly obvious, but apparently “there is no widely accepted definition”. Two other particularly useful sections were a comparison of post-test rating schemes (their suggestion was to use SUS) and a detailed examination of how many participants to use.
After briefly covering web analytics data, card-sorting and accessibility, the book moves on to six case studies before concluding. Considering that Measuring the User Experience covers many topics in great depth, the final chapter does an impressive job of summarising their recommendations. Tullis and Albert give ten key lessons to take away, all of which are practical steps that you can apply to your work. These are indicative of the book as a whole; plenty of technical detail, but wrapped up in a way that you can actually make use of them.
If Steve Krug’s Don’t Make Me Think! is a first step into the world of usability, then Measuring the User Experience is great choice for where to advance your other metaphorical foot to next. For someone like me, who is familiar with many of the concepts but has never had a chance to apply them in a formal study, Tullis and Albert’s book is highly recommended.
About the author
Philip Morton is a user experience consultant at Foolproof in London. He's also the Editor of Thunderbolt, which he has been running since 2000.
You might also enjoy: A comparison of concert tracking websites | Subscribe to the feed