1 Suddenly I'm back at university in HCI class

I must confess, at uni I found HCI stuff incredibly tedious, yet now that it’s of use to my work, my interest is sparked. A nice, light midweek read. Thanks

posted at 11:30 am on March 28, 2007 by Mathew Browne

2 Monitoring instead of asking

Since the question you ask already influences the user I think it is better not to ask at all. Instead you should monitor the user while using your product/website/whatever. You can see so much when just analyzing the real world usage of your product.

posted at 12:49 pm on March 28, 2007 by Dave Bishop

3 Monitoring and Asking

It is very important to get feedback, when users testing a new product. By monitoring you get only results for the usabillity, but you will never know what the user is thinking about the product and this could be something completly different.

posted at 01:50 am on March 29, 2007 by Thorsten Schiemann

4 Market research or usability test

The discussion above shows that the article did not make clear whether it’s about market research or “only” about usability. If you want to figure out what new products to launch of course you have to ask questions. But if the product is really new and the person you ask does not know such a product yet, the answer might not be accurate. As the author said the person is influenced just by asking this question in a specific environment. This is ok – as long as you are aware of that and maybe ask the same question in different environments.

posted at 06:35 am on March 29, 2007 by Handy Andy

6 Speaking of "syntax"...

...how about this one: “Should they’ve peas or beans for tea? Catch the bus or the tube?”

I’ve never seen the contraction “they’ve” used that-a-way before, but I love it! And why not, Professor? Exactly what rule of English grammar does the construction break? Speaking of “break”: I’ve seen little evidence of the slightest concern for usablility in the Nokia and Motorola mobiles that Cingular has palmed off on me—just in handling phone calls, address books and speed dials. Forget about “surfing the web” on these doorstops.

If I were keep a diary, I’m afraid that it’d be full of [expletive-deleted]s about breaking the devices in question over the designers heads.

posted at 05:44 am on April 02, 2007 by Jeff Stanley

7 Untitled

What about the problem of observation with ethnography – if I know I am being observed I will act quite differently to when I am alone? Example, if you were watching what I do with my computer at work I’m sure you’ll see me do a very industrious 8 hours of non-stop work. If I am not being observed, I will do a lot more web browsing, personal emailing, phone calls, etc – I see it as a similar issue as “life intruding” that your users gets when they walk out the door.

Even the act of keeping a diary forces you to think more about what you are doing, which is behaviour unlike a typical user. I think some diet gurus say to keep a food diary as once you’re observing how much you’re eating you change your behaviour.

I think the problem of trying to please the ethnographer would probably be quite similar to trying to please the usability examiner.

Just a thought.

posted at 07:10 am on April 04, 2007 by Cheryl Lead

8 Cultural limitations

One big problem that’s evident from the last-mentioned scenario in the article —mobile phone users wanting a small networked pc in their palms— is that culture and (lack of) expertise impose limits on imagination and accuracy.

Culture: Most end-users testing for a new type of cell phone will have significant experience using other similar devices. People are used to the UI’s that ship on current models. Therefore it’s unlikely they’ll comment on design flaws that are present in both the tested phone as well as their previously used devices.

(lack of) Expertise: Since people are not articulate about or consciously preoccupied with the many minor frustrations they have to cope with using a device like this, test-subjects are unlikely to give feedback that the basic functionality of the phone is flawed. It’s common for people to think that if making a call is hard, it’s their fault. If they’re typing something with the keypad and it takes a bit while longer then they’d like for the letters to appear on the screen, it’ll disturb them, but not on a conscious level.

That’s why you’ll always get the kind of feedback that tells they want a more advanced device, offering more functionality, offering more features. While actually making the basics work better and stripping out the clutter of unused ‘power features’ would make most users happier in the end.

(Please note that my definition of culture in this context specifically encompasses the way the people have grown accustomed to today’s mediocre interfaces —in it’s literal definition: of average quality— and the acceptance of difficulty and the minor frustrations that are part of contemporary mediocre user experiences)

Only people with domain expertise in usability can articulate these problems, but they’re also not the best test subjects precisely because of the mind-luggage that comes with their domain expertise. Again culture rearing it’s ugly head.

This type of research should not be used as the fundament of usability studies. Whilst it might bring some flaws to light, the results are culturally biased and therefore mostly useless. We must accept that mediocre usability has become part of our culture and thereby the expectations of our test subjects. The only way to advance beyond the current level of quality is by eliminating the influence of this culture of mediocrity in our decision-making process.

I propose the following method. Designers must embed themselves in the field of their target market, becoming their own persona. Besides starting to ‘own’ the problem they set out to solve, this allows them to become intimate with every bit of the culture of their local peers and will allow them to better judge the kind of feedback they get back from the people around them. This part of the process is art, not science.

Usability testing should be confined to gathering measurable data (how long does it take to perform this task? which parts does the user look at when trying to figure this out?), this raw data should finally be aggregated into conclusions and translated into words by people articulate in the domain. This part is science and should be treated as such.

(this should really be a blog post, not a comment, but since my blog isn’t up yet..)

cheers, Dirk

posted at 10:28 am on April 08, 2007 by Dirk Stoop

9 Untitled

Dirk, thanks for your long comment. I really feel the same regarding the need of scientific research when doing usability testing. SO let us know once your blog is up.

posted at 02:27 pm on April 10, 2007 by Handy Andy

10 Great work Ruth!

Thanks for very interesting article. I really enjoyed reading all of your articles. It’s interesting to read ideas, and observations from someone else’s point of view… makes you think more. Keep up the good work. Greetings

posted at 03:15 pm on May 10, 2007 by Jessica Prescott