Internal User Testing

Posted on 02/11/2011 by


Yesterday, we took a nerve-wracking step and invited folks from within Talis to help us test Kasabi. We asked a range of people—developers, engineers, and even some humans—to take part. This was the first time most of them had seen any part of Kasabi, and the first time anyone outside the team has been invited to spend significant time on it.

It feels like a bit of a milestone, and I was very excited—though that may have been in part due to the SPARQL blend coffee we had on hand.

Follow the Monkeys

Follow the Monkeys We wanted to test various site features, but didn’t want the testers to be too heavily directed. So, we created a couple of surveymonkey questionnaires that would only really let people answer the questions by accomplishing a set of tasks. So, by answering something like:

“As a user, what kinds of content can you contribute to Kasabi?”

a tester would have to actually register and work out how they could contribute before answering.

There were big, empty boxes for answers, and we asked for as much detail as possible.

This meant I could simply ask people to follow the monkeys, and don’t spare the feedback.

Watch this space

We’ve only had a day to go over the feedback, but I can certainly say we’ve had some thorough comments. Never ask for honest, complete feedback if you’re not prepared for it! :)

Not only have we found things to tweak and fix, but it’s also been a useful exercise in testing itself. As community manager, I’ll no doubt be around for a lot of testing and hacking, and it’s been helpful to see how people work with Kasabi first-hand. For me, it’s preparation for introducing Kasabi to the wider world. And, though there are many tools available for testing and capturing and monitoring, there is huge value in first-hand accounts of what a particular person was thinking and doing when they completed a task, or ran into a problem. I also get the impression that giving lots of room for feedback lets people get into the emotion of the thing. Although most of the feedback was straightforward, I could still get an impression of how something feels to use through the words they choose. I wouldn’t get that from just a tick-box or 1-5 rating. (Though that can come in handy for larger groups, I’m sure!)

So, we haven’t set any dates yet, but you’ll see more about the process here on the blog, and I’ll also be up on various channels as we roll them out. We’re already on twitter, for Kasabi-stalking, and will be announcing when and how we get more feedback on Kasabi (from non-Talisians too) soon.

image: “Monkey on a Mac” by thegarethwiscombe, via flickr.

Tagged: , ,
Posted in: Uncategorized