This year, some colleagues and I are doing a somewhat involved research project involving usability tests that will compare searching in the mobile catalog interface to the traditional web interface. One of our great challenges is finding a good way to capture what happens on the screens of the mobile devices (Android and iOS) of our test participants. For the Android users, we’ll be providing them one of our own phones to use during the test. I’ve been looking at using the GoToAssist app as a way to capture what’s on the screen of our Android (actually, it’ll likely be my own Samsung Galaxy S3) phone.
I thought I’d share what’s involved by making a recording on my desktop (using SnagIt) of what the screensharing looks like when viewed on a PC. This setup may work for testing we have in mind, although the video transmission from phone to my PC seems to be delayed by about 10-15 seconds. Also, if I scroll really fast on the phone, the screencapture from GoToAssist doesn’t completely capture that; instead you get kind of a snapshot effect in the transmission and not a smooth video. But this may work well enough.
For the iPhone we’ll be also using for testing, we are hopeful that the Reflector app will do nicely.
This past week, I’ve been working with with two colleagues from campus IT to run a first round of usability tests on library site redesign. Over the course of three days, we watched ten different undergraduate students perform tasks we had designed in advance (see our testing protocol if you’d like more details on what we did). We used CamStudio to record the screens and to capture audio from a USB mic and relied on one of us from the team to serve as an observer who would take notes during each test using this form.
I’m in the middle of re-reading the observer notes and watching the videos of the screen recordings as I try to write up a report summing what needs to be tweaked in the new site and what seems to be working. In the process of doing the tests this week, I learned a few things that will help us for the next round of usability tests on the redesign:
Make sure the testing situation is completely ready for the next test subject before they come in to the room. We asked students to run searches in a catalog search box on the home page. We realized after a while that we should have been clearing the browser cache after each subject was done; we noticed that the previous test subject’s search query was visible in the drop down list below the search box once the next test subject starting typing the same query.
Use more stable screen recording software. Although CamStudio offers two great features–it’s free and it works fairly well–it isn’t the most stable software. I think we’re going to want to look into getting Camtasia Studio or Contribute, which I am pretty sure my college has a site license for.
Tonight, I stumbled on a great post by Matthew Reidsma about how he does usability testing at the libraries at Grand Valley State University. He had a great idea of doing monthly tests with just three test subjects. Even more interesting was his way to having colleagues from the library watch the test in a room separate from the one where the test subject is (the test subject’s computer and the computer in the room where the librarians are watching are connected via screensharing software). This sounds like a great way for the staff doing usability testing to get buy in from colleagues about the changes being made. It also seems like it offers great evidence about how our students actually use our sites, thereby curtailing arguments over design issues where everyone is making suggestions based on theoretical ideas about user behavior.