Using usability tests to create better mobile experiences

Illustration showing inside the mind of a usability test subject.

This year at ODDI, we have big plans for expanding our mobile presence. We’re partnering with Radio Free Europe/Radio Liberty (RFE/RL) to introduce a series of new mobile products for BBG entities. We’re updating the mobile site, introducing new apps and rolling out a responsive site that works across devices. And to help ensure that we’re on the right track we’re launching an aggressive schedule of small, iterative usability tests.

Designers, developers and editors like to think they know how users will engage with a website. They like to believe they know how a user will tap and swipe through an app. But in reality, actual users may take an unexpected path to find what they’re looking for.

Fortunately it is possible to get a glimpse inside the mind of an end user.

Usability testing is a structured interview technique that can help evaluate and inform the development of a digital product. It gives direct insight into how real users actually navigate the digital interface.

As our office transitions to an agile development environment, the UX team is running smaller, more frequent tests with five or six participants, taking time out for corrections in between each round of testing.

Steve Fuchs, ODDI’s UX and Design Studio manager, is designing and coordinating the studies to help identify major user interface flaws and point out possible content area concerns that need further study.

A short 2-minute video explaining ODDI’s recent usability studies.

Our first mobile usability study

For our first test, we focused on the VOA mobile site. The results of this test served as a benchmark for future development. RFE/RL is redesigning the mobile site, and can use this information to make informed decisions as they move forward. Usability testing helps ensure an improved user-experience.

The tests also yielded information about how users felt about content, and recordings of the studies were made available to the language services for evaluation.

Language Participants
Chinese 3
Spanish 3
Russian 2
Farsi 1
Hausa 1

We tested the site with 10 English as Second Language students from various countries around the world. Each test consisted of an hour-long interview with a user equipped with his or her personal phone (in the end we tested on eight iPhones, one Android and one Blackberry phone).

After a short introduction, the user was asked to perform a series of tasks designed to gauge how easy and intuitive the mobile design was. Tasks included:

  • finding a story to read
  • listening to an audio broadcast
  • demonstrating how they would submit a news photo to Voice of America
  • and finding a story in their native language.


Split screen view of two cameras were used for mobile usability testing. Each session was videotaped with one camera attached to the phone to record the user’s taps and gestures while a second one was focused on the user’s facial expressions. Observers in a separate room watched the live video feed and took notes.

After testing was completed, the information was extracted from the video and combined with the observers’ notes. Metrics were collected to determine whether or not each task was successfully completed and highlight any false starts. The data was analyzed and synthesized into a report with a summary of the findings and a list of recommendations.

One more tool

Usability testing won’t provide the answer to all of your design questions. It’s less of a science and more of an art. But when used by a skilled team of designers and developers it can help provide insight into how to move forward and improve a website or app.

We’ve now successfully performed three rounds of testing for mobile websites and apps for VOA and RFE/RL. We’re currently evaluating tools and techniques to test tablet devices.

These periodic usability tests will help ODDI identify mistakes and prioritize changes, so that we can help U.S. international media sites continue to evolve. The tests will help us make intelligent decisions about design, navigation and information architecture.

- – - – -
How do you evaluate and improve your apps and websites? Post in the comments section below or tweet us @BBGinnovate.
- – - – -

(The foregoing commentary does not constitute endorsement by the US Government, the Broadcasting Board of Governors, VOA, MBN, OCB, RFA, or RFE/RL of the information products or services discussed.)

The following two tabs change content below.
Brian Williamson

Brian Williamson

Brian Williamson is the senior UX designer for the Office of Digital & Design Innovation. Follow him on Twitter: @drawinghands.

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *