How to Do UX Testing on Tablets, In Cartoons

feature

One of my great passions is traveling — I get a kick out of understanding other cultures, and how they use technology. As a graduate student, I am rarely able to travel, but working at the BBG is truly the next best thing.

We frequently interact with people from a broad array of cultural backgrounds, with fluency in various languages, who each share their unique perspective and offer a fascinating glimpse of a distant place. (The holiday parties and hallway networking spreads offer a tasty glimpse of distant places — BBG staff can really cook.) This cultural diversity also happens to be convenient for user experience testing because we are able to tap into our ‘natural resource’ to test our digital products, which are often targeted to a specific segment of our global audience.

Typically, when we’re doing in-house testing, we test each application in it’s intended language as well as in English, whether it’s a product for one of the 44 language services at Voice of America or another BBG entity. Most recently we have done testing for Radio Free Asia and Radio Free Europe/Radio Liberty (RFE/RL).

The cultural diversity within BBG is convenient for user experience testing because we are able to tap into this ‘natural resource’ to test our digital products.

And so we need bilingual participants, who also fall into our target demographics, to provide our English-speaking developers, designers and testers with feedback we can understand. Thankfully, our gracious multi-lingual employees and interns are happy to oblige.

We’ve been using software called Morae for in-house UX testing on tablets and mobile phones. Morae allows us to capture video — with more than one camera angle — and record scoring as we go. Having video of a participant’s hand movements allows us to do a more accurate and thorough analysis of how they reacted during certain tasks, as opposed to just having a heat map and data that provide only part of the story. Also, since it’s in-person, we can ask follow-up questions immediately after a task to find out why a participant may have been confused about a task.

This comic strip breaks down how we do UX testing. Hat tip to the illustrious Steve Fuchs (one of our lead UX testers and designers at ODDI) who illustrated these, and who I often work with on UX testing.

1-stake-holders-managers

2-3-design-book-volunteers

4-tv-tray-plus-chair

5-6-table-top-mic

7-two-cameras-record

8-laptop-software

9-test-team

10-collect-analysis-report

One of the more interesting UX tests that I worked on was for RFE/RL mobile websites, which we tested in English and in various languages common in Russia and nearby. Without knowing anything about the site, several of our participants commented right away that it must not be from a Russia-based news network solely from the editorial choices, referring to the top stories on the home page. I recall one participant commenting that no Russian news outlet would dare have the Pussy Riot story front and center, if they reported on it all, because in-country news typically paints the Kremlin in a positive light.

Screen shot 2014-02-03 at 10.01.22 AM

In addition to a responsive website, RFE/RL now offers a mobile app for Android and iOS.

Comments like those remind me why it’s so critical to provide an alternate point of view for people in countries where the press isn’t truly free — where people only hear one side of the story because the media is subject to repercussions for content published.

I find it ironic that in the U.S. we have so much freedom to chose what we read and so many options of media offering all points of view, and yet pundits and others are saying that we are increasingly choosing to consume news from organizations that support or confirm our existing point of view. Are we self-imposing what is forced on people in less free countries? It’s a similar concern to that voiced by Netflix CEO Reed Hastings, at the recent BBG tech panel, who cautioned that we should consider the downside of personalized content streams.

It’s something I want to be more conscious of — in fact, I’ll be checking multiple news outlets tonight in order to get a more balanced view of the world, starting with the VOA news app that I recently helped test. Speaking of the VOA news apps, I have to brag that our mobile team at ODDI led by Will Sullivan has just been dubbed one of the top five mobile platforms in the world by the GSMA, which means we’re in the running to receive a prestigious award at the Mobile World Congress in Barcelona.

Please comment here if you have any questions or thoughts! We’d love to hear how you do UX/UI testing, especially for products intended for global audiences. You can also find us on Twitter: @BBGinnovate, @ericamalouf and @stevefuchs1.

The following two tabs change content below.
Erica Malouf

Erica Malouf

Erica Malouf manages Marketing for ODDI (and is the communications go-to, copy editor, and UX researcher). She is currently earning her master's at Georgetown University, focusing on communications and marketing. Follow her on Twitter: @ericamalouf or connect via LinkedIn.

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *