Innovation @ BBG » Research http://www.innovation-series.com Fri, 20 Nov 2015 18:47:05 +0000 en-US hourly 1 http://wordpress.org/?v=3.6.1 Usability Research aims to create a better Radio Free Asia website http://www.innovation-series.com/2015/11/20/usability-research-aims-to-create-a-better-radio-free-asia-website/?utm_source=rss&utm_medium=rss&utm_campaign=usability-research-aims-to-create-a-better-radio-free-asia-website http://www.innovation-series.com/2015/11/20/usability-research-aims-to-create-a-better-radio-free-asia-website/#comments Fri, 20 Nov 2015 16:07:16 +0000 Xi Rotmil http://www.innovation-series.com/?p=6589

Every time a person has a great experience with a website, a web app, a gadget, or a service, it’s because a creative team made crucial decisions about both design and implementation—decisions based on data about how people interact with a computer interface.

During August, September and October of this year ODDI and Radio Free Asia collaborated on an in depth user experience review of the the RFA desktop and mobile websites.

Radio Free Asia broadcasts domestic news and information of specific interest to its listeners in China, Tibet, North Korea, Vietnam, Cambodia, Laos, and Burma.  All broadcasts are solely in local language(s) and dialects.

Remote testing sets the stage.

ODDI used the CrazyEgg platform to get an overview of user behaviors during the month of August. 15 pages were followed for a total of 150,000 impressions. This allowed us to see where people were clicking, and where they were not. We also got an idea of how many people scroll down the pages and where most people stop. Finally testing allowed us to see where those users are coming from to begin with, and who clicks on what the most!

Remote testing gave us an overview of users interaction with the pages, and where some follow up with in-person testing might be useful.

From there ODDI:

DEVELOPED TEST PLAN

We sat down with the RFA team and agreed on the test objectives, the questions used in the test, and characteristics of the people who will be trying out the design.

CHOSE A TESTING ENVIRONMENT

Radio Free Europe provided an excellent partitioned room. Video and audio was delivered from the testing room to the observer’s room via network connections.

FIND AND SELECT PARTICIPANTS

The best place to perform these kinds of tests would be in the target countries. Since travel and recruitment would be prohibitively expensive, we sought out English as Second Language students at local universities.

Most of our participants did not know anything about the site prior to the test, and we are grateful for their fresh and valuable insights.

We recruited six participants to test the Chinese Web site and six participants to test the Chinese Mobile site: they were screened to be:

·         Native Chinese speakers

·         Very active news seekers when in China – especially those who visited blocked sites

·         Particularly interested in sensitive Chinese domestic news

·         All under 30

We also recruited six participants to test the Vietnamese Web site and six participants to test the Vietnamese Mobile site: they were screened to be:

·         Native Vietnamese speakers

·         Very active news seekers when in Vietnam ­‑ especially those who visited blocked sites

·         Particularly interested in sensitive Vietnamese domestic news

·         All under 30

The final group, while adhering to the screening parameters mentioned were an interesting mix of backgrounds, including students majoring in electrical engineering, environment science, computer science, applied math and information technology, who also displayed a range of feelings and reactions to the website.

PREPARED TEST MATERIALS

In the test materials, we included specific background and warm up questions to ask, prompts for follow-up questions, tasks, as well as closing, debriefing questions that we want to ask each participant and an evaluation survey.

CONDUCTING THE SESSIONS

rev-test1

Each session was videotaped with one camera attached to the phone to record the user’s taps and gestures while a second one was focused on the user’s facial expressions. Observers in a separate room watched the live video feed and took notes.

We used software called Morae for in-house UX testing on tablets and mobile phones. Morae allows us to capture video — with more than one camera angle — and record scoring as we go. Having video of a participant’s hand movements allowed us to do a more accurate and thorough analysis of how they reacted during certain tasks. Also, since it was in-person, we asked follow-up questions immediately after a task to find out why a participant might have been confused about a task.

We also had a team of people from Radio Free Asia, who were observing the tests in a separate room and participated in the test by asking questions through Morae’s chat window at the end of each session.

The tests consisted of a detailed hour-long interview in English with a subject using his or her phone. After a short introduction, the user was asked to perform 9 tasks on the RFA mobile site. These questions and tasks were videotaped and timed (through Morae) to assess the ease with which the user could interact with the mobile site.

Participants were told they would be videotaped and asked to sign a photo release.

After an initial introduction and discussion of web news, each participant was read a set of instructions. The tasks were given to each participant one at a time on separate sheets of paper.  He or she was asked to read each task out loud before attempting to interact with the website. Mobile users were asked to bring their own phones and used them in the test.

The tests were administered in English, but each participant engaged with the website in their native language.

A native speaker in Mandarin or Vietnamese was on hand if the participant had trouble putting his or her views into English. About half the participants took advantage of this option. Some particularly taciturn participants were debriefed in their native language to ensure the test team was getting all of the results and not suffering from a language gap.

Participants were not coached by the moderator. When something did not go well, they were asked to assess the website and offer advice on how the user experience could be improved.

Occasionally at the end of a task the moderator revealed what should have happened, and asked the participant how the website could be improved.

DEBRIEF WITH PARTICIPANTS AND OBSERVERS

rev-survey

At the end of each session, the moderator asked: “How’d that go?” Also, we invited observers from RFA to pass follow-up questions to the moderator or to ask questions themselves. We also prepared an evaluation survey for participants to fill out.

ANALYZE DATA AND WRITE UP FINDINGS

When we looked at those observations after the test, the weight of evidence helped us examine why particular things happened. From that examination, we developed theories about the causes of frustrations and problems. After we generated these theories, RFA team members can later use their expertise to determine how to fix design problems.

OUR FINDINGS AND SUGGESTIONS

The quality of design is an indicator of credibility. 

Our overall suggestion is to refine and redesign the site. Three users mentioned that RFA’s website looked like a blog or Facebook page, and they doubted its trustworthiness for that reason. Our tests show that elements such as layout, consistency, typography, color and style all affect how users perceive a website.

In addition, the RFA design is three years old and needs to be updated. Among the changes that research suggested:

  • Refine typography and visual hierarchy to be easier to read for mobile first, since this is the most challenging device to design for.
  • Add timestamp to news articles.
  • Create a shorter page, heatmaps show 50% of users are only viewing 25% of the current page.
  • Icons and text do not have sufficient touch/clear area for touch screens on smart phones.
]]>
http://www.innovation-series.com/2015/11/20/usability-research-aims-to-create-a-better-radio-free-asia-website/feed/ 0
Radio Sawa Arabic music streaming app launches on Android & Apple iOS http://www.innovation-series.com/2014/06/30/radio-sawa-arabic-music-streaming-app-launches-on-android-apple-ios/?utm_source=rss&utm_medium=rss&utm_campaign=radio-sawa-arabic-music-streaming-app-launches-on-android-apple-ios http://www.innovation-series.com/2014/06/30/radio-sawa-arabic-music-streaming-app-launches-on-android-apple-ios/#comments Mon, 30 Jun 2014 11:34:47 +0000 Will Sullivan http://www.innovation-series.com/?p=6063 Seven 24/7 streams of Arabic music from around the Middle East and North Africa region is featured in the Radio Sawa app.

Seven 24/7 streams of Arabic music are featured in the Radio Sawa app.

When Radio Sawa launched as a network of FM radio stations across the Middle East in 2002, the Arab world had not really experienced anything like it as many countries had state-controlled media environments. Radio Sawa provided an alternative, with a mix of Arab and Western popular music coupled with newscasts and social issue programs.

Radio SawaTwelve years later, Radio Sawa has a lot of competition on the radio dial and has limited radio reach in countries like Saudi Arabia and Egypt because authorities have not permitted FM licenses. To evolve and engage new audiences in the digital age, Radio Sawa needed a new platform to reach closed markets and serve a new generation of listeners; that opportunity lay in designing an innovative audio mobile app for Arab youth.

So the Office of Digital & Design Innovation and the Middle East Broadcasting Networks joined forces to design and build an app unlike any in the Arab market. We focused on building a gorgeously design, touch-driven, mobile-only application which played to the huge popularity of streaming radio on smartphone devices. Streaming radio ranks higher than any other content format as the most popular form factor according to a comScore’s Multi-Platform and Mobile Metrix report released this year:

Streaming radio use on digital devices

Streaming radio use on digital devicesThe application is an audio-first experience, with newscasts and featured podcasts from Sawa’s journalists available on demand. But the crown jewels of the experience are the seven Arabic live-streaming radio stations from around the Middle East and North Africa region. Arab youth can listen to music and news 24/7 and in the background while they read, play or use other applications.

The Sawa Chat feature allows audiences to record and send their thoughts to our programs instantly.

The Sawa Chat feature allows audiences to record and send their thoughts to our programs instantly.

Another fantastic feature of the app is our expanded Sawa Chat functionality. For years, Sawa journalists have asked people on the street questions about social concerns. With the app, users can now respond to weekly questions by recording and uploading their thoughts and sending them to our producers in a couple quick taps. These submissions will feed expanded Sawa Chat content on the air and on the app.

Quick sports, tech and pop culture news is featured in the application.

Quick sports, tech and pop culture news is featured in the application.

The app also showcases articles from Radio Sawa’s digital journalists with six news feeds rich in original journalism, music news and sports. It also showcases Sawa’s innovative Have You Read This? youth-focused journalism which builds stories around trending ideas and conversations spreading on social media.

Check out the Radio Sawa app now, it’s live in the app stores and available for mobile devices running Android 4.0 and above and Apple IOS 6 and above. We’d love to hear your feedback and if you dig it, please give us a review and rating and share it with your friends!

Davin Hutchins, Editorial Director of MBN Digital, and his rock star team collaborated on this blog post and the Radio Sawa design, development and launch.

]]>
http://www.innovation-series.com/2014/06/30/radio-sawa-arabic-music-streaming-app-launches-on-android-apple-ios/feed/ 0
How to Do UX Testing on Tablets, In Cartoons http://www.innovation-series.com/2014/01/27/how-to-do-ux-testing-on-tablets-in-cartoons/?utm_source=rss&utm_medium=rss&utm_campaign=how-to-do-ux-testing-on-tablets-in-cartoons http://www.innovation-series.com/2014/01/27/how-to-do-ux-testing-on-tablets-in-cartoons/#comments Mon, 27 Jan 2014 16:01:33 +0000 Erica Malouf http://www.innovation-series.com/?p=5682 One of my great passions is traveling — I get a kick out of understanding other cultures, and how they use technology. As a graduate student, I am rarely able to travel, but working at the BBG is truly the next best thing.

We frequently interact with people from a broad array of cultural backgrounds, with fluency in various languages, who each share their unique perspective and offer a fascinating glimpse of a distant place. (The holiday parties and hallway networking spreads offer a tasty glimpse of distant places — BBG staff can really cook.) This cultural diversity also happens to be convenient for user experience testing because we are able to tap into our ‘natural resource’ to test our digital products, which are often targeted to a specific segment of our global audience.

Typically, when we’re doing in-house testing, we test each application in it’s intended language as well as in English, whether it’s a product for one of the 44 language services at Voice of America or another BBG entity. Most recently we have done testing for Radio Free Asia and Radio Free Europe/Radio Liberty (RFE/RL).

The cultural diversity within BBG is convenient for user experience testing because we are able to tap into this ‘natural resource’ to test our digital products.

And so we need bilingual participants, who also fall into our target demographics, to provide our English-speaking developers, designers and testers with feedback we can understand. Thankfully, our gracious multi-lingual employees and interns are happy to oblige.

We’ve been using software called Morae for in-house UX testing on tablets and mobile phones. Morae allows us to capture video — with more than one camera angle — and record scoring as we go. Having video of a participant’s hand movements allows us to do a more accurate and thorough analysis of how they reacted during certain tasks, as opposed to just having a heat map and data that provide only part of the story. Also, since it’s in-person, we can ask follow-up questions immediately after a task to find out why a participant may have been confused about a task.

This comic strip breaks down how we do UX testing. Hat tip to the illustrious Steve Fuchs (one of our lead UX testers and designers at ODDI) who illustrated these, and who I often work with on UX testing.

1-stake-holders-managers

2-3-design-book-volunteers

4-tv-tray-plus-chair

5-6-table-top-mic

7-two-cameras-record

8-laptop-software

9-test-team

10-collect-analysis-report

One of the more interesting UX tests that I worked on was for RFE/RL mobile websites, which we tested in English and in various languages common in Russia and nearby. Without knowing anything about the site, several of our participants commented right away that it must not be from a Russia-based news network solely from the editorial choices, referring to the top stories on the home page. I recall one participant commenting that no Russian news outlet would dare have the Pussy Riot story front and center, if they reported on it all, because in-country news typically paints the Kremlin in a positive light.

Screen shot 2014-02-03 at 10.01.22 AM

In addition to a responsive website, RFE/RL now offers a mobile app for Android and iOS.

Comments like those remind me why it’s so critical to provide an alternate point of view for people in countries where the press isn’t truly free — where people only hear one side of the story because the media is subject to repercussions for content published.

I find it ironic that in the U.S. we have so much freedom to chose what we read and so many options of media offering all points of view, and yet pundits and others are saying that we are increasingly choosing to consume news from organizations that support or confirm our existing point of view. Are we self-imposing what is forced on people in less free countries? It’s a similar concern to that voiced by Netflix CEO Reed Hastings, at the recent BBG tech panel, who cautioned that we should consider the downside of personalized content streams.

It’s something I want to be more conscious of — in fact, I’ll be checking multiple news outlets tonight in order to get a more balanced view of the world, starting with the VOA news app that I recently helped test. Speaking of the VOA news apps, I have to brag that our mobile team at ODDI led by Will Sullivan has just been dubbed one of the top five mobile platforms in the world by the GSMA, which means we’re in the running to receive a prestigious award at the Mobile World Congress in Barcelona.

Please comment here if you have any questions or thoughts! We’d love to hear how you do UX/UI testing, especially for products intended for global audiences. You can also find us on Twitter: @BBGinnovate, @ericamalouf and @stevefuchs1.

]]>
http://www.innovation-series.com/2014/01/27/how-to-do-ux-testing-on-tablets-in-cartoons/feed/ 0
ODDI Demo Day Kicks Off the New Year http://www.innovation-series.com/2014/01/13/oddi-demo-day-kicks-off-the-new-year/?utm_source=rss&utm_medium=rss&utm_campaign=oddi-demo-day-kicks-off-the-new-year http://www.innovation-series.com/2014/01/13/oddi-demo-day-kicks-off-the-new-year/#comments Mon, 13 Jan 2014 18:41:16 +0000 Erica Malouf http://www.innovation-series.com/?p=5605 Friday marked the last demo day at ODDI in the current format. In the past, project owners (team leaders) have given demo day presentations in ODDI’s office with an occasional note from a team member. From here on out, the emphasis will be on having the team members take the lead in talking about their work, instead of the project owners.

In the past, stakeholders from within BBG have always been invited, but have rarely joined–it’s usually just ODDI staff who attend the Friday demos. Going forward, ODDI demo days will be centered around the stakeholder. Teams will schedule time with their various project stakeholders within BBG. The goal is to get internal customer feedback on a more regular basis as a part of our Agile, iterative approach.

ODDI scrum master Son Tran says that team-driven presentations provide team members with an opportunity to show that they are delivering on goals and owning the work they’ve done. He also notes it’s about the iterative process:  ”Closing the feedback loop and making it shorter is better for improving projects.”

What are we working on at ODDI?

For most teams, Sprint Zero was a time of research and planning, defining goals and determining KPIs. Adam Martin, our newly minted Director of Innovation, asked teams to come to the January 9 demo day “prepared to discuss their Charter as described in the Strategic White Paper, their shared vision in response to the Charter, the team’s goals, how they will measure their success against those goals, and their product(s) roadmap for Q2 of FY14 (and beyond if available).”

Now the teams are ready to see their brilliant ideas into fruition. And some teams are also managing ongoing projects like Relay, RIVR, the BBG-wide analytics roll out, and mobile app updates.

Here’s a look at what’s happening:

image mobileprez

Will Sullivan presents on the latest mobile app updates and the Symbian launch.

MOBILE TEAM

Project Owner: Will Sullivan

The Mobile Team is continuing develop, update and support the suite of umbrella news applications for all BBG entities, which supports more than 82 language services now, and has an install base of more than 400,000 users. We are launching new applications with Radio Free Asia (RFA) on Google Android and Apple IOS for both mobile and tablet form factors and just launched VOA’s Africa-focused Symbian application (the third largest mobile OS in the region, after Android and IOS, which we launched for VOA services last year). This quarter we will be updating the entire suite to a more magazine-style iPad design, building new Android home screen news widgets and moving the app analytics over to the shared Adobe Omniture SiteCatalyst system. We’re also beginning work on an live audio streaming and on-demand podcast Android and IOS application for the Middle East Broadcast Network’s Radio Sawa that is visually-rich with a touch-centered interaction experience and deep user-generated and social sharing integration.

AFFILIATE DIGITAL SERVICES TEAM

Project Owner: Doug Zabransky

The Affiliate Digital Services (ADS) team represents a new chapter for USIM and affiliate relationships. Existing and new BBG Affiliates will be offered up to three tiers of digital service. Each tier represents levels of digital-hosted offerings including live streaming, adaptive html 5 digital players, and an internet broadcast station which will allow for content source switching between BBG live and on-demand content, as well as other affiliate content within the ADS community. All tiers include customer service and support.

Essentially, BBG hopes to build a robust network of affiliate partner on-line stations. Growing the BBG affiliate digital audience will grow BBG’s audience as well.

DATA-DRIVEN INSIGHTS TEAM

Project Owner: Rebecca Shakespeare

The insights team is focusing on setting up tools that collect and present objective information about digital performance to inform BBG leadership and editorial about what is actually happening with their digital products and content. The team is currently focusing on the rollout of the new web analytics tool which measures digital properties owned and hosted by the BBG. It is also contracting outside validation of the numbers that are collected and reported to ensure accuracy of the information presented. Beginning in February 2014, the team will start to focus on displaying weekly performance analytics from BBG’s range of digital reporting tools, side-by-side in a dashboard, to present a complete picture of digital performance.

 

Screen shot 2014-01-13 at 12.30.53 PM

Brian Williamson’s illustrations illuminate the Storytelling Team’s vision

STORYTELLING TEAM

Project Owners: Steve Fuchs, Randy Abramson

The storytelling team is determined to revitalize and update USIM storytelling around the globe. We are brainstorming innovative ways to tell stories that inform, engage and connect with audiences based on their needs and expectations. One of our main goals is to build community engagement with younger audiences by using a toolbox of highly relevant, visual, trans-media storytelling techniques. We plan to not only count standard metrics–such as time spent, return visits, videos watched, social engagement, and so forth–but we also aim to make a real-world impact that affects conversation and behavior. Randy will continue to work on Relay, and the entire team will work on projects like finding innovative ways to cover sports in developing countries, among others.

Other Teams & Projects

In addition to the teams that demo’d last Friday, ODDI also has several other teams that are kicking A and taking names.

The Research & Analysis (R&A) team functions as support for all other teams. R&A was recently pivotal in helping the Storytelling team and the Affiliate Digital Services team determine their next projects. During Sprint Zero, the R&A team dug deep to find data on countries around the world, interviewing internal experts and BBG’s Regional Marketing Officers, diving into BBG research reports and library databases, and translating that data into insights and strategic recommendations. The R&A team includes Son Tran, Ashley Wellman, Yousef Kokcha, Ahran Lee and myself (Erica Malouf).

image RIVR screen

Ongoing Project: Doug Zabransky will continue to lead the IVR project called RIVR. Look for a blog post update to come soon.

ODDI also has various teams working on ebooks, UX testing and more. Follow the action here on the blog, on Twitter (@BBGinnovate) and on our new website portal (http://oddi.bbg.gov/).

]]>
http://www.innovation-series.com/2014/01/13/oddi-demo-day-kicks-off-the-new-year/feed/ 1