Interview with the Authors of Remote Research

Research is what Bolt | Peters does best. As a digital research firm, their job is to uncover and determine the ways in which real users use real products with their “natural” environments. In order to achieve the most accurate data possible, researchers at Bolt | Peters employ a broad variety of research methods. One method in particular, known as Remote Research, has garnered considerable attention.

The book cover for Remote Research, published by Rosenfeld Media.

In their forthcoming new book, authors Nate Bolt and Tony Tulathimutte address the questions they receive while practicing Remote Research. From the mundane, “what is remote research,” to the more intriguing “how does it compare with more conventional design research practices?” their book promises to cover it all.

In an odd turn of events, we at the UX Booth found ourselves with the opportunity to sit down with these esteemed researchers and give them a dose of their own medicine, asking questions of our own. In the following interview, learn what design research means to Nate and Tony and even how you can get started. If that’s not enough incentive to employ research within your own design practice, we’re even giving away some copies of the soon-to-be-released book. Enjoy!

I would hazard a guess that most of our readers who are interested in experience design (and remote research), aren’t in a position or at a company where it’s treated as a first-class citizen. Although the question is frequently posed, I have to ask: how do you suggest these people introduce remote research into their product-development process?

Add a couple of informal UX research sessions into your next design project, whether or not design research is specifically included in the budget or timeline. Don’t ask anyone if they think it’s a good idea, just do it. It only takes 2-3 hours, and costs nothing. All the standard remote UX tools have free trials, and you can grab a screen sharing tool like GoToMeeting or Acrobat Connect, a screen recording program like Quicktime 10 or iShowUHD, and you’re good to go. Then edit the recording in iMovie or another free video editor. You can also use an in-person tool like Silverback.

When you show photos, user quotes, or quick video highlights to your stakeholders, it will blow them away if they weren’t expecting it, and it can all be done in roughly the amount of time we each spend on Facebook and Twitter every day.

Can you briefly explain the idea behind time-aware research and “live recruiting?”

In traditional user research, you ask someone in advance to come in to a boring usability lab and pretend to care about the interface or design you’re researching. With time-aware research, you intercept them live with a web tool like ethnio or a mobile-based form, and ask them to participate in a phone interview just as they’re about to use an interface, on their own initiative and timeline. What’s important is that you’re capturing someone right at the moment when they care about what they’re doing, whether it’s making a donation, buying a phone, purchasing enterprise software, etc. The criteria by which they make click decisions are totally different when they really need to book travel that minute.

A great example of a time-aware research project is the popular Track Your Happiness project at Harvard, which uses SMS messages to intercept people and answer questions about their happiness into a mobile form. The happiness data is reported in “real-time,” which provides far more accurate data than people filling out some survey about how happy they were last month. Basically, it eliminates our misrepresentations about interactions by peeking into real-life behavior and time.

Do you find that you obtain better results presenting/observing reactions to early sketches or wireframes vs. functional prototypes or fully-functioning applications/products?

The answer to this question is actually a funny comment on the current state of the web. We’ve tested 2,300 participants over ten years, with at least 40% of those sessions involving a prototype, a scanned napkin sketch, static JPGs, or wireframes. We never use the word “prototype,” and almost always, after we redirect people to a static wireframe, they say “yeah, this looks like other sites I’ve used.” In other words, even with a crap sketch, as long as it clicks through to another crap sketch, people can relate that to their normal browsing experience. That’s how crappy parts of the web are. People aren’t shocked by a black and white version where only one thing works.

That said, the more functional interactions our participants have, the more insight we get, so the results do get incrementally better with more fidelity, but testing sketches is still incredibly useful.

In today’s web application space, two terms are hot topics: “agile development,” and “user experience design.” How do you see research (in general) keeping step in a fast-paced, agile development environment? Applications that are in-development simply don’t have the users with which to conduct time-aware research, so I wanted to ask for your thoughts on how prudent experience/interaction designers can proceed in this “pressure-cooker” situation. (I’m aware of—and eagerly anticipate reading—Anders Ramsay’s book on Agile Experience Design)

Go guerilla. Andy Budd from ClearLeft loves this approach and so do we. The philosophy is: instead of forgoing testing, or trying to get it approved, just grab someone whose paycheck doesn’t involve your design or development, and watch them interact. The same goes for remote testing—you don’t even have to leave your chair to observe someone’s functional experience with an interface.

We did a recent study for Mint.com, who don’t practice Agile but do release very rapidly, and they made changes on the fly based on what the designers and developers were seeing in remote testing. We conducted it from a conference room where it was easy for their team to come observe, minimizing the gap between developers and designers and “the research.” They just saw UX issues and addressed them.

Research is seen primarily as an academic pursuit: how did you get started and how do you suggest others do the same?

Everyone here at Bolt | Peters studied some variation of Cognitive Science, Linguistics, or Human-Computer Interaction. I worked as a design intern pushing pixels for a web development company in 1998 called Clear Ink, and I convinced them to let me pitch usability to their clients, since everyone was listening to punk 21 yo. kids back then. Eventually, one of the clients thought it sounded like a good idea and our first study yielded designs that got a 150% conversion rate increase and millions of dollars more in web sales. Then it was off and running.

So the steps I’d suggest are:

• Take a class or two in a related field.
• Intern somewhere that conducts UX and IxD.
• Just do research without asking.
• Measure results and highlight the awesomeness.

That’s all, folks!

Thanks again to Nate and Tony for taking the time to answer our questions and share their thoughts with the community. Remote Research is an exciting, emerging methodology which increasingly appeals to contemporary design endeavors.

From what I’ve read thus far, it’s going to be an awesome book. So good, in fact, I would recommend purchasing two copies: one to keep and one to share—which, come to think of it …

Want to win a copy?

In case you hadn’t noticed, Rosenfeld Media is one of the primary sponsors of UX Booth this quarter. As a consequence, we’ve got 3 copies of Remote Research to give away to our community. To enter, simply leave a comment below and make sure to include your twitter ID. Next, be sure to follow us (@UXBooth) on twitter. If you’re selected, we’ll be in touch via a Twitter DM to obtain your mailing address. Thanks, and good luck!

Rather just purchase one?

Readers of UX Booth are entitled to a 15% discount off any of the fine products found at Rosenfeld Media. Simply enter code UXBOOTH at checkout.

About the Authors

Additional Resources

• Nate Bolt’s Presentation “Remote Design Research” , presented at Interaction ’10 Conference
• What is Remote Usability Research?
• A Listing of Remote Usability Tools, compiled by Bolt | Peters