Earlier this week we shared part one of Steve Portigal and Maish Nichani’s conversation concerning the practice of user research. In the final half of their conversation, Steve asks Maish how he and his firm structure their approach to design research as well as how Maish thinks research might be facilitated in the future.
- Hey, Maish! What are some internal initiatives you’ve used to develop interviewing skills within your own organization?
- PebbleRoad is a small team comprised of “t-shaped” researchers, so when it comes to collaborating on a project we work to ensure that the “horizontal part” of our T’s overlap. For example, everyone needs to know the process and methodology of research interviewing.
We used to think we could rely on our people to carry the knowledge of our research process, but we were wrong. We kept finding interesting stuff we did many years ago that nobody knew about! To establish common ground, we created processes and tools to guide our practice, including:- A toolkit, comprised of guides, templates and forms that frame the practice;
- Artifacts, which are physical things that give quick access to our shared knowledge;
- Resources, such as collections of books and articles for new knowledge; and
- Rituals, which are regular activities to keep the ideas flowing such as sharing sessions called Jolt Fridays!
- Interesting! What’s in the toolkit, then?
- The research toolkit consists of:
- A document detailing the PebbleRoad research process.
- A folder for research methods, such as the KJ technique and Empathy Map.
- A folder for templates, such as an Interview Planning Guide (a guide to help our clients understand the research activities), Audio/Video Recording Consent Form and Interview Highlights (a template to present findings). These are similar to the ones published by usability.gov.
- A folder containing how-tos for research activities, such as How to Write a Research Report and How to Store Research Data.
The toolkit is kept in our Google Drive and can be updated by everyone on the team. It helps establish a common ground upon which further conversations can take place. For example, we can quickly start discussing the research need and plan activities to meet that need.
- I’m assuming the books and articles are physically present in your space?
- Yes, we have a nice collection of 300+ books in our library. We read a lot! Current favourites include Service Design – From Insight to Implementation by Andy Polaine, Lavrans Løvlie & Ben Reason, Thinking Fast and Slow by Daniel Kahneman and Designing the Search Experience by Tony-Russell Rose and Tyler Tate. Our entire catalog is on LibraryThing!
- Could you give me an example of an artifact? Where does something like that live?
- In one of your webinars, Steve, you outlined the different types of interview questions one could ask and what they are used for. We found that list to be very useful when writing the interview planning guide, so we created an interview-types keychain!
This is an example of an artifact. When we want to use it, we open the ring and spread all the cards on the table. We then discuss which types of questions will be most useful for the interview. When we’re done, we put the cards back on the ring. In fact, for those who want to create their own version, download the PDF or Indesign file.
- Okay, so, I’ve got to know: what happens on Jolt Fridays? It sounds dramatic!
- Jolt sessions typically happen on a Friday morning and anyone on the team can volunteer to host one. The host arranges for breakfast, chooses a topic, and researches it beforehand. We’ve had Jolt sessions on topics like How to Listen, Digital Marketing, Gamification and, yes, Qualitative Data Analysis. We also have screenings such as the documentary Art & Copy: Inside Advertising’s Creative Revolution. The big idea is to widen the organization’s collective thinking – to give it a jolt!
- In the past you mentioned that you were developing an app to facilitate user research. What is the app? Who uses it and for what?
- A few years ago we spent almost half a year developing a research data management app called Insightico. Sadly, we “archived” the project late last year.
The idea was to collect and store research data in chunks. Researchers could import audio and video recordings, pdf files, ppt templates and other types of files into the app. These files could then be “chunked” up; for example, a video file could be broken up into individual segments and tagged. The big idea was that if we had chunked and tagged data from all sources we’d be in a better position to find patterns. Also, since chunks exist independently we could mix and mash up chunks from different projects for a session on crazy connections!
The app was hosted in the cloud and expensive to maintain. We did this without any external investment. It was our pet project.
Very quickly we learned that ideation is an “all-at-once” activity. Researchers are more effective when we can “see” all findings at the same time, not linearly like a blog post, which is how we presented it in the application. We know of only two ways of doing it today: Post-It Notes on a wall and spreadsheets!
The app design team moved on with their projects and the development team disbanded so we had no other option but to “archive it”. I say “archive,” not “kill,” because I still think the idea has merit. If we can get a good team in place we might just have a go at it again!
- But it sounds like the technology had a limitation in that it didn’t support the “all-at-once” way of engaging with the data? Is that something you would design around if came back to the project?
- Yes. Even the pro research apps, like MaxQDA, are trying to get more visualization capabilities, but they’re not there yet. I think the problem is screen size. Unless we get something larger where many people can simultaneously interact with (multi-touch, multi-user), the physical wall with yellow stickies is going to win. Collaborative ideation needs something like what Jeff Han demoed on TED a few years ago!
- You’ve explored creating very analog and very digital tools to support the research process. What do you think the tradeoffs are in the different approaches?
- Digital is fantastic for capturing data and analytics, but when it comes to inference we need to take it out of the computer, as Jon Kolko advises.
My biggest peeve is that we should be able to reuse findings across projects. This is where digital can really help but only if we’re strict about collecting it the right way. Vijay Kumar’s recent book, 101 Design Methods, talks of a “User Observations Database” that students can refer to when doing background scans. This database contains results of observation studies across different projects. This is the kind of reuse I’m interested in and it seems to be something for which digital is extremely well suited.
- If you could wave a magic wand and create any kind of tool or artifact to support the research process, what would it be?
- Research is really all about creating new knowledge and the more people who have access to that knowledge, the better. Currently, our research findings and insights are all locked up in (what Karen McGrane calls) “blobs.” We need it, instead, to be “chunked” (as Sara Wachter-Boettcher says) so that it can travel more freely and be mixed and mashed up to create, again, new knowledge. I don’t know of any existing project or initiative but I was thinking about using a schema (like what is already available on schema.org) for research findings. That way anyone writing up research findings could use the same markup and then search engines and specialist apps could read and move those findings more efficiently.
- A while back, I asked my team to avoid actively discussing examples from a previous project while synthesizing a current one because it felt awkward. While our clients hire us because of the benefit of our expertise, is prior “data” off limits? Every time I hear about sharing across projects, I think about what might be legal or ethical issues. What do you think is our obligation to keep our data and our insights from a project to just that project?
- The main objective of research is to point to a design direction or decision. The multitude of these decisions leads to a design solution, in our case websites, intranets or apps. If we keep this frame, then the criss-crossing of findings across projects becomes a necessity. As an example, we recently did a project on the job market in Singapore. One of the core findings was a deep sense of “entitlement” of young Singaporean jobseekers. The reasons for this attitude are “macro” in nature—recent economic success, changing culture, political freedom, etc.
Now, our latest project centers on workforce productivity. It would be a disservice to ignore a possible causality between the entitlement mindset and a productive workforce. This is the type of reuse I’m referring to; not about identifiable data, but broad strokes of understanding.
And that’s a wrap! Thanks, again, Steve and Maish; we’re always delighted to listen in.
Considering that the interview ended on such an ethical conundrum, I’m curious what readers think: is sharing research data a good thing or a bad thing? What about knowledge, the flashes of understanding we get as researchers? Finally, if anyone knows about a schema for structuring research findings, please share it with Maish in the comments, below.