Picking the Right Tool for your Remote User Testing

Image courtesy Svadilfari

With an ever-expanding utility belt of user research tools, finding the right one for your needs and getting the most out of it can be daunting.

The last couple of years have seen explosive growth in remote user testing. What used to cost thousands and take weeks you can now set up in a couple of minutes for under 10 dollars. As a result, the number of people testing has steadily been climbing, especially among those from marketing, design and business backgrounds. While it’s great to see so many people testing, those with little-to-no experience running user tests often don’t get the most out of their tests.

For those starting out with user testing, this article aims to provide an overview of the types of tools available, common use cases, and potential pitfalls to watch out for. The information should make it easier to find exactly the right tool to help you achieve your testing goals.

Information architecture analysis tools

image courtesy CannedTuna

Information architecture forms the foundation of your website, so it’s something that you want to get right. Tools that help you assess your architecture can produce both quantitative and qualitative results.

With card sorting, users are asked to create logical groupings of elements. This type of tool produces qualitative results and works best when no information architecture has been put into place.

Tree testing, on the other hand, helps assess existing information architecture. It works by asking users to find specific sections by navigating a basic node tree. The results produced are often qualitative, such as number of clicks taken.

Your testers are an important consideration with tests of this type. Technical users, for example, may group content differently than those less technical. Testing with a representative cross-section of your audience ensures that you avoid biased results.

Tools in this category include: PlainFrame, OptimalSort, TreeJack, WebSort and Buy-A-Feature

Click analysis tools

Click tests are very popular, and there literally dozens of apps available for testing both live sites and mock-ups. This type of test works by presenting users with an interface and a task to carry out. Users are then instructed to click on the interface in order to successfully complete the task.

This type of test works best with interfaces that have a clearly defined purpose. Ideally these are either landing pages, or pages that contain calls to action (eg. purchasing an item or making a donation). Testing home pages that have multiple valid paths often yields poor results. Similarly, it’s best to avoid using click analysis on pages with forms where problems with understanding content may result in users abandoning the test without clicking.

Tools in this category include: Usabilla, IntuitionHQ, Clicktest, and ChalkMark.

Content analysis tools

This type of test helps you assess how well a user understands the purpose of a web page or interface. These tests often produce results that are more qualitative in nature, and while they can take longer to properly analyze, they’re well worth the effort.

They work by asking users to complete a task or present a scenario for why they are viewing an interface. Users then either make observations or carry out an interaction, after which they describe what they saw or did.

image courtesy Alpha Six

These tools are excellent for soliciting feedback and gauging how clear your content is. With the addition of screen recording, as found in UserTesting, it is possible to use these tools to test workflows as well. Keep in mind that simple instructions tend to yield better results; asking users to perform overly complicated tasks can quickly lead to frustration and abandoned tests.

Tools in this category include: Feedback Army, UserTesting, and Fivesecondtest.

Conversion analysis tools

image courtesy kevinzhengli

Conversion analysis tests usually involve getting the user to complete a process involving one or more steps. Funnels come into play when there are two or more steps. As users carry out the task, their clicks and associated metrics are recorded for later analysis.

This type of tool is most effective for testing a well-defined process such as registrations and purchases. It is important to carefully select the starting point for the test and to focus on the process at hand. For instance, a purchase process begins at a product or product listing page, therefore your users should be shown those screens when commencing the test.

Tools in this category include: Loop11 and Navflow.

General rules of thumb

• Test with real users as early as possible.
• Test iteratively with small groups; Jakob Nielsen makes the following recommendations:
  • 5 users for qualitative tests
  • 20 users for quantitative tests
  • 15 users for card sorting
• Use different testers on successive iterations.
• Testing should be a continuous process and occur during all stages of development.

Hopefully this article has given you insight into some of the usability testing tools available and helps you get the most out of your future testing sessions.

Related material

• Usability Testing: Don’t Guess, Test by Jacob Creech
• Information Gathering: A Roundup of UX Applications by James Costa