Can User Experience Designers be Lean?

October 21st, 2010

Written by

Published on

October 21st, 2010

When I asked Jared Spool (whom I met at Dave McLure’s Warm Gun conference) why so few UX researchers join early stage startups, he replied (a bit tongue in cheek): “Startups are run by entrepreneurs. Entrepreneurs know everything. They don’t listen. A UX researcher would be frustrated.” That made me wonder… is there a class of startups where the entrepreneurs are eager to listen? and how could UX researchers be valuable in a startup?

The answers I found pointed to the lean startup movement.

The lean startup movement

In fact, it appears that a growing number of startups are fitting the bill. These startups adhere to a philosophy of development (recently codified by Eric Ries) known as the lean startup movement. The principles guiding these startups are really very simple:

For a particular entrepreneurial vision, test your hypotheses against reality very, very early. Learn, evolve and repeat.

The Lean Startup mantra should be music to the UX professional’s ears. Steve Krug has long advocated starting testing far earlier than you think makes sense. In fact, chapter 4 in his latest book—which lists excuses that companies tend to use for not testing early—reads almost identically to some of Ries’s posts.

But wait! Before you decide this is a match made in heaven (and run off to join your neighborhood startup) I’d like to clarify the job description. Testing as defined by Krug refers to usability testing in the narrow sense that all UX professionals are familiar with. Testing in the Lean Startup is something quite different; centered around a concept known as the Minimum Viable Product.

Minimum viable product

The term Minimal Viable Product (MVP) is an unfortunate one in my opinion because it invariably conjures up an incorrect image. An MVP is almost never a product nor a single thing as implied by the name. It is the least expensive experiment designed to directly test the most proximate fundamental hypothesis or assumption underpinning the current formulation of the entrepreneur’s vision for the business. When I learned about MVPs, though, it reminded me of the very same process I went through with an earlier idea.

Filltherec.com — a real-life MVP

A few years ago I put an ad in Craigslist to hire a writer and got 746 responses in the span of 8 days. As you can imagine, the logistics requires to rank applicants, collect resumes, log calls, remember who was who, etc. proved to be a nightmare. Being an entrepreneur at heart, I immediately thought “if I’m experiencing this much pain there must be a business opportunity here.”

My business partner and I quickly envisioned a web app to solve the problem. In addition to submitting a resumé, each applicant would fill out a qualification questionnaire. They would be automatically pre-ranked based on their responses. Their resumé, status of their application and all communication with them would be organized and instantly available at all times. You get the picture.

Our vision was underpinned by three fundamental hypotheses:

People experiencing the same pain would be eager to learn about a solution.
They would be eager to sign up and pay $29 – $59 per month for it.
They would not be able to live without it, continue using it, tell others etc.

filltherec.com was built in days, not months.

Rather than building the app we designed prototypes to test the two most proximate assumptions. We wrote a simple tool to gather email addresses of posters of employment ads—folks who were probably in resume inundation hell at that very moment. We threw up a quick website with dummy screenshots, pricing, and sign up. We crafted an email that brought out the pain and offered hope with a link to our fake site. We emailed a statistically significant sample.

In a couple weeks we had vital information about two of our three assumptions without spending 6 to 9 months to create a product. The process was simple: distill our app to its core functionality. In our case the MVP was satisfied with a short email and a mailing list of target clients.

Now, perhaps Jared Spool has never recommended that a client like Amazon create a fake site. But in the case of a two-person startup it is just the kind of guerrilla user research that’s required.

Today’s entrepreneurs desperately need someone who can

Identify the fundamental hypotheses underpinning the current formulation of their vision.
Devise and run cheap, quick, user-driven experiments that prove or disprove their assumptions.

Or in Ries’s terminology: create and test a MVP.

Entrepreneurs know too much

It turns out that even entrepreneurs who are sold on lean are often pretty bad at running these experiments themselves, suffering from what Chip and Dan Heath term The Curse of Knowledge in their excellent book Made to Stick:

Imagine a simple game where subjects are assigned to one of two roles: “tappers” or “listeners.” Tappers received a list of twenty-five well-known songs, such as “Happy Birthday to You” and “The Star Spangled Banner.” Each tapper was asked to pick a song and tap out the rhythm to a listener (by knocking on a table). The listener’s job was to guess the song, based on the rhythm being tapped. (By the way, this experiment is fun to try at home if there’s a good “listener” candidate nearby.) Tappers thought listeners would guess the song right 50% of the time. They actually guessed right 2.5% of the time.

Why?

When a tapper taps, she is hearing the song in her head. Go ahead and try it for yourself — tap out “The Star-Spangled Banner.” It’s impossible to avoid hearing the tune in your head. Meanwhile, the listeners can’t hear that tune — all they can hear is a bunch of disconnected taps, like a kind of bizarre Morse Code.

In the experiment, tappers are flabbergasted at how hard the listeners seem to be working to pick up the tune. Isn’t the song obvious? The tappers’ expressions, when a listener guesses “Happy Birthday to You” for “The Star-Spangled Banner,” are priceless: How could you be so stupid?

It’s hard to be a tapper. The problem is that tappers have been given knowledge (the song title) that makes it impossible for them to imagine what it’s like to lack that knowledge. When they’re tapping, they can’t imagine what it’s like for the listeners to hear isolated taps rather than a song. This is the Curse of Knowledge. Once we know something, we find it hard to imagine what it was like not to know it. Our knowledge has “cursed” us. And it becomes difficult for us to share our knowledge with others, because we can’t readily re-create our listeners’ state of mind.

Entrepreneurs have been hearing the same song playing in their head day and night for a long time. They have so internalized the fundamental assumptions of their vision that they don’t even see them anymore. When they describe their vision they often use a shorthand that simply does not resonate with an ordinary person who isn’t hearing the same tune in their head. They can’t imagine their listeners state of mind. They often have a hyper-inflated notion of what percentage of their audience understands their message as currently formulated. Moreover, they are so accustomed to selling and pitching their idea they cannot help but bias their experiments.

Entrepreneurs don’t research well

In light of this pervading problem, my team created and run a couple of remote usability testing services – TryMyUI and OpenHallway.

At the show we interviewed founders of dozens of startups. They described their vision and we helped them create a remote usability test for the current incarnation of their product. In almost every case we found:

They had a promising vision when we were able to understand it.
Their website did a poor job of conveying that vision.
Founders had trouble writing a simple unbiased usability test for their product.

The entrepreneurs’ mistakes in creating tests in general were:

Worrying about complex use cases and scenarios far down the path when prospects were getting lost at square 1.
Difficulty in articulating the likely initial frame of mind of a potential visitor to their service.
Tendency to lead the witness by using lingo that also appears on their site when writing a usability test.

Invariably in my interviews with founders I would end up saying something like this:

As I understand it your vision is X. When I look at the copy on your site I think you are saying Y. I wouldn’t be surprised if others have the same misunderstanding. Instead of testing L,M,N,O,P lets just see if users correctly recognize your vision from the copy on your site.

And sure enough the test would come back showing that users got lost at square one.

Can trained UXers help?

A usability professional is probably less likely than an entrepreneur to make these mistakes. UX researchers are trained to avoid the curse of knowledge and to conduct unbiased experiments. They know not to lead the witness and should be thoughtful about understanding initial state of mind and crafting unbiased scenarios.

If you are a professional researcher and have gotten this far in this article you might be thinking: I would never make those rookie mistakes. I can design Minimum Viable Products. I’m bored with my high paying consulting gigs. I am an entrepreneur at heart. Why not join a startup.

Well before you pick up the phone to your corner startup you might consider the rest of the job description.

Testing business assumptions isn’t testing usability

First, as you can tell by now a UX consultant in the strictly traditional sense would not be a good fit. While UX professionals love to talk about testing very early, virtually all their experience as consultants is in testing very late. They are accustomed to working with large budgets in a silo using a fixed toolset against a largely finished product. They are accustomed to testing the details of a UI.

You must be adept at testing the fundamental assumptions underpinning a business in a lean, low budget environment potentially using non standard techniques.

Wearing multiple hats

Second, you need to wear more than one hat. You must be both the “hands” and the “brains”. Jared Spool showed a slide at Warm Gun Conference crammed with all the disciplines required in a startup. Obviously the more of these you can cover the more valuable you will be to a startup. One ideal hat to wear would be the ability to build. In a startup it would be very helpful if you could build the “fixtures” for the experiments you design.

If you can’t do rails or html it would at a minimum be helpful if you can prototype.

Sign me up

So at this point in the article I trust I have lost the consultants who are happily married to big companies with large budgets. Hopefully, the hands without brains and the brains without hands have stopped reading as well.

And that leaves you – a super smart, creative, hands on, out of the box thinking UX professional who is an entrepreneur at heart. And you may be wondering:

How do I find a startup with a vision I believe in?
How will they pay me?

Here are some suggestions:

Finding a startup

Go to conferences (ideally about usability) attended by designers and developers at startups that are not attended by the UX community. Warm Gun is a great example.

Ask every founder you meet to describe their vision. They love to do this. We talked to dozens of founders with great vision at Warm Gun until our jaws hurt and only hit a tiny fraction of the startups at the event.

If it is a vision you believe in ask to see the latest formulation of their vision – probably a website. At this point if you are right for the job your head should be spinning with questionable assumptions underpinning the current formulation of their vision and clever ways to test them. Suggest a few.

Now listen carefully. Do you hear an eagerness to test assumptions rather than someone who knows it all? If so then you may have a match made in heaven.

Getting paid

As you probably know startups typically have a surplus of equity and a deficit of cash. One way to skin the cat is to moonlight for equity while you keep your consulting gig to pay the bills—a la 37 signals—until the company can raise enough capital to pay you.

If you are good at validating the underlying assumptions of the business quickly and cheaply your work should go a long way to convincing an intelligent investor (yes they exist) to fund the project. But more importantly your contribution will be essential to making the business a success.

Additional Resources