In Search of Search: Discussing Analytics with Lou Rosenfeld

Aside from being online, nearly every website is a unique endeavor. Likewise, its content, its structure, and its users, too, are all markedly different. For this reason, those of us who design websites rely on pattens to provide a consistent experience. One pattern that’s changed relatively little – at least from a user’s perspective – is search. Sure, Apple recently broke new ground with Siri, but the premise remains the same: queries and results come out.

Book Cover for Search Analytics for Your Site: Conversations with Your Customers, published by Rosenfeld Media.

Simple, right?

Simple, that is, unless you ask Information Architect Lou Rosenfeld. In his new book, Search Analytics for Your Site: Conversations with your customers, Lou argues that we could all benefit from taking a closer look at that deceptive little box.

Search queries are gold: they are real data that show us exactly what users are searching for in their own words. This book shows you how to use search analytics to carry on a conversation with your customers: listen to and understand their needs, and improve your content, navigation and search performance to meet those needs.

Conversations, indeed! In celebration of the book’s release, we asked Lou for a dose of the SSA medicine. But instead of conducting the traditional interview, Lou suggested we provide some raw search data from UX Booth and analyze the results, together. Eagerly, we agreed.

Read on for some insights into the search logs of UX Booth based on simple techniques that you can start employing today. Finally, for those of you who like to dig deeper, you can even win a copy of the book just by sharing the results you garner from analyzing your own websites. See all the details, below.

Hey Lou! Thanks so much for taking the time to talk. Before the interview, we sent over the top ten queries made on UX Booth. Do you mind telling us what you make of them?

Hey, Andrew, yes; thanks for having me and thanks for sending these my way.

To start, it’s worth mentioning that I do know a little about you and what you do, and I also know a little about analyzing search queries. But really, none of this is required to get started. The biggest thing is to just get your hands on some queries, and just look at them; you don’t need any kind of special training.

In the past three months, it looks like your highest search terms were “mobile,” then “home page,” “e-commerce,” “forms,” and “navigation.” Usually what you see is a very steep drop off as you move from the most frequent [query] on down. So we see a short head and a sort of hockey-stick-shaped curve — that’s called a Zipf distribution — that levels out into a long tail. “Mobile,” for example, may be 2 or 3 times more common than “home page.” We all know mobile is big. I don’t think anyone is going to be very surprised by that.

One interpretation of the Zipf Curve, via Eva-Lotta Lamm

What might be interesting, though, is to look at variations of mobile—synonymous— and see how they compare. For instance, when we talk about mobile in the context of phones, we also use the word “cell.” I think it’s a pretty good assumption here that people are talking about mobile design, or design for mobile devices. Do they also use other terms, like “cell design,” “portable design,” or anything like that? You start thinking about these things, start thinking “hmm, well this is obvious and, yet again, might people be using different search terms when they’re thinking mobile?”

Oh, wow, that is interesting. It’s probably a good idea to go back over our existing articles and highlight variant terms for the topics that they discuss. What about your thoughts about the term “homepage”? We seem to get that a lot.

Are they looking for “homepage design,” or are they actually trying to find your homepage itself? Now, it’s possible it’s the latter; people do a lot of things you don’t expect them to do, and the data suggest that in spades. For example, people used to sometimes enter searches in URL address fields and URLs in search boxes! I’ll bet if you went further down into your long tail you’d see a lot of URLs showing up there.

Just so you have accurate coverage of the “homepage” term: add up the query ‘home-space-page’ and then some variations such as ‘main-space-page’ and even “mainpage,” as a single word. Once you include variant terms, or synonyms, the clusters might sort out a different way. It’s possible that more people are interested in home page design (with its variants) than mobile design. The same could be true about “e-commerce.”

And why do you think people do that—searching for home page and main page? Does that suggest they care more about homepage design as a collective than, say, mobile design?

Analytics can only tell you what happened; it doesn’t really give you any information beyond that. We don’t have the users in front of us to say, when you typed “mobile” or “homepage” or whatever, what exactly did you mean? Were you looking for the homepage or looking to learn homepage design? But we can get some indicators.

If we look at a term like “homepage” in the context of a session, we might see that it’s pretty common that people type “homepage” and then follow that up with a refinement by saying “homepage design,” “homepage branding” or something else where they’re qualifying their interests further. Session data—where you can actually see how it worked from first search, where they’re at their laziest and most terse, on through one or more iterations—can tell you a lot more.

What I always try to do with clients is to get this data into a really simple Excel spreadsheet. Don’t worry about affording Tea Leaf or Omniture, whatever the most expensive ridiculous analytics tool is, because even those tools are going to give you a few generic reports. What you care about is what emerges from your own analysis, as well as what types of things are important to your organization. The analytics data often fuels metrics, which then fuel your own KPI (key performance indicators). But if you don’t have any ad-hoc ability to play with the data and come up with your own reporting and your own metrics, you can’t really tie it back into your own KPI.

Long story short, try to get the data into a database (like Excel) where you can play with it, learn things, and come up with more custom types of analysis that make sense for your organization.

Looking at our internal searches, our numbers are pretty low. Most come from “organic” search results such as Google. How is this problem addressed within the discipline of Site Search Analytics, when the vast majority of searches on some sites originate, well, offsite?

Keep in mind a couple of things. Sure, a lot of people are coming on through organic searches like Google; the truth of the matter is that they still may need search once they’re on your site. If I find my way to a really good article on UX Booth and I want more like it, you must support contextual navigation at that point…but you won’t know all the types of contextual navigation you ought to have. In effect, what users are doing when they run a search on your site is establishing their “desire lines” or “desire paths” when they’re in a certain content. Pay attention to what kinds of searches originate from certain types of content.

For example you may have a certain set of related links from book reviews at UX Booth. It would be really interesting to look at all the types of queries that originate from all your book review pages; it would be really interesting to look at the types of content known to Google as “book reviews” and see the types of navigation you might want to build in based on what people are searching. The assumption here being, of course, that if they didn’t see a link here for what they want to do next, they’re going to ask for it in the form of search.

It’s useful to think about search as something more specific—not just an overarching thing that gets people around the site from the main page. It’s really part of contextual navigation. You can also apply the same approach to not just important content types, like book reviews, but if you have a handful of really useful articles or book reviews—content that you know is high traffic. You can also see what types of searches are originating from your most popular content. You’ll learn quite a bit about what people want next, and you’ll be able to support them effectively.

Thanks for your insight, Lou. It looks like we’ve got our work cut out for us! Turning the discussion around for a second: I don’t think you identify strictly as a site search analyst, right? You identify as an information architect. Are these two things symbiotic, you think?

It’d be foolish not to do site search analytics if you’re an information architect. The problem is that a lot of people’s understanding of IA concerns navigation of very much the top-down variety. But there are so many other ways to ensure findability. I don’t understand why anyone whose job is to ensure that information is findable wouldn’t look at search data unless there was simply no search tool on the site they work on.

Everyone’s brain responds to search data, so I always do exercises with my workshop students where I make them look at it. There, everyone sees the rich array of types of insights you can get. We’re designing for all different levels, but we often don’t force ourselves deep into the data like this. When we’re forced to, we’re going to see so many different ways to use this data. It’s just amazing how great our brains are at doing this.

Thanks so much for taking the time to analyze UX Booth, Lou! We’re certainly the wiser for it.

In case there was any doubt: there’s tons more useful information just like this in Lou’s new book, Search Analytics for Your Site. And just to get you started on your own, we’re giving away four copies to readers.

Now you try

Normally, we don’t have many requisites when it comes to winning books from UX Booth. You simply leave a comment and win. This time, though, we’d like to try something different, and it involves opening your copy of Google analytics and a conducting little bit of research on your own. If you’re not already familiar with web analytics, mosey on over to our handy beginner’s guide and meet us here when you’re done. Go on, we’ll wait.

Once you’ve installed and launched it, take a look at your most frequent queries. What’s the oddest query you’ve found, and what do you make of it? Post your thoughts—and your twitter handle—in the comments below.

And, okay okay, if you don’t have a copy of Google Analytics, you can still enter just the same. Just let us know where you plan on using Site Search Analytics in your work and we’ll be happy to oblige! Again, please provide your Twitter handle along with your comment. Entries will be accepted for the next week, after which point you’ll be contacted on Twitter if you’ve won. Thanks for reading and good luck!

About the Author

Related Reading

• Adaptable Information Architecture Workshop | Lou Rosenfeld
• Complete Beginner’s Guide to Web Analytics and Measurement | Andrew Maier
• Book review: Search analytics for Your Site by Louis Rosenfeld | Jon Hume
• Kick Butt With Internal Site Search Analytics | Avinash Kaushik
• Diving Into Site Search Analytics: Interview with Louis Rosenfeld | Jacob Guide
• Are You Into Internal Site Search Analysis? You Should Be| Avinash Kaushik