In my last post, I talked about the reasons why improvements to your site search are important for a good user experience. This post is a little more practical guide describing how you can improve your results.
“In short, best bets (where an editor can select the top results for certain search queries) is seen by many information professionals as about the cheapest and best way to improve your search engine, but the enterprise search industry doesn’t have much of a clue.” – Peter Van Dijck
Improving your website search starts with finding out EXACTLY what people are searching for on YOUR site to find what they want. (Note — don’t confuse Google Analytics referring keywords — how people get to your site — with the search phrases people are using within your website’s search.) Do this: talk to your web staff and have them show you the top search words and phrases. Most likely, the list of common search terms they will provide is different than what you thought your users were looking for.
“Search queries are gold: they are real data that show us exactly what users are searching for in their own words.” – Lou Rosenfeld.
Before you begin honing your search results, put yourself in the shoes of your users and test your site search to see what you get. You might be surprised by what you see.
1. Evaluate the top 20% of searches in your logs
Why does this work? Because search results follow a common Zipf curve. In other words, a few number of searches are performed many times, and a very large number are performed a few times.
A site’s search queries can be universally displayed as Zipf Distributions. — Lou Rosenfeld
Targeting the top phrases will serve most of your user’s needs. Remember, roughly 80% of your user’s needs will be covered by targeting the top 20% of phrases people are using to find information on your site. The remainder of the results, representing the “long tail,” trails off infinitely.
More on Zipf’s Law:
2. Match content on your site to the top 20% of search phrases
Now that you’ve got your top 20, group similar search phrases together. Then, rank your list from top to bottom to get your top 20. People use different words to get at the same thing. Disambiguating these phrases and grouping them together will greatly improve your results.
Now, ask yourself, “What pages might users be looking for when they use these words?” Manually match the correct pages in your site with the phrases people are looking for. For instance, searching for “excel” on Apple.com, you get Apple’s spreadsheet program “Numbers” in the search results.
There may be one or more pages that match the query. If so, strategically decide if each of these pages is unique or if they include redundant content. If redundant, consolidate the pages and point people to the joint page. If it is not redundant, provide two separate links, with clear annotation about what makes them different.
3. Create Best Bets for the top 20
With the list of top 20 and matching pages, you can now display those pages at the top of the search results for each search phrase. If more than one, just include all the listing in the search results. Best bets don’t always have to be separate from the other results. In fact, depending on your site’s specific content, it may be the best user experience to just include the best bets at the top of your results, incognito—looking just like everything else. As long as the user gets what they need, your job is done.
This may be the biggest hurdle to overcome in the process of improving your site’s search results. It may require fiddling with a database or modifying your search code. Work with a seasoned professional to insert a table in your database to serve up the matched results. e.g.: “When a user searches for “This phrase”, serve up these pages, in this order.”
Some content management systems have this feature built in or available through a plugin. Drupal and Expression Engine, for instance, have options for best bets. (See http://www.solspace.com/software/detail/super_search/ and http://drupal.org/project/apachesolr)*
4. Tweak the display of your site’s search results
You’ve worked so hard to display the correct results in the right order, but how do your search results actually look? Include these features in your search results for the best user experience.
- Keep it simple. There will be a lot of words in a search result list.
- Clean interface
- Easy to read
- Don’t hide content
- Features
- Display the searched-for phrase at the top of the page as a reminder of what was searched for. “Displaying search results for ____________.”
- Title (linked)
- “View page” link
- Title attribute in link
- Summary or description
- Publication Date
- Clustering of like content
- Document type (Website page, PDF and Word doc differentiation)
Other improvements to consider
- Filtering or sorting
- “More results like this”
- Consistency in naming
- Remove or group duplicate results
- Preview of page
Features that are not necessary
- Ratings
- Size
- Ranking
- Advanced search
- Hide/show summaries
5. Remove the cruft
If you have a lot of content on your website, chances are there are some old, dusty pages out there you either forgot about, or are keeping around “just in case” someone looks for it. Check your page analytics again. If this content is seldom looked at, kill it. Chances are, it’s getting in the way of what people are actually looking for.
The more content you have, the more likely it is that old, irrelevant information will show up much higher than the content that SHOULD be at the top of the list based on the search terms. The old content may very well be the best match based on relevancy or frequency of terms. Get the old stuff out of the way so the best stuff has room to move to the top.
Conclusion
Now that you’ve researched what your users are searching for and displayed appropriate matches to content in your site, you will not only improve the user experience, but create passionate, delighted users. Create the passion, and brand loyalty will go up.
References
- Thanks to Bitmanic, AJP, and Phil VanDrunen for the suggestions on CMS best bets.