SEO for site search
I find it a bit and somewhat alarming that an industry built around the concept of search engine optimization has published (and optimized) so little content about site search optimization. Site search is one of those “after-market” projects that people latch onto when looking for ways to improve their conversions. Site search is one of the least well-developed disciplines in search engine optimization, even though at one time (about 9 years ago) it was considered very important by many of the people who were active in the industry. I don’t think it takes a genius to figure out why site search fell into disrepute. People concluded that they could safely use Yahoo! and Google for their site search.
I recently discussed Google’s Custom Search Engine with a client. The client had paid for full inclusion in Google’s CSE index (a third index that is not currently integrated with Google’s Web search). The value of being fully crawled and indexed was obvious to the client. Alas! They were unable to configure the tool to properly categorize their pages.
There are some decent site search tools out there that fully index your pages and allow you to tweak their algorithms. But low-cost site search solutions don’t provide that full functionality (the Google Custom Search Engine does not, for example). So if you have specific documents — say .PDF files you cannot optimize because you don’t have the source code — that you want found for specific searches, you have to create special wrapper pages.
And now that you’ve created the wrapper pages, you have to decide whether to block the major search engines from indexing them (if you do, you have to provide alternative navigation to the .PDF files). We create wrapper pages for .PDF content, Flash content, image files, and other multimedia content. Wrapper pages are little more than doorway pages but they do get the job done when you’re dealing with content that you cannot embed in an HTML document.
Another roadblock to successful site search is the keywords meta tag. Most of the SEO community now (wrongly) feels it’s obsolete. Why? Because Google and Microsoft don’t index it. A lot of these obsoleters concede that Ask and Yahoo! use the meta tag but argue that it’s impact on search results is minimal, except for long tail queries.
Well, there are still site search tools that use the keywords meta tag. If you’re going to implement site search with one of those tools, it makes sense to optimize the keywords tag. Unfortunately, the SEO community doesn’t stop to talk about site search while trashing the meta tag.
The vehemence with which the SEO community attacks a frequently abused search resource wastes energy that could be put to productive use. Clients who implement site search solutions that use the meta tag need to be educated in the difference between optimizing for the major search engines and optimizing for your own search tool.
On-page optimization techniques usually work pretty well with most site search tools, but sometimes you find that your site search produces awkward results. A lot of news organizations, for example, have very bad site search tools. So do many eCommerce sites. Why? Because they don’t provide for advanced search capabilities. People need the ability to filter out undesired keywords, to specify date ranges, and perhaps to zero in on article authors, titles, and/or specific content.
When you’re optimizing a large site, you need to spend some time evaluating site search tools. Look at the CGI scripts, look at the third party indexers, look at the major search tools. Use them hundreds of times to search for pages on the large site.
The reporting functions provided by a site search tool are equally important to its indexing and query-resolution capabilities. You need to know how often the tool crawls the site (and if you can launch a crawl on demand). You need to know how long it takes to crawl the site. You need to know how much disk space you need to maintain the index.
You also want to know what people are searching for. You want to know what they are finding and how long it takes them to find it. You want to know how closely your site search results match your major search engine results. With enough content people will view your site as a searchable resource and not simply as a large site that is too big to navigate through links and menus.
Site search straddles the fence between alternative navigation and functional content. That is, with a robust site search capability, you can provide your visitors with content that the major search engines do not. For example, suppose your site search tool organized its results so that users could read a full paragraph of text, view a picture, and see which section of your site holds the pages it finds? What if the site search tool went a step further and suggested similar or related pages?
You need to design your content with site search in mind. The easier it is for people to find what they want through site search, the more effectively the site engages with its visitors. Simply optimizing for Google undercuts the site search experience. In fact, site search needs to work very differently from Google’s Web search. Whereas Google wants to show only vetted pages in its search results, you don’t need to worry about “good” and “bad” pages. In site search there is relevance and there is freshness.
If a site has enough content to warrant a site search, people will want to know which content is freshest — especially for sites that host multiple articles from multiple sources. If you have enough content to include multiple pages for specific topics, people will want to know which pages are the most important, which pages offer the most detail, and which pages most precisely match their needs.
You may be able to include content on your pages that helps site search more than Web search because you don’t have to worry about links, PageRank, and violating search engine guidelines. In fact, search engine guidelines often provide better advice for site search optimization than the majority of search engine optimization specialists.
When you’re just optimizing for site search you’re not concerned about where you get links from or which pages have the most PageRank. With site search you want all the content to be found for the right queries and you’re not worried about competition. You therefore have no incentive to spoil the site search index. You have nothing to gain from building links.
When you just think about site search you think about your pages and what you include on them in a very different way. But you also need to think about how your search results appear, what they tell your visitors, and how quickly your search results help your visitors find what they are looking for. You’re not in competition with yourself or anyone else. You’re just focused on providing a better search result.
And that is really what all search engine optimization should be about, so there is much we can all learn from implementing and optimizing effective site search tools.
www.seo-theory.com
published @ September 5, 2008