SEO for Subdomains - Subdomains and SEO

SEO digest

Search engines have been tolerant of subdomain spam for a very long time. Google has been tightening its subdomain rules over the past couple of years but they still don’t make a very clear distinction for what is and is not acceptable. I’ve been told privately there may be a bug in their subdomain analysis software and that eventually they’ll do a better job of clearing spammy subdomains out of their search results.

Subdomains have proven to be useful to Webmasters and users alike through the years but there are no standards governing the creation, use, or search evaluation of subdomains. On a large enough site you can embed a great deal of content on subdomains that are never indexed by search engines and yet drive a lot of traffic to those subdomains.

Massive content providers like subdomains and it’s relatively easy to think of examples of sites that use subdomains to serve content which is relevant to users by city, state, country, or language. There are also many Webhosting services that provide subdomains to users. Some of those services have hundreds of thousands, perhaps even millions of subdomains indexed in the major search services. But a lot of those services seem to have tripped filters or are subjected to high trust thresholds.

Search engine optimization spammers have ruined many a subdomain hosting provider’s search credibility by populating large numbers of subdomains with autogenerated content, doorway pages, and spammy link crawl pages. I have noticed a trend among new Webhosting providers where new users are required to apply for acceptance into the hosting community. Some of these providers even require that you go to a forum and introduce yourself.

Wouldn’t it be a dream if you could just be open and honest with them about what you intend to do?

Spammer: “Hi. I’m Spammy Ravis and I’m here to create 100,000 subdomains to promote my real estate empire.”

Hosting Community: “Hi, Spammy. We’re all blackhat search spammers, too. Welcome!”

Blackhat Borg spammers have assimilated quite a few free hosting services into their spam empires through the years. Time was when you could create an account pretty easily on any of hundreds of services and no one cared too much what you did. But those days are gone.

Some people have experimented with creating their own subdomain empires. Earlier this year I read a major SEO pundit’s complaint about how he tripped Google’s subdomain threshold filter and got a domain banned. I actually set up an experiment this year with three domains where I added subdomains at a rate of several per week. All three domains were banned by Google within a matter of months. There were fewer than 30 subdomains on each domain (that is, by the way, the first time I have ever gotten a domain banned from a search engine — but as these were experimental domains, they included no client content and none of our clients’ websites were affected).

“How,” I asked myself, “does one launch a Web hosting service if adding subdomains quickly results in bans or penalties?”

I do have subdomains on Xenite.Org and they seem to be doing okay. Xenite isn’t hurting, either. Then again, I haven’t added 20 subdomains to Xenite. I don’t know what I would do with 20 subdomains, but it would be nice to know I could do it if I had a need.

I think that Google has indeed developed some subdomain evaluation criteria (though Google’s Webmaster guidelines say very little about subdomains). For example, since I see new subdomain hosting services appear in the search results, I deduce that Google has figured out a footprint (or perhaps two or three) for what a legitimate hosting service’s site looks like. Googlers and Xooglers have also occasionally suggested that people can use subdomains for translations and geotargeted content.

Now, I get translating your Web site into 30 languages and creating 30 subdomains. But I’ve seen situations where people who create subdomains for cities and states (within the U.S. and U.K.) have been subjected to penalties or bans. What’s up with that if geotargeting subdomains is acceptable?

I’ve come across a couple of conversations where Googlers have suggested to penalized Webmasters that they did not have enough unique content on their subdomains to justify the subdomains. I have looked at several such Web sites and I feel there are certain characteristics they all share that probably contributed to their banning or penalizations.

1. They were highly dependent upon user-generated content
2. They populated uniformly templated subdomain networks with mashup content
3. They lacked user-generated content
4. They were attempting to invade well-established query spaces and compete with larger sites that had tons of content
5. Even with their mashups, they offered nothing substantially unique to searchers

If you ignore the existence of search engines for a moment, you can evaluate the probabilities of success for such sites pretty quickly. If they had unlimited budgets to use for advertising, could they build core audiences with sufficient loyalty to sustain them?

A generic Web site that provides a comfortable user experience could succeed. All you have to do is provide a better user experience than all the other sites that have already established themselves in your niche. Once people get to your site and see how much better organized it is than the older sites, some of them will stay with you, and many of those who stay with you will recommend your site to other people.

But if a better user experience is all you’ve got going for you, is that enough to work for a search engine where everything else seems to be working against you? I would not want to bet my future on such a plan.

There are plenty of sites that rely on mashups and affiliate links dominating search results today. It’s easy for people to think, “If they can do it, I can do it”. Unfortunately, that no longer seems to be the case.

So complicating the process by spreading thin, hardly unique content across multiple subdomains doesn’t sound like a winning plan to me. Maybe some people make it work but I haven’t seen any recent examples.

Mashup sites can be competitive in search results. If your combination of mashed up content is unique enough to create value, you should be okay. There are some advantages that subdomains offer for search engine optimization and Web marketing.

1. It’s easier to create brand value for a subdomain than a subdirectory
2. Subdomain URLs may be easier to remember than subdirectory URLs
3. Subdomains can act independently of their mother domains
4. Subdomains can be named for the most important keywords
5. Subdomains can have their own unique page design and architecture
6. Subdomains are viewed as more credible than subdirectories

In order of priority, a large content site can use subdomains to:

1. Package unique, distinctive content in different styles and formats
2. Diversify brand value to attract unique user audiences
3. Repackage content for specific languages
4. Organize content for important regions or categories

I think that merely using subdomains to geotarget content doesn’t make much sense if the content is not specifically unique and relevant to the regions. If the subdomain organization adds value to the user experience (maybe users realize it’s easier to type in city-name.example.com) then I would say it’s probably worthwhile to focus on the potential geotargeting benefits.

If you have an existing domain where you want to redistribute content to enhance geotargeting, I would recommend doing it piecemeal rather than all at once. Pick a couple of cities, states, or regions and move the content to appropriate subdomains. Use region-specific language/idiom (expressions) as much as you can so that the content really is localized. Implement 301-redirects, update your HTML and XML sitemaps, and update your on-site navigation and other internal links.

Clock the changes as they occur in the search results so you can develop a reasonable expectation for how long it will take to transfer the content completely. You can accelerate the process after you have benchmarked your changes in traffic, referrals, rankings, and indexed locations.

There are some abuses to watch out for if you want to work with subdomains:

Monitor your DNS settings - Spammers have been known to hack Web control panels and insert CNAME records that point to subdomains on other servers. If your Web site’s admin tools are compromised, you could have millions of subdomains hanging on your domain and you would never know it. Make sure you actually control each and every subdomain associated with your domains.

Block outbound linking from user-generated subdomains - If you’re in the free Web hosting business, you naturally want your Webmasters to be able to link to whomever they like. But if you use subdomains to create user profiles or otherwise reward your users with special content for registration, don’t let them link out to sites you cannot vet. If you do allow users to embed links, figure out a way to reward good users with links that pass value.

If you want to create subdomains through free hosting services, understand that you may have to live with some limitations:

1. Some services only allow you to use 50 megs of free space
2. Some services won’t handle XML sitemaps (you have to use .TXT sitemaps)
3. Some services won’t allow you to install a .htaccess file
4. Nearly all free hosting services run some sort of ads on your sites
5. Some services will help you build your site, but you are locked into their templates
6. Some services implement their own 404 error handling, so check your links carefully
7. Some services interconnect their subdomain accounts through either open or hidden navigation

The search engines will crawl subdomains on free-hosting services but that doesn’t mean your Web sites will be indexed. Some perfectly good Web sites are excluded from Google’s index, apparently only because they are hosted on free hosting services Google doesn’t trust (I have found links pointing to the sites in Google, so Google should be able to find them). Many hobbyist and personal Web sites in particular enjoy little to no Google search visibility because Google just doesn’t list their pages.

Subdomains can still pass value as if they were independent domains — but they need to obtain value before they can share it. That value has to come from being recognized as unique and useful enough to be included in the search indexes; from being trusted enough to rank for relevant expressions; and from links on other, trusted sites to establish validity.

Whether you’re moving content to new subdomains or creating new sites on subdomains, you have to apply the same general rules of search engine optimization as you would for whole domains. If there is not enough unique quality and value to distinguish a subdomain from the rest of a Web site, it’s probably not worth your time to create the subdomain.

The only way you can be sure of what works best for you is to apply the SEO Method: Experiment, Evaluate, and Adjust.

Just be careful with your experiments. Creating subdomains in volume is risky. If they don’t look distinct enough, you may wake up one morning to find that several months’ work has simply vanished overnight. It’s a lot less painful when the lost effort is an experiment than when it’s a live site you’re counting on.

www.seo-theory.com

published @ October 29, 2008