Beware the Dark Site
Aug 13th, 2007 by Jason Irwin
Discussing SEO, SEM, internet marketing and other such techniques is typically something I leave to the experts but, after some recent troubles on my site, I’d like to discuss one of the biggest threats to any SEO effort: web site down-time.
Over the last few weeks, large numbers of bloggers have had trouble accessing their sites hosted with AN Hosting. The excuse from the administrators has been “server migration,†but this is either grossly inaccurate or the admins were not sufficiently prepared to perform the tasks. Without getting into the details of the problems found with this hosting company, and the fancy workarounds they have to claim a 99.9% uptime, I’ll instead share the results of the downtime.
In the first twelve days of August, my site was down for just over 73 hours. In that time I had lost at least 400 potential unique readers. While this is not a huge number by any means, it’s still a large chunk of the reader base. During this time, the various search engine bots tried and failed to access the content and rank it accordingly. However, when bots cannot access a page, the content’s ranking is reduced.
Less than a month ago, j2fi.net was the first result on several regional Google pages when searching for several of the keywords used throughout the site. As of today, the page can now be found on the fourth and fifth pages for the very same words. As 70% of my traffic comes from Google, this is a pretty severe hit. For anyone that has an internet-based business or a monetized blog, this situation is unacceptable. We work hard to build our sites and earning those SERP rankings, and it’s unfair to lose any advantage due to actions outside of our control.
After digging around the Google Groups in an effort to repair the damage, I happened to find this thread that discussed the same problem. If a site was temporarily down, I had assumed that Google (and other search engines) would not de-index the site but instead give it some time to return. However, this does not seem to be the case.
A user by the name of “bigmamaslings†posts the following problem:
I’ve been having problems with downtime from my hosting company (am moving tomorrow as this is the final straw) and I’ve now practically disappeared from Google. I was in 3rd position for 5 of the more popular searches for my business and now - nothing….
Vanessa Fox of Google replies:
If the host is down when Googlebot tries to access your pages, then those pages may disappear from the index until Googlebot can crawl them again. In webmaster tools, do the pages you want indexed appear in the crawl errors section? If so, then Googlebot was unable to access them….
So from this little exchange, it seems that Google might just remove the pages from the index completely if the site is not available due to server issues. With that said, Ms. Fox does explain that the Googlebot will try to access the pages a few times before dropping you from the index. As for how long it takes for a page to be ranked again after the site is back up depends on a number of factors, though, such as how often the site is crawled in general.
This makes total sense as Google would want to keep their results page fresh. If people were to continually receive a bunch of dead links after a query, that search engine would quickly go the way of Altavista.
To avoid this situation, site administrators need to keep an eye on their downtime. If it gets too excessive, then it might be time to move hosts, or perhaps consider a dedicated server. Nobody wants to see all their hard work and patience be obliterated because their site has gone dark.
[This post was written by Jason Irwin from Jason’s Random Thoughts, a blog with topics ranging from technical solutions to political opinion.]
[If you enjoy reading MsDanielle.com, please feel free to subscribe via RSS or get free email updates.]
Server reliability is very very very important… I use bluefur hosting personally, and I have been very happy with them.
Also the robustness of the control panels and database handling is very important too. While most hosting providers give you access to cpanel and phpmyadmin and all that basic stuff, some will neglect it.
When I take on a contract for web design services, I try to pick which hosting server i want to work on. And I’m starting to contemplate charging extra if the server is hard to work on.
Server reliability is key!
I have been struggling with my host for a while and it’s aways frustrating when your site is down!
i’ve only heard good things about bluefur (from all the bloggers!) as well. what if my hosting is paid through jan 2008? i have to wait if i want to switch?
bluefur is pimp! i have always gotten amazing service from them!
Does anyone have opinions on godaddy.com?
hey tyson! jon lee did a registrar review which you can check out here http://www.jonlee.ca/domain-registrar-review/
good luck!
Yay! My first guest post is published
Server reliability is the lifeblood of webhosts. One would think that these companies would start threatening admins with bodily harm if the machines were down for more than a few minutes a week.
Well … maybe not bodily harm, but it wouldn’t be a fun work environment for anyone that couldn’t keep their servers healthy.
Thanks for sharing this, Danielle
thanks jason! I really like how this article came together, and so quickly. thank you so much for bringing up a very important topic. i haven’t experienced my hosting going down for very long, but on occasion it gets sluggish. i guess for now i’m one of those people who’s crossing my fingers!
I’m quite surprised by this since I’ve only heard good things about AN. I’m with (mt) and they seem very serious about their web hosting which is great to see in a hosting company.
Getting de-index by google is just about the worse thing I can think of happening to my site :S
I just had to tell you, gorgeous site. I’m just starting off and I’m hoping I’ll be where you’re at one day
thanks richard!
your site looks great, and i especially like your header image. good luck makin that $$$ 
This is something I have definately given a significant amount of thought to. The server that houses my site sits in my home office. I currently live in an apartment and plan to move eventually. I have wondered how I can get things to work to minimize downtime while I transport the server from one location to another. The other thing that concerns me is that something could happen that would bring it down when I am out of town.
Nice guest post! I hope to read more from you in the future…
Thanks, Chris
I wasn’t sure if this would be a subject that people would be interested in as everybody seems to have a decent webhost.
[…] all the discussions about this topic you can only come to one conclusion: uptime is very important if you don’t want to rely on pure […]
As the bigmamaslings mentioned in the article, I can honestly say that this was one of most painful episodes in our 4 years in business. It taught me a valuable lesson in being a cheap-wad about my hosting company though and I now pay for reliability. Just to add that once I complained and kept pushing for re-inclusion (and getting hold of a real person at Google to help me took some doing..) the site was back up to our previous rankings within a week, almost as if it never happened. Not fun though…