Les Jones

Blogging > Web Server is Back Up

It appears everything's back to normal. If you can see this, the Web hosting company has moved my blog from their old hosting center where a router went down a few days ago to their new facility. Here's the letter they sent out this morning:

Dear WestHost Client,

On Wednesday, October 19, at approx 2:00 a.m. (MST) one of our core routers in our legacy (1.0) datacenter experienced a hardware failure. The redundant router failed to sync up as it was designed. When new hardware arrived work began immediately to restore network connectivity. Continued hardware issues with the replacement parts compounded the problem further, resulting in extended troubleshooting procedures.

As of Thursday morning, engineers continued to troubleshoot and additional testing showed that the new hardware was not going to work as planned. Both Cisco (current router provider) and Juniper (precautionary backup) are telling us that neither of them can get us the right hardware until late Friday evening at the earliest.

Instead of waiting any longer, we have moved to our contingency plan and servers in the 1.0 Chicago legacy datacenter are being transported at this very minute to a backup datacenter facility (this process started today at approx 4:00 p.m. MST). Currently, we are expecting the servers to be online early tomorrow afternoon.

We perform regular testing with our network hardware, but unfortunately this time failover did not work as designed. Our 2.0 environment is setup in a more robust, more redundant network design. As you know our goal has been to upgrade all remaining 1.0 clients to our 2.0 environment. Many have been moved, but unfortunately your account was one of the few still remaining. We will continue to aggressively push our goal to have everyone on 2.0.

I apologize for any inconvenience this has caused and understand the impact it has had on you. Our engineers and support team have been working diligently. No expense is being spared in resolving this matter.

Sincerely,

Brian Chambers
Vice President of Operations
WestHost Inc.