Tuesday, March 31, 2009

Resolved -- GoGrid Network Connectivity Issues

** UPDATE @ 13:16 PDT **
At this time we are no longer experiencing network connectivity issues with the public GoGrid Network Infrastructure. While the problem does seem to be resolved, we will not label it as such until a full Root Cause Analysis is complete. Please stand by for further updates.

** UPDATE @ 11:01 PDT **
We are making progress on the network connectivity issue which is related to specific ISP's and Routes coming into the GoGrid Network. You may notice that your systems are becoming accessible, though we are not noting this as "fixed" yet. If you are still experiencing a failure, please take the time to run a traceroute to your systems IP and send that to the GoGrid Support Staff

Customer Impact
We are currently experiencing an issue where our network is inaccessible from certain parts of the world and other networks. We are treating this as a top priority and expect resolution shortly. If you are opening a Case for this issue please try and include a traceroute so that we can use that information to help us isolate the issue.

Details
On Friday, March 27,2009 11:10 PDT, and again yesterday, Monday, March 30,2009 12:25 PDT, GoGrid suffered a series of large scale distributed denial of service (DDoS) attacks that affected the network connectivity of many GoGrid servers.

These network attacks were of a type that we had not seen before, and which our automated network attack prevention hardware was unfortunately unable to prevent.

We are still experiencing network connectivity issues resulting from the attacks, and a number of GoGrid servers remain unreachable from certain source networks.  Our network engineers are working continuously to resolve the problem.

No comments: