As you have probably realised, Canadian Content has suffered a serious outage of over 30 hours which has been linked to an explosion in the data centre where our Domain Name Services server is hosted. Along with that, we also have a legacy server in the same building on the same floor which was effected by the outage.
The outage occurred at 7:29pm Eastern time on 31 May 08 caused by a serious fire in the main power supply base. 3 walls were blasted and we're thankful to report that nobody was injured. Fire safety personnel rejected the backup plan of using the backup power supply because the damage to the facility was too intense to give a straight go-ahead due to safety concerns. Our service provider has assured us that no hardware other than the power supply was damaged and our site will be sustained by the backup power supply which has now been approved by the fire chief.
As we've never had an outage of such magnitude before, nor have we ever witnessed such damage, we have learned from this lesson and are subsequently working on having a 3rd and 4th DNS service setup and moreover a skeleton backup server with some features stripped in case an event such as this occurs again.
Unfortunately, the redundancy we have setup in the past did not help us as the “big mamma” server hosting our DNS also suffered the outage.
Rest assured, this has not been brought on by a security breach but a real serious explosion over the weekend.
A few grey hairs later, I'd like to welcome you back online our web site and hopefully a break from the Internet on Saturday night and during Sunday has given you some time to relax.
(ps. It could take half a day or more for everybody to be able to access the site again)