Board is back again - Until PlusNet's next disaster

General slalom chatter...rant about the bad, rave about the good
Post Reply
User avatar
SteveM
Site Admin
Posts: 146
Joined: Wed Jul 28, 2004 7:28 am
Contact:

Post by SteveM » Mon May 02, 2005 12:32 am

Hi all, the board is back from yet another downtime. This time PlusNet managed to completely trash their CGI server, affecting all their CGI clients including business ones too. Ooops.

The database was safe, plus I had a backup of the BBS but I'm afraid it's taken me a while to find time to upload it properly and sort the permissions out again - please accept my apologies.

Well, I hope you've not lost faith in the BBS and are still happy to post here :cool:

All the best

Steve

PS. One day I'll put all the emoticons back on too

User avatar
SteveM
Site Admin
Posts: 146
Joined: Wed Jul 28, 2004 7:28 am
Contact:

Post by SteveM » Mon May 02, 2005 12:34 am

For your info, PlusNet's summary of the problem:

"2005-04-18 18:45:16 CGI Disk Failure and Data Loss - UPDATE

Following the problems experienced last week affecting both the Frontpage and CGI Platforms, we would like to take this opportunity to summarize the major events surrounding the disk failure on the CGI-FP storage platform, and remind customers of our intentions for the replacement CGI platform.

It was announced earlier this month that we would be performing routine maintenance on the CGI platform:-

http://usertools.plus.net/status/archive/1113421041.htm

This work was scheduled to ensure the stability and scalability of the CGI service and amongst other things, improve redundancy and resilience until the new replacement platform takes over later this year.

At 1.00am maintenance began as scheduled, when the CGI storage disks were remounted as read-only. Our network engineers then proceeded to backup the admin partition. This proved successful however it highlighted some issues with one of the disks in the array. This is not normally something that would cause concern due to the nature of the disk pack.

Our engineers continued to prepare for the backup of the stored data prior to performing the maintenance work. Whilst assessing the size of customer directories located on the storage array, there was a simultaneous disk failure which resulted in an issue with the disk pack itself. This, coupled with the initial failure, prevented the entire disk array from being rebooted.

We made every effort to resolve the problem by liaising with the associated hardware vendors as soon as the severity of the problem became evident. After much review and analysis, it was collectively decided by late afternoon on the 15th April, that the data was not recoverable.

We would like to remind customers that for security considerations the current CGI platform is completely segregated from the rest of our network. This means that any solutions for CGI involve dedicated hardware which can only be used for that purpose.

There are currently live CGI and Frontpage platforms that can be used should customers wish to restore any of their web content. These platforms are configured as such that customer's will have no problem running any of their existing scripts or applications.

We understand that the loss of data has caused an inconvenience to those that had sites hosted on the CGI platform but would like to assure customers that we will be putting measures in place to ensure that a similar situation does not arise with the the replacement solution.

Kind Regards,
Bob Pullen
Customer Support"

User avatar
Geebs
Posts: 458
Joined: Fri Jan 28, 2005 10:35 pm
Location: Doncaster
Contact:

Post by Geebs » Tue May 03, 2005 2:18 am

Well done Steve for recovering the post's etc
Paddle fast,,,Paddle safe Yorkshire Canoe Coaching

User avatar
SteveM
Site Admin
Posts: 146
Joined: Wed Jul 28, 2004 7:28 am
Contact:

Post by SteveM » Sun May 29, 2005 11:47 pm

It's back. Again. I'll investigate another host for the BBS....

Apologies for the downtime, hope you're all not too pee'd off with the board and will continue to use it.

Steve

Post Reply