[INPROGRESS] Email server crashed AGAIN
The email server crashed again. The crash an hour or so ago showed indications of hardware and/or firmware failure, a serious issue. Your beleaguered CTS staff continues to attempt to resolve the problem.
The email server crashed again. The crash an hour or so ago showed indications of hardware and/or firmware failure, a serious issue. Your beleaguered CTS staff continues to attempt to resolve the problem.
The email server has just crashed. CTS staff are investigating.
The CTS system administration team has been working through the night on this problem.
The copying of users’ email folders continued through the night, but early estimates of the progress were overly optimistic. As of 6:00 am today (Friday), about 2/3 of the data has been moved from the drive that failed, onto a new drive. Half a terabyte remains to be copied.
Experiments with restoring access to the copied folders, while also keeping mail that has been delivered during the last day, show that this is trickier than originally thought. Work is ongoing.
We apologize for inconvenience. Rest assured our system administration team has restoring email service as their top priority.
Campus WiFi has been updated and is back online.
Tomorrow morning at 6am, Network Services will be apply updates the the campus firewall. This will cause a brief 10 minute outage to any incoming and outgoing services to include the VPN.
– CTS
Tonight at 10pm, Network Services will be updating the campus WiFi software. This will cause a brief interruption to SUNY_W@SP only, not the Dorm wifi.
-CTS
The failed legacy storage holding email folders has been recovered. All email folders and contents are being copied to new storage, after which access can be restored. Over a terabyte is being copied. This will take a few hours for the copy, and another hour or two for restoration of access. CTS thanks you for your patience.
The email server reboot is complete, and CTS staff are now able to assess the situation with inaccessible folders of old/saved email.
The email server (“hermes”) will be rebooted in a few minutes, at 3:00pm, as part of CTS’ attempt to restore email folders for those affected by today’s events.
Email sending and receiving should now be working for all users. However if you were affected by the problem earlier today, your existing email folders are presently not accessible — it will look as if your email account has been freshly created.
CTS staff is working to restore access to email folders, however, there is a chance we may have to restore them from backups. There has been a failure in legacy hardware that the email system still uses.
Thank you for your understanding and patience. CTS is actively moving to have a far more robust email infrastructure in place by the end of the summer.