Access to email has been restored after a routine patching process failed. We are referring the issue to the vendor before re-attempting this upgrade. Please accept our sincere apologies for any inconvenience caused by this scheduled maintaince window activity. No email will have been lost as a result of this issue as our backup MX is available to store inbound mail should it be necessary.
Our upstream provider will be performing updates to a number of routers between 00:00 and 06:00 on the 25th October 2011. During this period we anticipate a period of 30 minutes that will be ‘at risk’, within this window you may experience intermittent issues accessing services. This activity does not impact our offsite DNS and backup MX servers.
We experienced file system corruption on Linux server VLINWEB1 which caused inconsistent serving of web pages and email. A ‘fskc’ has been completed and the server rebooted.
All services on this server are now operational. If you are still experiencing an issue please contact support and we will investigate your specific issue.
Our data centre provider has advised this morning that they will be performing maintenance on 28th July between 00:05 and 02:00. This will involve them updating the firmware on one of their routers which we anticipate will result in a short outage of 5 to 15 minute during this window.
During the brief outage offsite DNS and MX servers will ensure no inbound email is lost.
Update: Completed by 1am with sub 5 minutes downtime
We experienced an issue with users attempting POP3 based connections to our mail servers for approximately 1 hour. We have restored service and now investigating the root cause. During this issue customers connecting using IMAP or Webmail were unaffected.
Our monitoring has highlighted a packet loss issue which is impacting our external connectivity. Customers may see this as slow or intermittent access to services. We are currently in contact with our upstream network provider.
We experienced a network issue this evening on one of our cluster nodes this evening at approximately 23:00 which resulted in some websites and email being unavailable for approximately 10 minutes. Further investigations will be conducted to isolate the cause of the crash and the reason why automated recovery did not occur.
Our monitoring has identified an issue exchanging traffic with some third party networks. We are currently investigationg this issue.
A data centre generator test will occur at 09:30 tomorrow morning. There should be no impact to services as UPS protection will power devices whilst the data centre switch from mains to generators and back again.
This morning we experienced a system outage due to an issue at our partner’s data centre. We are currently investigating the issue with them and will provide further specific information once we have it.
During the outage our offsite DNS and Mail services ensures that your domain names continue to resolve which allowed our backup MX servers to store inbound mail which now being forwarded to the mailbox servers for collection.
All systems were restored at approximately 09:30.
Thank you for patience with this unplanned incident.