Here are PCIT’s network uptime results for May 2017. We also report any security incidences along with the measurable impact. It is disappointing that our networks under management had more security breaches in May 2017 than the rest of 2017 and 2016 combined. All of these incidences occurred in one 7 day period. Adding in an extra 3+ days of clean up I’ll call it 10 days of stress.
Below are the client results averaged across the entire client base. These averages don’t tell the true picture as at least three clients were severely impacted by malware. In the worst case the malware destroyed not just 1 server and 1 computer it also encrypted all of the data on the backup. There is a very sinking feeling when all of the data the client has is found to be destroyed. Let me advise every reader who may have data to protect, you don’t want this to happen to you. Thankfully there was an offsite copy.
With these results now on the record books it is cause to eat some very humble pie. Especially with the one client who lost their onsite backups. Since those memorable 10 days PCIT staff have since taken new steps to better isolate and lock out our customer’s backup data from the rest of the network.
Globally as a company our customers experienced 99.86% uptime across all the systems we review down from 99.93% in April.
Only 77.8% of our customers experienced 100% uptime in May down from 94.4% in April. Ouch these results really hurt. We’re a small company and I look every one of our clients in the eye at least twice a year. Downtime is not usually a great topic unless there happens to be none.
98.35% of all customer managed computers and servers operated in May without any malware or virus incidents. This is down from 99.80%. That difference of 1.45% was a big difference.
Recommended takeaways from incidents in May
One lesson we have learned in May is that Remote Desktop connections without a second layer of authentication are like playing with dynamite in 2017. We can say this because some of the incidences in May occurred via a brute force attack that broke into the Remote Desktop session. Remote Desktop can also be called RDWeb, private cloud, Terminal Server and more. Most importantly, we learned this was an area of network security that can not be managed in the same manner it was previously.
In improving the network security we have subsequently helped several clients enforce a password rotation policy. We have also audited and enforced a policy to limit multiple login attempts on Windows Servers, In addition we have recommended and tested two-factor authentication solutions. Finally, there have also been some restrictions on Remote Desktop services by IP range where this is practical.
Backups – Without 100% rock solid backups May would have been a lot worse than 10 days of stress. I’d rather not think about the consequences if the backups hadn’t have been in place. Suddenly all the money spent on StorageCraft ShadowProtect on behalf of our clients looks worth it given the alternative.
Contractors – In at least two cases trusted contractors of our clients look to have made some poor security related decisions while working on our clients infrastructure via a remote connection. This led to security incidences in May that were severe for the clients. If it has been a while since an audit of who has remote access and why has taken place we recommended that as a helpful next step. More importantly, discuss this vulnerability with each party. We suspect in both cases the contractor had no idea their actions caused so much damage to operations.
We welcome your feedback and perspective.