MSPs must now make cybersecurity their top priority
Tuesday, January 19 2021
Headlines about cybersecurity hacks seem to appear on a daily basis, with big-name customers and vendors often affected. These stories are sometimes accompanied by spurious calculations of the amount of money lost by companies to ransomware attacks. The truth is there is no way of calculating the loss, even simply financial, because it happens so often, and the details are almost never known. But for the growing number of managed service providers and their customers, these threats are real. Lives and livelihoods are now at stake. The latest generation of attacks is causing even more concern as the trust we place in the software we all rely on has been shaken to its foundations.
Hacks take on a new dimension for vendors, MSPs and customers
In early 2018, security threat researchers found a number of remote monitoring and management (RMM) tools (such as ConnectWise, Webroot and Kaseya) were being exploited through flaws in their software and used to gain access to end-customer systems. Typically, a hacker would target a managed service provider (MSP) that employed these tools to manage their end customers’ IT environments. They could infiltrate a system and once inside push ransomware, change passwords and extract data. One MSP in Norway, Visma, was targeted between 2017 and 2018 by APT10, a group linked to China’s Ministry of State Security, using stolen Citrix/LogMeIn credentials, which gave it access to Visma’s network.
Many companies, such as Datto, champion their backup and disaster recovery software as a way to protect against some ransomware attacks using cloud backups. If used properly, they can be effective. Unfortunately, the nature of RMM and professional services automation tools makes them ideal for attacks, even where no flaws have been exploited. In September 2019, a story emerged that showed stolen credentials from an MSP employee allowed hackers to disable and delete these cloud and local backups, before deploying ransomware. In this instance, it was also found the employee did not use multifactor authentication, and there were no system alerts in place to detect the fraudulent activity.
In November 2019, ConnectWise once again discovered what it described as a vulnerability in its Automate RMM software. The vulnerability came from an unpatched flaw in the software, which ultimately allowed server ports to be exposed to hackers who could control customer networks and install malware. In these instances, it was deemed simple procedures, such as multifactor authentication and proper configuration of firewalls, could have prevented these issues. These hacks and exploits continued throughout 2020, typically targeting MSPs and using their own RMM tools to attack their customers.
In December 2020, something different happened. SolarWinds announced that an update to its Orion network management software from March of that year had been hacked by Russian intelligence division SVR. It added malicious code to the update, which was pushed to customers and was even used to infect the systems of its vendor alliance partners, including Microsoft and cybersecurity specialist FireEye. This was more complex and sophisticated, and required resources greater than those used in the earlier incidents. Those previous vulnerabilities were akin to someone finding you had left a window open in your house and sneaking in. The SolarWinds scenario was closer to someone finding a duplicate key to your front door and keeping it so they can get in whenever they want.
Most recently, in January 2021, Microsoft discovered a certificate used by Mimecast to authenticate users on its email server had been compromised. Mimecast stated the certificate was used by around 10% its customer base, around 36,000 customers. The hacker may have been able to use this certificate to read and modify encrypted Microsoft 365 data to target a group of customers. This would have required sophisticated knowledge and possibly insider access. Once again, a vendor was in the position of calling customers and partners, sending emails to tell them to update their certificates. The full extent of all these breaches will never be known, but the personal nature of these hacks is clear.
MSPs are often the threat vector
In the last three to five years, MSPs have become a serious target. As a repository of many customers’ data, and with access to their systems, this is logical. Companies outsource their IT for reasons of cost and expertise. Not every customer can afford to source all their own IT or have a fully staffed IT department with the skills required to manage the digital technologies they employ. But if you eliminate typical human error, put in place best practices and training, and even monitor for events where you might “leave the window open” now and again, the channel and its customers should be able to consider themselves relatively safe. Unfortunately, software supply chain hacks make these measures less effective.
Hacks and backdoors are certainly not new but the scale and breadth of the threats we now face have grown. Every software product (and its updates), vendor alliance, process and practice must now be scrutinized. For MSPs and their customers, it means yet another layer of care must be applied. Even common best practices, such as multifactor authentication (MFA), properly configured firewalls and endpoint detection and response (EDR), are not silver bullets. Researchers found that a Russian state-sponsored group dubbed Dark Halo (also known more commonly as APT29 or Cozy Bear) had created a way to bypass the MFA tool Duo (acquired by Cisco in 2018) using the Outlook Web App, to allow them to use a single sign-on process when accessing systems. This was not a simple procedure and involved the kinds of tools and skills not readily available to many. But now it has been carried out at least once (it is safe to assume this procedure has been used and replicated many times), it will become more accessible to others outside the state-sponsored domain.
MSPs must prepare for a new type of customer relationship
In 2021, more of these incidents will occur and both vendors and MSPs must admit they are nearly powerless to stop them. The focus for many must include being ready to minimize ransomware success while monitoring customer environments with more insight. For example, monitoring network activity to determine where a threat actor is using a real employee’s details to access data but is behaving in a fashion that is not consistent with their profile. This and other measures will require greater collaboration between the MSP and the customer.
When engaging with an MSP, the customer is often primarily concerned with making sure the MSP is doing their job. Where that includes cybersecurity, they are simply expected to do what it takes to keep their businesses safe. Incidents are generally treated as being the remit of the MSP, even sometimes in cases where it is the customer’s error that has brought about the security incident. MSPs need to set up increasingly tight measures on the technology but they will also have to prepare their customers with more training. They must also be prepared to refuse to work with customers that do not follow certain pre-approved guidelines. As cyber-insurance grows, the onus is on both parties to make sure they are aware of the threats they face and take steps to minimize them. If they do not, the damage may be too great to bear. Below is a non-exhaustive list of things MSPs will have to consider for each of their customers. Of course, these steps are only what can be controlled by MSPs, and the kind of vulnerability caused by hacks further up the software supply chain will still pose a significant risk.
COVID-19 has taken us into an alternate channel dimension
As a result of the move to remote working for many office-based roles in the last 12 months, home security has come under the spotlight. MSPs report some customers installing business-grade networking setups for some of their employees, particularly those in finance or legal roles. Some are even moving to provide government- and military-level log-in and authentication procedures for their customers. The growth of cloud-based software and infrastructure has also contributed to the threat to MSPs and end customers.
The greater the diffusion and digitalization of work, the more risks we face. Microsoft, AWS and Google are now under far greater threat from Russia and China than from one another. If a vendor’s products are not just exploited but become themselves the delivery mechanism for malware, the loss of customer trust could be fatal, even for the hyperscalers. As the majority of MSPs in EMEA are Microsoft-focused, the vendor has become the number one target for threat actors. It has also played a crucial part in discovering infiltrations in some of the incidents discussed in this report. It will continue to play a pivotal role, and all MSPs must spend a greater proportion of their time checking their inboxes to be aware of the latest incidents and threats and not just ignoring them. A delay in response or threat intelligence from MSPs when dealing with customers can exacerbate an already difficult situation. One of the last things an MSP wants is its customers hearing of an issue before it has.
As we move forward to a (hopefully, soon) post-COVID world, many workplaces will inevitably become more hybridized. This makes the management of threat vectors even harder, moving from home to office and all other places where people can work. For software and cloud vendors, the importance of protecting, auditing and monitoring their product releases has been made publicly clear. The alliances in which Microsoft engages with other vendors, for example, will almost certainly be even more carefully controlled as the threat to its business is so acute.
MSPs have an opportunity to differentiate themselves further through their processes and practices. Certifications and specializations from vendor programs could be updated, and training must certainly be a prerequisite for any of the MSPs’ customer relationships. Though cybersecurity issues have almost always been an inevitability since the digitalization of workplaces, there are things that all parties can do to minimize the risks. Channel companies can play a vital role in managing this new era of threats, and their skillset will expand to protect their customers’ businesses. If, as the cliché goes, every company is now an IT company, then cybersecurity must be the highest priority.
For more information or to receive a copy: