Is the Kaseya Hack Actually a Supply Chain Attack?

[This was originally published on RiskBasedSecurity.com as part of a larger series on the Kaseya breach.]

---

What is a Supply Chain Anyway?

Within hours of the Kaseya breach becoming public, some critics called out that it was being incorrectly labelled as a supply chain attack. As Nick Carr pointed out, “precise language is important in security”. These calls only grew louder as we learned that devices weren’t compromised due to a malicious software update injected upstream, despite the Kaseya Virtual System Administrator (VSA) devices impacting customers downstream.

If the term ‘supply chain’ is being used incorrectly, it is important to define what it means and then examine what might constitute such an attack.

Let’s start with the definition in common use, and then unpack it.

Googling for “supply chain definition” gives us “the sequence of processes involved in the production and distribution of a commodity“. While accurate, that also has a fairly broad meaning. Looking at Wikipedia we get the following:

“In commerce, a supply chain is a system of organizations, people, activities, information, and resources involved in supplying a product or service to a consumer. Supply chain activities involve the transformation of natural resources, raw materials, and components into a finished product that is delivered to the end customer.”

This definition leans more heavily on physical items, since it specifies resources, materials, and components along with the term “finished product”. Since software is often updated, and pushing those updates to customers is more about modifying an existing product, that definition may not be entirely accurate for our purposes. This gets more convoluted if we look at the Wikipedia article on Supply Chain Attack, which includes an example of ATM malware that is exploited by walking up to an ATM and removing cash if it is infected. It does not specify how the malware gets there, and a majority of ATM “black box” attacks require physical access to carry out. Such a case is not a supply chain attack at all.

To add even more confusion, the UK National Cyber Security Centre says that watering-hole attacks count as supply chain, which we simply cannot agree with. For readers who are not vulnerability and threat management professionals, a watering hole attack is when you are lured to a page that hosts an exploit which compromises your machine. The relationship is directly between the attacker and victim, or to put it another way, there is no (supply) chain at all.

If the Kaseya incident doesn’t qualify as a supply chain attack, either from vendor to MSP, or from MSP to customers, then what does? We’ll begin by looking at six different types of attacks that have been called “supply chain”, even if some may be arguable.

1. Third-party software, whether you call them “libraries” or dependencies”:
   
   • A pure vulnerability in a third-party software package.
     Equifax is perhaps the highest profile example of this, as they were compromised via a vulnerable Apache Struts that was not patched in a timely manner. Some consider this a supply-chain attack.
     
   • Malicious code implanted in third-party software.
     This may come in the form of repo-jacking, dependency confusion / repo name squatting, or a malicious commit. These are arguably three different methods of obtaining the same result. The attack designed to steal the cryptocurrency Agama  is a good example of this.
     
   • Third-party software related service, rather than the software itself.
     The recent Codecov attack was a significant example of this.
     
     
2. A physical supply chain attack that involves implanting a malicious chip / module in a commercial product before it ships from the vendor.
   
   These have long been theorized and talked about, but the highest profile example, as covered by Bloomberg, has been disputed by professionals including vendors, journalists, government, and security experts. New York Times journalist Nicole Perlroth briefly talked about a case where the CIA “infiltrated factory floors”, but curiously offered no citations for the story.
   
   
3. An upstream vendor is compromised to push a malicious update.
   
   This type of attack relies on hijacking a software update service process so that when the software or device “phones home” to check for updates, it finds one and installs it, even if malicious. The recent Solarwinds compromise was performed this way as well as Russian hackers that launched an attack against Ukraine by compromising the update process of the M.E. Doc software.
   
   
4. Compromising a Managed Service Provider (MSP), either service or software, as previously discussed in the context of Kaseya.
   
   Both theories of how the attack happened would qualify here, one way or another. If Kaseya was compromised and the attackers pushed out a malicious update, it would count as #3 above. But then as the devices pushed out updates or additional malicious code to the computers managed by the Kaseya device, that would count as #4 here.
   
   Please note the distinction between MSP, Managed Security Service Provider (MSSP), and Remote Monitoring and Management (RMM); all provide similar or overlapping functionality but can ultimately be very different things. Kaseya VSA devices are considered to be an RMM product, while some of Kaseya’s customers are MSPs, and Kaseya is a software vendor.
   
   
5. Injecting malicious content via a Content Delivery Network (CDN).
   
   We all love those advertisements on almost every web site out there, and a vast majority are delivered by CDN. Compromising a CDN may be limiting as far as the type of content you can inject (e.g. “just” JavaScript), but will potentially reach more computers. A recent example of this is the January 2021 attack on NoxPlayer.
   
   
6. Compromising a contractor’s access to privileged resources.
   
   An attacker that can gain access to a company that provides operations support  services may have legitimate access to dozens or hundreds of customers via passwords, certificates, or tokens. Using that access the hackers can, in turn, compromise the target and potentially more. Some consider this a supply chain attack, as was the case with the Target incident.
   

With these six categories in mind, if we revisit those definitions we found in common use, you can begin to see a flaw in the logic. If a vulnerability in a third-party dependency can represent a supply chain attack, then tens of thousands of attacks have been carried out. While we cited Equifax and the Apache “Struts-Shock” vulnerability that was used, think back farther to the Heartbleed vulnerability and it may have been used against thousands of hosts. Looking at the last item in the list, if any stolen credentials represents a supply chain attack then thousands more have occurred.

To better qualify what constitutes a supply chain attack, Ax Sharma of CSO Online wrote an article on the topic and defined the term, using a very specific detail that we feel better limits what counts (bolded for emphasis):

“The umbrella term “supply chain attack” covers any instance where an attacker interferes with or hijacks the software manufacturing process (software development lifecycle) such that multiple consumers of the finished product or service are impacted detrimentally.”

When we modify the definition to require malicious intent, it means that #1a and #6 from the list above are not supply chain attacks. This is absolutely important to qualify as such attacks gain more and more attention or we run the risk of falling into the “threat intelligence” trap. That term means a lot of different things and security companies that try to do all of it often fall short on most.

Supply Chain Attack Origins

When it comes to supply chain attacks, Solarwinds dominated news cycles, but now the discussion will shift to Kaseya. Ignoring the current digital examples, where did these types of attacks originate? Was there a time before computer-based attacks where maybe the military or an economy suffered a “supply chain attack?”. It seems probably, but we haven’t found many documented examples in the real world.

Interestingly enough, the best non-computer example we could find was the man who bought 18,000 bottles of hand sanitizer last year with the intent to turn a profit (and thereby perhaps to attack the supply chain in his area as an intended physical disruption). But let’s get back to the digital realm.

The First Computer-based Supply Chain Attack

Aaron Bray at Phylum points out a great early example of a software supply-chain hack in Ken Thompson’s paper, “Reflections on Trusting Trust“. This seminal work set the tone for many aspects of security and risk modeling by making us question something that we should conceivably be able to trust. What if your own compiler was introducing malicious code at compile-time to produce backdoored software every time? While Thompson’s paper is heavily cited for this proposal, the same idea was put forth in 1974 by Karger and Schell in their paper “Multics security evaluation: Vulnerability analysis“:

“In Multics, most of the ring 0 supervisor is written in PL/1. A penetrator could insert a trap door in the PL/1 compiler to note when it is compiling a ring 0 module. Then the compiler would insert an object code trap door in the ring 0 module without listing the code in the listing. Since the Pl/1 compiler is itself written in Pl/1, the trap door can maintain itself, even when the compiler is recompiled.”

Page 55, “Multics security evaluation: Vulnerability analysis”

During their evaluation of a Multics system they demonstrated this type of trapdoor by placing it in the Multics system at MIT. In this 1974 paper the relationship between the MIT system and the vendor is not clear, but in the 1979 paper Schell clarified what happened:

“Trap door installed. The tiger team penetrated Multics and modified the manufacturer’s master copy of the Multics operating system itself by installing a trap door: computer instructions to deliberately bypass the normal security checks and thus ensure penetration even after the initial flaw was fixed. This trap door was small (fewer than 10 instructions out of 100,000) and required a password for use. The manufacturer could. not find it, even when he knew it existed and how it worked. Furthermore, since the trap door was inserted in the master copy of the operating system programs, the manufacturer automatically distributed this trap door to all Multics installations.”

Lieutenant Colonel Roger R. Schell

In summary, a 1974 red team formed by the United States Air Force penetrated the Multics operating system and inserted a backdoor that made it into Honeywell’s master copy. The backdoor was distributed to “all Multics installations” but fortunately, the backdoor was neutered and was only a proof of concept. It was not one that could be exploited. 47 years ago we saw the first computer-based supply chain attack as a proof-of-concept. That incident would be indirectly cited for decades to come before being essentially forgotten.

What 20 Years of Supply Chain Attacks Looks Like

In a recent article in Wired, Andy Greenberg goes into detail about the 2011 RSA breach. Based on interviews with several RSA employees who are no longer bound by NDA, the perspective on the breach and the resulting fallout is interesting. Since the cryptographic seeds of RSA’s two-factor authentication (2FA) key fobs were stolen, it mimicked a supply chain attack of sorts when the criminals used that information to allegedly compromise some of RSA’s customers. The article quotes Mikko Hypponen of F-Secure who was part of a third-party analysis of the breach saying “It opened my eyes to supply chain attacks.“

Given what we know of Karger and Schell, Ken Thompson, as well as how hackers in the late 80s and early 90s operated, this quote is a surprise. In those early days of compromising machines, the technology and infrastructure was immature when compared to today. While a supply chain attack was possible, it primarily would have created a malicious update that would be made available on a vendor’s FTP site.

This type of attack was largely before the days of automatic updates when patches were downloaded and installed by an administrator. Such attacks can be seen in 1994 for WU-FTPD and ircII being backdoored on the official distribution sites, again in 1997 for the AtlantiS IRC Script, in 1999 for TCP Wrappers and util-linux, and so on. Even bigger, more prominent software like OpenSSH and Sendmail were backdoored and distributed via their official sites in 2002. In fact, these incidents happened so many times that they were all lumped into a single CVE ID.

Here are some significant supply chain attacks within the last 20 years:

• 2001 – The Thruport CDN was compromised to deliver a custom GIF image that displayed on SecurityFocus. While the content was not malicious and only served to ‘deface’ their web page, it certainly could have been.
• 2011 – RSA was hacked in 2011 and the cryptographic seeds for their 2FA tokens stolen. They were allegedly used to attack some of RSA’s high-profile customers afterwards. To this day, RSA denies they were used for that purpose. However, some of the compromised organizations are adamant that was the source of compromise.
• 2012 – Flame is a self-propagating piece of complex malware that contained a cryptographic attack that allowed it to generate fake Microsoft security certificates. This was used to hijack the Windows Update service so that it could spread.
• 2013 –  The Dragonfly campaign subverted legitimate software on vendor websites that were used by ICS equipment in the energy sector. This represented a serious targeted attack by criminals on a specific sector.
• 2013 – In South Korea, bad actors compromised the auto-update mechanism of SimDisk, a file-sharing service with end-user software that impacted both government and news websites.
• 2015 – The Syrian Electronic Army hacked Instart Logic to inject malicious content into the InStart CDN, allowing them to cause pop-up alerts on the Washington Post website, informing visitors the site had been hacked.
• 2017 – Security company Avast suffered an attack that led to 2.27 million downloads of a trojaned version of CCleaner. The auto-update mechanism pulled in the new code and of those, approximately 1.65 million installations phoned home to the attackers who subsequently launched follow-up attacks. Even though only 40 were targeted, all of them were “technology and IT enterprise targets.“
• 2018 – A suspected Chinese-backed group stole two legitimate ASUS certificates which were used to push updates to almost a million ASUS systems in an operation dubbed ShadowHammer.
• 2018 – A vulnerability in the Unpkg CDN could have allowed an attacker to write arbitrary files on their servers, which could have been served up on thousands of websites. If abused, arbitrary JavaScript injection might have led to credential theft and more.
• 2019 – Researchers discovered the Magecart credit card skimmer on some Amazon CloudFront CDN servers. Instead of this being served up to hundreds or thousands of web sites, it only affected the customers hosting their own content there. Still, it has the potential to cause serious issues and highlights the risks of cloud security.
• 2019 – Attackers used a ConnectWise Control remote access tool to compromise the MSP, TSM Consulting. 22 town and county sites in Texas were then infected with ransomware.
• 2020 – Researchers discovered and analyzed the Octopus Scanner malware, which searched for and backdoored NetBeans projects hosted on GitHub and used the build process as a method for propagating itself.
• 2020 – In one of the most high-profile supply chain attacks, software company Solarwinds was compromised by what is believed to be APT29 (Cozy Bear/Russian SVR). Once inside, the attackers used a valid certificate to sign and then distribute malware via a software update.
• 2020 – Using default passwords found in software vendors of ICS products, the Kwampirs malware campaign exploited the situation to then inject and distribute a remote access trojan (RAT) through the supply chain leading to backdoor access.
• 2021 – The heavily used Codecov Bash Uploader was modified by a bad actor who was able to use access for over two months to gain access to information in users’ continuous integration (CI) environments.

This timeline shows a significant gap between 2001 and 2011. But despite the gap, many attacks during that time may have been carried out and gone unnoticed, or were simply not labelled as “supply chain” attacks. But post 2011 we see a steady series of incidents that highlights the growing frequency as well as a growing severity. The message is clear: while such attacks may require more time and technical skills, the potential payoff makes it worth it to malicious actors.