Introduction
When a researcher finds a security vulnerability that impacts more than one vendor, and they wish to coordinate disclosure with both, it creates a situation where an embargo must be put in place. In this context that simply means that all three parties agree not to make the information public until a given date. This is done to allow both vendors to have a fix ready before publication. Otherwise, if one vendor releases a fix before the other it can allow third parties to examine the fix and potentially determine it affects the second party that has not fixed it. Basically, it creates a window of opportunity where one vendor’s software is not patched and could open them up to attack.
This invokes various quotes and sentiments around secrecy. Perhaps the most infamous of which is a quote from Benjamin Franklin. The quote fits perfectly into the narrative of this blog because I argue that embargoes being broken in vulnerability disclosure coordination have happened for a long time, and continue to. Historically, embargo breaks could happen if a third party independently discovered a vulnerability and opted to disclose before all parties were ready. That absolutely is not a new factor that threatens these breaks, but that factor is now a much greater risk.
Disclosure embargos can work, but the odds are further stacked against it as the amount of parties participating grows. Two vendor embargoes will be much more likely to succeed where twenty vendors run an extremely higher chance of being broken prematurely. This problem is further compounded if one of the vendors is open-sourced, meaning their code and developer work is largely or entirely public. It just requires one vendor to publish a fix, e.g. via a commit landing in a developer branch, for someone to notice it and figure out it represents a security fix.
On the surface you might think an embargo would be easy to maintain. Afterall, how many people at a company need to know about it? Obviously the developer that is to fix the problem would be one or more people. But then consider a security team that receives the report, management of that team, and management of the developers. But then consider it could potentially include executives, legal, or compliance. One vulnerability report could be seen by a couple dozen people at minimum in such cases and that is at just one of the companies. Next, add any third-party coordination body like a national CERT. Finally, consider any malicious actors that can intercept any of those emails or may have any system compromised where the emails reside or a developer will work on the fix. That’s a lot of opportunity for the information to fall into the hands of someone that it was not intended for.
Beyond all of those concerns you have some vendors that are known for disregarding embargoes or not being as invested in the embargo as others. They receive the information, commit the fix quickly but publicly, and move on before other vendors are ready. You also have situations where a vendor might silently patch the vulnerability and push the fix out to customers thinking it is still private. The reality is that many legitimate security researchers as well as bad actors analyze patches to look for vulnerability fixes. That means the “silently” fixed vulnerabilities are no longer silent if someone takes the time to look.
Prior Embargo Break Examples
History is littered with embargo breaks in the realm of vulnerability disclosures. There are cases where a vendor broke their own embargo by mistake such as Phoenix Technologies and the ‘LogoFAIL‘ vulnerability. The vendor had published their security advisory on November 28, 2023 but removed it later. The embargo date for the LogoFAIL vulnerabilities was set for December 6, 2023, meaning they preempted the coordination by 8 days. In another case, Dell released a fix for an AMD BIOS vulnerability and linked to the “amd-sb-4002” security bulletin which did not mention the CVE. It was unclear if it represented an incorrect CVE assignment, a Dell-specific vulnerability, or if Dell prematurely broke embargo for an AMD vulnerability. Such cases make it difficult to know if an embargo was even broken which highlights the challenges in tracking the breaks.
In another instance, the “SegmentSmack” vulnerability (CVE-2018-5390) experienced an embargo break among multiple vendors that were to coordinate disclosure. In a similar case, fixes for the “Meltdown” and “Spectre” vulnerabilities had an embargo break when one vendor “quietly” released fixes in their public repository thinking that no one would notice it as a security fix. The “KRACK” vulnerabilities suffered an embargo break due to OpenBSD publishing fixes without regard to other vendors. The named vulnerability “Data Bounce” in Intel Processors, which still does not have a CVE assignment, experienced an embargo break because the vendor opted not to provide a fix. When the embargo release date was reached the researchers published their paper. A few years ago, the maintainer of the “forgejo” project outlined problems with their embargo workflow in a public bug ticket which gives a good view of the problems they faced.
A Recent, More Complicated Example (Dirty Frag)
In the last few weeks there have been a series of particularly bad vulnerabilities in the Linux Kernel disclosed. They share the same attributes: reliable, trivially exploited, and impacting a wide variety of Kernel versions. While several of them were released without any form of coordination, one in particular was intended to be embargoed. The “Dirty Frag” vulnerability disclosure embargo embodies several of the problems previously mentioned. The sheer number of vendors impacted by any Linux Kernel vulnerability makes a true, fully coordinated release impossible. In this case there were several players involved that led to the embargo breaking and introduction of confusion on the vulnerability.
The vulnerability was discovered by Hyunwoo Kim who reported his ‘Dirty Frag’ find to the Linux Kernel security team. At some point Kim determined that information had been made public despite the embargo. He posted about it on the OSS-Sec mail list saying “Because the embargo has now been broken” and then provides some technical details along with a functional exploit. In a reply, someone questions if there is a CVE assignment and asks how the embargo was broken. Greg K-H from the Linux Kernel security team replies providing the CVE ID but ignores the question about the broken embargo. Greg replies again providing a second CVE ID, still ignoring the embargo question.
Then on May 7, a researcher who goes by _SiCk replies explaining the embargo break. He outlines how a series of events led him to see a fixing commit, which others had noted previous to his own disclosure. In his release, he called the vulnerability “Copy Fail 2” which introduced some confusion initially, as for a short time it was easy to think that Copy Fail 2 was a distinct vulnerability from Dirty Frag. The timeline _SiCk provides spells out in clear detail why embargoes can break when an open source vendor is part of it:
As he often does, Brad Spengler (@spendergrsec on Twitter) linked to the commit without context which typically means it is a fix for a vulnerability despite no mention of that in the commit message. From here _SiCk was able to determine the nature of the vulnerability, technical details, and ultimately write a functional exploit. When he did that and published on May 7, that was five days before the embargo ended. In the post above he is clear that is how the embargo broke and that he had no privileged knowledge of the vulnerability.
Spengler is not the only one who does this with Linux Kernel commits. There are hundreds, likely thousands of researchers that do this for a wide variety of open source projects. There are hundreds more that effectively do the same, but using patches from closed source software like Apple or Microsoft. Instead of reading a commit, they use tools to ‘reverse’ the patch to determine what was fixed. Those tools have become powerful and make the task easier every year. That is why releasing a patch, something that seems helpful and benign on the surface, can ultimately cause a world of headache and pressure on other vendors and organizations.
It’s worth noting that once again, Spengler et al disprove Linux Kernel Security’s notion that “Only those individuals with deep expertise and intimate knowledge of the subsystem can effectively assess the validity and scope of a reported vulnerability and determine its appropriate CVE designation.” Just by skimming fixing commits to the Kernel source repository, many security researchers can quickly identify if they are vulnerability fixes. Some of them can go farther and add important details about requirements for exploitation, potential impact, and more.
The Dirty Historical Truth
There is another increasingly common event that can lead to an embargo break, if one is in place. But this event demonstrates that you can never assume that only one party knows of a vulnerability. Historically we have called it “‘researcher collisions” where two or more researchers find the same vulnerability independently. This has been surprisingly common for over 25 years now, primarily in higher visibility software. It’s logical that software from Microsoft, Apple, Oracle, Cisco, and other big vendors would be scrutinized more heavily.
I personally remember the early days of vulnerabilities disclosed in RealNetworks and Microsoft in particular, where it was not uncommon to see two to four independent parties that found the same vulnerability. Given the longer time vendors took to fix a vulnerability, it presented a bigger window and opportunity for more to discover the same thing. Jumping to 2009 I can find a great example of researcher collision in Microsoft Windows.
This vulnerability was discovered independently by at least six different researchers. I say “at least” because that is an important caveat. Those are the people we know discovered it but have to assume there were others that did as well. Instead of reporting the vulnerability to the vendor they sat on it or perhaps used it to compromise systems.
Anonymous – iDefense Labs VCP
Paule Byrne – NGS Software
Anonymous – Zero Day Initiative (ZDI)
Bing Liu – Fortinet
Dave Lenoe – Adobe Systems Incorporated
Will Dormann – CERT/CC
This historical glimpse is important to set the stage for what we’re going to see in the future.
The Future Is Worse
Given it is the future, how do we know it is worse? I’m not a fan of InfoSec predictions because most are blatantly obvious, or blatantly wrong. In this case it is less ‘prediction’ and more a very educated guess heavily based with historical evidence to back this claim. One of the primary reasons this happens is the tools used by security researchers keep evolving and improving. As they become more turnkey the faster the user finds the same thing as the next person because the tool is doing more and more of the heavy lifting.
With the advent of so-called AI, these general purpose tools are increasingly capable of reliably finding vulnerabilities. When you can point one of these tools to a source code repository and instruct it to “find vulnerabilities”, that is not promising for the future of vulnerability disclosures. In addition to volume it means that more researcher collisions will happen. That in turn will lead to attempted embargoes getting broken if one researcher decides not to coordinate disclosure. It means that threat actors using the same tools will find the vulnerability and potentially exploit it as a zero-day, and that will be the first we learn of it.
“AI” company Aisle has already published a blog this year about them recreating several Mythos findings using much cheaper tools. Granted, they used a shortcut to do it so quickly and cheaply since they had used vulnerable code previously identified. However, this methodology will be sound regardless of that prior knowledge. Anyone willing to spend time and money can set such a tool loose and use powerful hardware to speed things along.
We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis. Eight out of eight models detected Mythos’s flagship FreeBSD exploit, including one with only 3.6 billion active parameters costing $0.11 per million tokens. A 5.1B-active open model recovered the core chain of the 27-year-old OpenBSD bug.
Finally, the Reddit post that kind of spawned this blog! I have talked about researcher collisions for over 25 years and mentioned it in prior talks and blogs. It’s certainly not a new topic but with the advent of so-called AI tools that are designed to find vulnerabilities, we’re witnessing the next evolution of this phenomenon. Reddit user LeoRiley6677 posted on May 10th about their own efforts in using these tools to find vulnerabilities. They immediately concluded “Embargos are dead.” Here is the first paragraph of his post:
I spent a week analyzing how AI breaks two vulnerability cultures. Embargos are dead.
For the last week, I’ve been running through the historical logs of CVE disclosures and comparing them against the leaked architecture reports of Anthropic’s unreleased Mythos preview. Something fundamental has snapped in how we handle software security. We are watching the quiet death of the 90-day disclosure window.
It’s both refreshing and disheartening to see someone come to the same conclusion I have made for decades, but via using tools that will make the problem worse while increasing the volume of vulnerability disclosures. The potential coming flood is something this industry is not ready for. With the U.S. National Vulnerability Database (NVD) recently giving up on enrichment along with my general vote of “no confidence” in CVE, an increased flood of vulnerabilities will continue to put organizations at more and more risk.
As always, demand better from your Congressional representatives. Tell them that while the CVE ecosystem is severely deficient, it is the only thing most of the world has. The quality, timeliness, and usability of vulnerability data must improve.
Leave a Reply