[This was originally published on RiskBasedSecurity.com.]
Google announced a new initiative called Project Zero. The basic premise of the project was that Google invests heavily in their own security and had for quite some time been also tasking their researchers part time work on improving the security of other high-profile products.
Project Zero is Google’s investment into a well-staffed team that they hoped to get the ball rolling as they felt that people “should be able to use the web without fear that a criminal or state-sponsored actor is exploiting software bugs to infect your computer, steal secrets or monitor your communications.” Recently, there have been questions that have been raised wondering if Google’s Project Zero is really unfair towards and targeting Microsoft?
From reading several news articles as well as the blog from Microsoft’s Chris Betz, who runs their Product Security Response, you would definitely think that this is the case. Since Project Zero was announced, we at Risk Based Security have been following it closely as we were interested in how it might influence vulnerability research. More specifically, we were curious if the project would help improve the code that many rely on such as third-party libraries and if it might ultimately impact the Bug Bounty economy.
With all of the recent conversations about Project Zero, we decided to publish some of our own analysis of with an attempt at using data to see if there is a bias towards targeting Microsoft, or any other company. One of the great things about Google and Project Zero is that they are generally pretty transparent with their research and try to clearly communicate their intentions about disclosure as well. But have they really been clear with their disclosure policy?
We know there is a 60-day Google company-wide intention for security disclosures that was published back in July, 2010. But what about the policy for Project Zero? We aren’t aware of a clear policy that has been published for the project or any sort of blog post on the requirements (nothing on their main blog, or the wiki in their tracker). In a Wired article last year, it mentions that Google staff stated:
“… they’ll alert the company responsible for a fix and give it between 60 and 90 days to issue a patch before publicly revealing the flaw on the Google Project Zero blog. In cases where the bug is being actively exploited by hackers, Google says it will move much faster, pressuring the vulnerable software’s creator to fix the problem or find a workaround in as little as seven days.”
Based on reading the individual tickets you can see that they have been communicating the 90-day policy since ID 44, opened on Jul 9, 2014 and since ID 89they have been very consistent about telling all vendors about the 90 day deadline and started automatically including the boilerplate text:
“This bug is subject to a 90 day disclosure deadline. If 90 days elapse without a broadly available patch, then the bug report will automatically become visible to the public.”
In the initial announcement of Project Zero, there was no mention of a hard line of 90 days for a disclosure timeline, but it does mention they want to get things fixed in a “reasonable time”:
“Once the bug report becomes public (typically once a patch is available), you’ll be able to monitor vendor time-to-fix performance, see any discussion about exploitability, and view historical exploits and crash traces. We also commit to sending bug reports to vendors in as close to real-time as possible, and to working with them to get fixes to users in a reasonable time.“
Chris posted a blog update in November 2014, where he highlighted many bugs that were fixed well within Project Zero’s 90-day disclosure deadline and that it represented a solid security response by the teams responsible for Adobe Flash, Microsoft Office, Internet Explorer, and Microsoft Windows. A more important point to mention is the following:
We calibrate our deadline to balance two important factors: the right of end users to know about a risk versus giving vendors a fair chance to cleanly fix a bug under a reasonable embargo. Given the ratio of deadline hits to misses, we think we’ve calibrated correctly for the current state of the industry but we do hope to tighten the deadline in the future.”
It seems that they could benefit from posting a formal policy at some point in the future to clarify their disclosure process, but regardless, they have been making vendors well aware of their expected timelines. What we can say with certainty is that by publishing details about their research, it allows us to get a better insight into their work, their focus, as well as a limited view of their interaction with vendors.
Show Me The Data
Project Zero has a tracker that they show their work in, and in some cases publish very detailed information for us to see their process. As of January 25, they have assigned 206 IDs in their tracker. One aspect that was not clear is the reason that so many IDs, especially earlier assignments, were restricted, meaning they were not open for viewing to the public. We speculated that these restricted IDs were ones that Google was not sharing for some specific reason, as they could have just been IDs that have been abandoned, or they were still being resolved with the affected vendors.
Considering they still have “Issue 1: This is a test” public, it didn’t seem that they were cleaning up older entries that are not relevant. We decided to reach out to Chris Evans who runs the Project Zero team asking for clarification on why so many entries were restricted, and he responded very quickly offering some great insight on the situation. Chris shared that most of the early IDs that are shown as restricted were due to a bunch of spam reports that were received when the project was officially launched. He mentioned that there are a few other causes of early bugs not being made public.
- The first few IDs were not filed as well as the project team would have liked, so they filed a new ID again and marked the earlier issues as invalid.
- A couple more bugs were marked as duplicates because they unfortunately filed the same report twice.
- There were a few more issues that he shared were valid, but just needed to be made public. In fact, our mail prompted him to open up (ID #125) which he described as “a very interesting Flash bug that has been fixed several days ago”. We thank Chris for opening it up tonight, even though it made us redo our statistics!
- And finally, as we expected, every ID currently from 144 onwards is still less than 90 days old which aligns with their stated 90 day disclosure timeline.
When looking at the 206 Project Zero IDs, the breakdown of tickets is as follows:
| ID Status | # of Vulnerabilities |
| Total Fixed | 83 |
| Total WontFix | 4 |
| Total Restricted | 73 |
| Total Invalid | 26 |
The “WontFix” designation means that the vendor does not consider the reported issue to be a security problem, or does not cross privilege boundaries. In terms of the severity and impact to organizations of the vulnerabilities that they have discovered, we looked quickly at the VulnDB assigned CVSSv2 score for each ID.
For those that don’t like seeing just a CVSSv2 average (as there are currently still a lot of issues with this version, and a new version hopefully around the corner), we pulled more data from VulnDB to help us understand Exploit availability.
Note that the high percentage of Proof of Concept (PoC) is largely due to Google researchers developing them to demonstrate the vulnerability, which typically makes it easier for the vendor to diagnose and understand the problem.
Microsoft
As mentioned, there are theories being voiced that Google is specifically targeting Microsoft and not practicing Coordinated Vulnerability Disclosure, putting customers at risk. A post from ErrataRob reminds us that Microsoft seems to be enjoying a double standard with their views on vulnerability disclosure. Additionally, it has been noted that Microsoft’s latest plea for CVD is as much propaganda as sincere, in a response to Microsoft’s Betz’s article published on the OSVDB blog. It is important to understand where this perception comes from.
Articles posted like the one from PC World that talk about a “third unpatched Windows zero-day” being released within a month is polarizing in the context of the vulnerability disclosure debate. It also makes it sound like this is a straight rivalry with one party intent on hurting the other. From what we can see by analyzing the data, this doesn’t appear to be the case. The disclosures are a byproduct of Google’s stated policy of disclosing after 90 days, fix or no fix. The use of “0day” denotes that Google published full details of the vulnerability (after ~90 days) without a vendor-supplied fix available.
| Statistics | Metric |
| Total 0Day | 19 Vulnerabilities |
| Average Disclosure Time | 59.68 Days |
It is critical to remember that not all vulnerabilities are created equal, even within the same organization. While Microsoft may be able to patch a particular product such as Office or Internet Explorer quickly, they may not be able to do the same for Windows depending on the numbers of platforms and versions affected. The life cycle of a privately disclosed vulnerability varies wildly as it has to be confirmed, patched, and then heavily tested.
Organizations want and really need Microsoft to test those patches adequately before releasing, or you run into updates that can do more harm than good. So this supposed “vulnerability disclosure war” as some are calling it between two companies is also mixing in one Google employee’s spare time activity with his day job. No matter how many times someone may say that “their thoughts and tweets do not represent their employers”, it can be a very fine line specifically when you are a researcher working for a company where a strong focus is on discovering vulnerabilities in other company’s products. It can lead to questions as to what constituents their employer’s versus their own time, and is it decided who has ownership when it is more convenient. This is especially true since that researcher’s day job and hobby are heavily intertwined.
In doing further analysis on the Google Project Zero tracker, we find that they have disclosed 20 vulnerabilities in Microsoft products as of this blog. Of those 20, Microsoft labeled 4 as ‘WontFix’ (they don’t consider them to be a security issue, and Google actually agrees with most), 13 were fixed within 90 days, and 3 were released without a fix due to the 90 day deadline being exceeded (the dreaded 0day!). On the surface, considering ~81% have been handled in a coordinated fashion that actually makes it seems Microsoft is doing pretty well. Not only have they gone through the process with Google many times already, they are obviously well aware of the 90 day disclosure deadline.
While it is still debatable that Google should provide extra time for companies when requested, it has become clear that this is not an option in the eyes of Project Zero. Since Microsoft now knows that Google is basically not flexible on that deadline, the onus is solely placed on them moving forward. If Microsoft does not want to have a 0day disclosed by Project Zero they are going to have to determine how to improve their response timeline and potentially dedicate more resources to a given vulnerability than they have for the three that were not fixed in time. It would be hard for anyone at Microsoft to argue that producing secure products isn’t critical and that they are short on resources to correct known vulnerabilities that may impact their customers. But the real question that begs to be asked; is Microsoft all alone in this supposed ‘war’ and being unfairly picked on by Google? Again lets take a quick look at the data to help us answer the question:
| Vendors | # of Vulnerabilities |
| Apple | 39 |
| Adobe | 36 |
| Microsoft | 20 |
| Linux Kernel | 7 |
| LibreSSL | 2 |
| glibc | 1 |
| PHP | 1 |
| Red Hat | 1 |
The data suggests that there is a simple answer; of course Microsoft is not being solely targeted. In fact, just a few days ago, Project Zero hit the 90 day deadline on a trio of Apple Mac OS X issues that went public without a fix. Ars Technica covered it in the same style as other articles covered the “0day” dropping on Microsoft. We found it very interesting that there is no mention in this article that these aren’t the first Google Project Zero “0day” dropped against Apple. In fact, between July 4, 2014 and September 30, 2014, Google released eleven vulnerabilities in Apple products without a patch. While Apple and Microsoft have each had “0day” issue due to missing the timeline.
We are also able to see that Project Zero has sent a substantial amount of vulnerability reports to the Adobe Flash team and they’ve hit all the deadlines, which is very impressive. That was the initial reason that prompted Brian Martin, our Director of Vulnerability Intelligence to really dig even further than we had previously into all of the public Project Zero vulnerabilities. The analysis focused on the vendor, vendor notification date, disclosure date (when the vulnerability was disclosed, which was by Google for some, and by the vendor for others), CVSSv2 score and exploit availability.
Conclusion
From our analysis of available information, we believe it removes some of the emotional arguments and allows one to make their own educated statements which appear to support that Google is not singling out Microsoft with their Project Zero efforts. Furthermore, it shows that Apple has received the most attention with Adobe close behind, both of whom have received considerably more attention than Microsoft. It just appears at this point, Microsoft is the only vendor that is failing to meet the 90 day disclosure deadlines sometimes, and then publicly criticizing Google for their work (or as some may say, providing free security work to improve their products and ultimately ensure their customers are better protected).
We sincerely hope that cooler heads will prevail as we all work together to better secure our organizations and protect consumer confidential information. Google’s work at Project Zero is impressive, and clearly making a positive impact on the security of high-profile products. We hope that companies will continue to work with Google and understand the importance of fixing issues as soon as possible to reduce the time of exposure customer’s face. But as with all policies, there are times when some level of flexibility does make sense when the purpose of the project is to improve security.
If you look at companies that Google has recently acquired, such as Dropcam, even their disclosure policy states, “We ask that you give us a reasonable amount of time to respond to your report before making any information public”. Of course, “reasonable amount of time” has been the cornerstone of the coordinated vulnerability disclosure debate for decades and doesn’t appear to be going away anytime soon.
[Update: Since starting a dialogue with Chris Evans, 24 issues have been published in the last 15 hours. Some of the stats quoted in this blog are already considerably different (e.g. Time to disclose). Further, we are now tracking additional information with each entry in order to generate more metrics.]