Apache Software Foundation Security Report: 2020

The state of security across all Apache Software Foundation projects for the calendar year 2020

Wilmington, DE, Jan. 25, 2021 (GLOBE NEWSWIRE) — Title: Apache Software Foundation Security Report: 2020

Synopsis: This report explores the state of security across all Apache Software Foundation projects for the calendar year 2020. We review key metrics, specific vulnerabilities, and the most common ways users of ASF projects were affected by security issues.

Released: January 2021

Author: Mark Cox, Vice President Security, Apache Software Foundation

Background

The security committee of the Apache Software Foundation (ASF) oversees and coordinates the handling of vulnerabilities across all of the 340+ Apache projects.  Established in 2002 and composed of all volunteers, we have a consistent process for how issues are handled, and this process includes how our projects must disclose security issues.

Anyone finding security issues in any Apache project can report them to security@apache.org where they are recorded and passed on to the relevant dedicated security teams or private project management committees (PMC) to handle.  The security committee monitors all the issues reported across all the addresses and keeps track of the issues throughout the vulnerability lifecycle.  

The security committee is responsible for ensuring that issues are dealt with properly and will actively remind projects of their outstanding issues and responsibilities.  As a board committee, we have the ability to take action including blocking their future releases or, worst case, archiving a project if such projects are unresponsive to handling their security issues.  This, along with the Apache Software License, are key parts of the ASF’s general oversight function around official releases, allowing the ASF to protect individual developers and giving users confidence to deploy and rely on ASF software.  

The oversight into all security reports, along with tools we have developed, gives us the ability to easily create metrics on the issues.  Our last report covered the metrics for 2019.

Statistics for 2020

In 2020 our security email addresses received in total 18,000 emails. After spam filtering and thread grouping this was 946 (2019: 620) non-spam threads.  Unfortunately many security reports do look like spam and so the security team are careful to review all messages to ensure real reports are not missed for too long.

Diagram 1: Breakdown of ASF security email threads for calendar year 2020

Diagram 1 gives the breakdown of those 946 threads.  257 threads (27%) were people confused by the Apache License.  As many projects use the Apache License, not just those under the ASF umbrella, people can get confused when they see the Apache License and they don’t understand what it is.  This is most common for example on mobile phones where the licenses are displayed in the settings menu, usually due to the inclusion of software by Google released under the Apache License.  We no longer reply to these emails. This is nearly double the number we saw in 2019.

The next 220 of the 946 (23%) are email threads with people asking non-security (usually support-type) questions.

The next 93 of those reports were researchers reporting issues in an Apache web site.  These are almost always false negatives; where a researcher reports us having directory listings enabled, source code visible, or the lack of various domain headers.  These reports are generally the unfiltered output of some publicly available scanning tool, and often where the reporter asks us for some sort of monetary reward (bounty) for their report.

That left 376 (2019: 320) reports of new vulnerabilities in 2020, which spanned across 101 of the top level projects.  These 376 reports are a mix of both external reporters and internal; for example where a project has found an issue themselves and followed the ASF process to assign it a CVE name and address it we’d still count it here.  We don’t keep metrics that would give the breakdown of internal vs external reports.

The next step is that the appropriate project triages the report to see if it’s really an issue or not.  Invalid reports and reports of things that are not actually vulnerabilities get rejected back to the reporter.  Of the remaining issues that are accepted they are assigned appropriate CVE names and eventually fixes are released.

As of January 1st 2021, 35 of those 376 reports were still under triage (i.e. the project had not yet determined if the report is accepted or rejected).  

The remaining closed 341 (2019: 301) reports led to us assigning 151 (2019: 122) CVE names.  Some vulnerability reports may include multiple issues, some reports are across multiple projects, and some reports are duplicates where the same issue is found by different reporters, so there isn’t an exact one-to-one mapping of accepted reports to CVE names.  The Apache Security committee handles CVE name allocation and is a Mitre Candidate Naming Authority (CNA), so all requests for CVE names in any ASF project are routed through us, even if the reporter is unaware and contacts Mitre directly or goes public with an issue before contacting us.

Noteworthy events

During 2020 there were a few events worth discussion; either because they were severe and high risk, they had readily available exploits, or otherwise due to media attention.   These included:

• February: An issue in Tomcat CVE-2020-1938 gained press interest when it was given branding and a name (“Ghostcat”) and was disclosed by a third-party coordination centre before Tomcat released an advisory (although after the issue was fixed in new releases of Tomcat). Although serious if exploited, it only affected Tomcat installations which exposed an unprotected AJP Connector to untrusted networks (which is already not a good thing to do even without this issue). That limits the number of affected installations.  Various proof-of-concept exploits are public for this issue, including a Metasploit exploit.

• May: The Cybersecurity and Infrastructure Security Agency (CISA) released a list of Top 10 Routinely Exploited Vulnerabilities including CVE-2017-5638, the remote command execution (RCE) vulnerability in Apache Struts 2 disclosed and fixed in 2017.  This issue is known to be exploited in the wild, however the first exploitation was discovered after the advisory and fix was published. 

• July: Versions of Apache Guacamole 1.1.0 and earlier were vulnerable to issues in RDP, CVE-2020-9497 and CVE-2020-9498.  If a user connects to a malicious or compromised RDP server it could lead to memory disclosure and possible remote code execution. 

• August: A vulnerability in Apache Struts (CVE-2019-0230) could lead to arbitrary code execution. In order to exploit the vulnerability, an attacker would need to inject malicious Object-Graph Navigation Language (OGNL) expressions into an attribute that is used within an OGNL expression. Although Struts has mitigations to address potential injected expressions, versions before 2.5.22 left an attack vector open which was fixed in updates for this issue.  A metasploit exploit exists for this issue.

• November: Previously each ASF project was responsible for writing up their own CVE entries and submitting them to Mitre. This leads to many delays in the CVE database being updated with Apache issues as entries are often rejected as the legacy format causes issues. We released an internal tool providing projects dealing with security issues a way to edit, validate, and submit their entries to Mitre.  We aim to have the CVE database updated within a day of an issue being published.

• December: The CVE project released a new automation API and the ASF became the first organisation to get a live CVE name using it. Instead of the security team holding a pool of names requested in advance we now allocate them on demand, with the service taking care of emails to the PMC and other previously manual parts of the process. We expect more automation available during 2021 allowing us to streamline the CVE process for projects even further.

• Proof of Concepts or Metasploit exploits were also released for 2020 issues in Apache OFBiz (CSRF, CVE-2019-0235), Apache OpenMeetings (DoS, CVE-2020-13951), Apache Flink (arbitrary read/write RCE CVE-2020-17518, CVE-2020-17519)

Timescales

Our security teams and project management teams are all volunteers and so we do not give any formal SLA on handling of issues.  However we can break down our aims and goals for each part of the process:

Triage: Our aim is to handle incoming mails to the security@apache.org alias within three working days.  We do not measure or report on this because we assess the severity of each incoming issue and apply the limited resources we have appropriately.  The alias is staffed by a very small number of volunteers taken from the different project PMCs.  After the security team forward a report to a PMC they will reply to the reporter.  Therefore if you have reported an issue to us and not received any response after a week please send us a followup email.  Sometimes reporters send reports attaching large PDF files or even movies of exploitation that don’t make it to us, so please ensure any follow ups are a simple plain text email.

Investigation: Once a report is sent to the private list of the projects management committee, the process of triage and investigation varies in time depending on the project, availability of resources, and number of issues to be assessed.  As we send reports to this private list it does not reach every project committer, so there is a much smaller limited set of people in each project able to investigate and respond.  As a general guideline we try to ensure projects have triaged issues within 90 days of the report.  The ASF security team chase any untriaged issues over 90 days old.

Fix: Once a security issue is triaged and accepted, the timeline for the fixing of issues depends on the schedules of the projects themselves.  Issues of lower severity are most often held to future pre-planned releases.

Announcement: Our process allows projects up to a few days between a fix release being pushed and the announcement of the vulnerability, to let mirrors catch up.  All vulnerabilities are announced via the announce@apache.org list.  We now aim to have them appear in the public Mitre list within a day of the announcement.

Conclusion

Apache Software Foundation projects are highly diverse and independent.  They have different languages, communities, management, and security models.  However one of the things every project has in common is a consistent process for how reported security issues are handled. The ASF Security Committee works closely with the project teams, communities, and reporters to ensure that issues get handled quickly and correctly.  This responsible oversight is a principle of The Apache Way and helps ensure Apache software is stable and can be trusted.

This report gave metrics for calendar year 2020 showing from the 18,000 emails received we triaged over 370 vulnerability reports relating to ASF projects, leading to fixing 151 (CVE) issues.  The number of non-spam threads dealt with was up 53% from 2019 with the number of actual vulnerability reports up 13% and assigned CVE up 24%.

If you have vulnerability information you would like to share with or comments on this report please contact us.

# # #

CONTACT: Sally Khudairi
Vice President
The Apache Software Foundation
press(at)apache(dot)org