In March 2021, Mandiant Managed Defense identified three zero-day
vulnerabilities in SonicWall’s Email Security (ES) product that were
being exploited in the wild. These vulnerabilities were executed in
conjunction to obtain administrative access and code execution on a
SonicWall ES device. The adversary leveraged these vulnerabilities,
with intimate knowledge of the SonicWall application, to install a
backdoor, access files and emails, and move laterally into the victim
The vulnerabilities are being tracked in the following CVEs:
Unauthorized administrative account
Post-authentication arbitrary file upload
Post-authentication arbitrary file read
Mandiant has been coordinating with the SonicWall Product Security
and Incident Response Team (PSIRT) for the responsible disclosure of
this information. SonicWall advises all customers and partners to
upgrade to the 10.0.9.6173 Hotfix for Windows users, and the
10.0.9.6177 Hotfix for hardware and ESXi virtual appliance users.
SonicWall Hosted Email Security product was automatically updated for
all customers and no additional action is required for patching
purposes. The hotfixes will also be superseded by the upcoming
SonicWall ES 10.0.10 release.
More information can be found by visiting the KB
article published by SonicWall.
All patches, upgrades, and hotfixes are available to download on the
Figure 1: SonicWall Email Security
ecosystem overview (via SonicWall)
SonicWall Email Security (ES) is an email security solution that “provides
comprehensive inbound and outbound protection, and defends against
advanced email-borne threats such as ransomware, zero-day threats,
spear phishing and business email compromise (BEC).” The
solution can be deployed as a physical appliance, virtual appliance,
software installation, or a hosted SaaS solution.
Figure 2: Sample SonicWall Email Security
Like many appliances, the solution provides a rich, web-accessible
administrative interface that serves as the main avenue for product
configuration. Depending on the customer’s deployment method, this
software is potentially capable of running under Windows or Unix
because it heavily leverages OS-independent Apache Tomcat and Java.
While the solution doesn’t require that this interface be exposed to
the internet, internet-wide scanning shows approximately 700 publicly
In March 2021, Mandiant Managed Defense identified post-exploitation
web shell activity on an internet-accessible system within a
customer’s environment. Managed Defense isolated the system and
collected evidence to determine how the system was compromised.
The system was quickly identified as a SonicWall Email Security (ES)
application running on a standard Windows Server 2012 installation.
The adversary-installed web shell was being served through the
HTTPS-enabled Apache Tomcat web server bundled with SonicWall ES. Due
to the web shell being served in the application’s bundled web server,
we immediately suspected the compromise was associated with the
SonicWall ES application itself.
When we contacted the customer, we learned that the installation of
SonicWall ES was the latest version available for download (10.0.9)
and that there was no publicly available information pertaining to
vulnerabilities or in-the-wild exploitation. To determine if a
potential application-level vulnerability was exploited to install the
web shell, Mandiant collected endpoint telemetry data.
We soon identified post-exploitation activity aimed at destroying
evidence on the system, executed in the context of the web shell. The
adversary executed the following command, shortly after installing the
cmd.exe /c “echo “” >
Figure 3: The Adversary clearing existing
entries in the current “webui.json” log
This command deleted the most recent application-level log entries
recorded by the SonicWall ES web application. While clearing log files
is a standard anti-forensics technique, understanding the location of
internal log files generated by applications is usually overlooked by
most spray-and-pray attackers. This added fuel to our suspicion that
we were dealing with an adversary who had intimate knowledge of how
the SonicWall ES application worked.
Fortunately for us, additional log files and a previously created
virtual server snapshot provided enough evidence to track down the
vulnerabilities and the adversary’s activities on the host.
Unauthenticated administrative access through improperly secured
The SonicWall Email Security application contains an authenticated
control panel to provide administration capabilities. One feature
available allows application administrators to authorize an additional
administrator account from a separate Microsoft Active Directory
Organization Unit (AD OU).
Figure 4: A redacted example of the vulnerable
endpoint associated with arbitrary user creation
Requests to this form, however, were not verified to require
previous authentication to the appliance.
Due to this vulnerability, an adversary with a well-crafted XML
document could either GET or POST their document to the application
and create a “role.ouadmin” account (Figure 4). This account could
then be used to authenticate to the application as an administrator.
Arbitrary file upload through post-authenticated “branding” feature
Like many enterprise products with a web-based user interface,
SonicWall Email Security includes a feature known as
“branding” which gives administrators the ability to
customize and add certain assets to the interface, such as company
logos. These branding assets are managed via packages, and new
packages can be created by uploading ZIP archives containing custom
text, image files, and layout settings. A lack of file validation can
enable an adversary to upload arbitrary files, including executable
code, such as web shells.
Once uploaded, these branding package ZIP archives are normally
expanded and saved to the <SonicWall ES install
path>databranding directory. However, an adversary could
place malicious files in arbitrary locations, such as a web accessible
Apache Tomcat directory, by crafting a ZIP archive containing a file
within a sequence of directory traversal notations such as in Figure 5.
Figure 5: Example ZIP archive containing
a Zip Slip web shell
It is important to note that the lack of validation which enables
Zip Slip attacks is not unique to SonicWall Email Security. As
detailed in Snyk’s
research on the topic, they exist within the many code libraries
from which applications have been built.
Directory-traversal leads to arbitrary file read in
post-authenticated “branding” feature
Mandiant confirmed another post-authentication vulnerability in the
administrative panel’s built-in “branding” feature which
allowed an adversary to retrieve arbitrary files from the host by
sending crafted HTTP GET requests to a particular resource. Figure 6
demonstrates the formatting of such request.
Figure 6: An example web request which results
in downloading the Windows calculator
While the working directory of this branding feature is <SonicWall ES install path>dataupdates,
a directory-traversal vulnerability allows an adversary to access
files located outside of this directory. As the Apache Tomcat
webserver handling this request is operating as the NT AUTHORITYSYSTEM account, any file on the
operating system can be accessed.
Combinations of all three exploits were leveraged interchangeably by
the adversary to perform the following actions:
- Creation of a new Administrator account on the SonicWall ES
- Exposure of the hashed passwords for existing, locally
configured Administrative accounts
- The creation of a web
shell in an arbitrary directory
- Real-time debugging of
exploitation success and failure
Upon obtaining administrative access to the appliance through
CVE-2021-20021, an adversary sent crafted HTTP requests to the
resource /dload_apps, a component of the
application’s “branding” feature, exploiting CVE-2021-20023.
These requests leveraged directory traversal attacks, enabling access
to two sensitive XML configuration files located at <SonicWall ES install
path>datamulti_accounts.xml and <SonicWall ES install
path>datamulti_ldap.xml, respectively (Figure 7).
Figure 7: HTTP GET requests exploiting CVE-2021-20023
These files contained details about existing accounts as well as
Active Directory credentials used by the application.
Next, the adversary uploaded a ZIP archive containing the BEHINDER
JSP web shell from the administrative panel’s “branding”
page. The crafted ZIP archive used a Zip Slip attack to exploit
CVE-2021-20022, which caused the web shell to be written to the
web-accessible Apache Tomcat directory <SonicWall ES install path>Apache Software
BEHINDER is a publicly available, multi-platform web shell that
accepts encrypted command and control (C2) communications. In
principle, BEHINDER operates similarly to CHINA
CHOPPER, a popular web shell that has been previously detailed by
FireEye. Like CHINA CHOPPER, an adversary operates a client-side
application to pass commands to the web shell within the body of HTTP
requests. As the core functionality of the backdoor is contained
within the client-side application, BEHINDER—much like CHINA
CHOPPER—has the added benefit of being small, with the variant
observed in this investigation weighing in at less than 1 kilobyte
Figure 8: The BEHINDER web shell observed
by Mandiant, which executes AES encrypted and base64 encoded commands
With the addition of a web shell to the server, the adversary had
unrestricted access to the command prompt, with the inherited
permissions of the NT AUTHORITYSYSTEM account.
After clearing the SonicWall application “webui.json” log file, the adversary escalated
their attack to credential harvesting in preparation of moving
laterally into the victim’s network. The adversary relied on “living
off the land” techniques rather than bringing their own tools into the
environment, which often has the benefit of potentially avoiding
detections from a security product.
We observed the adversary executing the reg
save command to dump the HKLMSAM,
HKLMSYSTEM, and HKLMSECURITY registry hives, which contain vital
information in recovering password hashes and LSA secrets.
Additionally, the adversary obtained in-memory sensitive credentials
through the use of built-in memory dumping techniques. The adversary
was observed invoking the MiniDump export of
the Windows DLL comsvcs.dll to dump both the
process memory for lsass.exe and the running
instance of Apache Tomcat as seen in Figure 9.
Figure 9: The adversary acquiring process memory
for lsass.exe (MITRE ATT&CK T1003.001) and Apache Tomcat
Mandiant typically observes adversaries employing short and
easy-to-type filenames during their operations, simply for efficiency.
As such, the aforementioned filenames initially stood out as being
peculiar, as a mix of case and symbols would require more effort to
type than is often necessary. While this could always be indicative of
a tool being used, the slight variations between the two commands—the
absence of a comma before the DLL export and the uppercase C: drive in the second—suggest that they were
manually typed. Considering that the C:WindowsTemp directory on a Windows host also
normally contains numerous similarly named temporary files, the
adversary was likely taking extra care to evade suspicion should the
activity reach the screen of a security analyst.
Continuing their effort to live off the land as much as possible,
the adversary located a copy of the archiving utility 7-Zip already
present on the host and used it to compress a subdirectory of <SonicWall ES install path>dataarchive.
This directory contains daily archives of emails processed by
SonicWall ES—again demonstrating the adversary’s familiarity with the application.
After a several-day lull in activity, the adversary returned to the
host, presumably after working to recover passwords from the registry
hives and process memory that was dumped earlier. At the time of
activity, the victim organization was using the same local
Administrator password across multiple hosts in their domain, which
provided the adversary an easy opportunity to move laterally under the
context of this account—highlighting the value of randomizing
passwords to built-in Windows accounts on each host within a domain.
We observed the adversary leveraging Impacket’s publicly available
tool to access several internal hosts, which enabled remote command
execution over Microsoft’s DCOM protocol via Windows Management
Instrumentation (WMI). The adversary managed to briefly perform
internal reconnaissance activity prior to being isolated and removed
from the environment.
Mandiant currently tracks this activity as UNC2682. Ultimately,
Mandiant prevented UNC2682 from completing their mission so their
objectives of the attack currently remain unknown.
Each investigation conducted by Mandiant includes analysts from our
Advanced Practices team who work to correlate activity observed in the
thousands of investigations that Mandiant responds to. At times, we do
not have the data available to directly attribute intrusion activity
to a previously known group. In these cases, we create a new UNC group
to track the activity that we observed. An UNC group is a cluster of
related cyber intrusion activity, which includes observable artifacts
such as adversary infrastructure, tools, and tradecraft, that we are
not yet ready to give a classification such as APT or FIN.
For more details on how Mandiant uses UNC groups, see our blog post:
Attribution: How Mandiant Tracks Uncategorized Threat Actors.
Investigation & Monitoring Tips
Mandiant recommends monitoring of the following endpoint telemetry
indicators for potential evidence of compromise:
- Child processes of the web server process “tomcat” on
SonicWall Email Security appliances, particularly cmd.exe
- The creation or existence of web shells on a server hosting
SonicWall Email Security
In addition to standard indicators, Mandiant recommends reviewing
SonicWall-related internal configuration files and logs for evidence
of previous adversary activity.
Evidence of malicious web requests and their values may be
identifiable in the following log files:
- The Apache Tomcat logs:
FilesSonicWallESApache Software FoundationTomcat
- The SonicWall application logs:
- C:Program FilesSonicWallESlogswebUIwebui.json
Evidence of unauthorized modifications to SonicWall configuration
settings can be confirmed in the following files:
- The administration user account file:
- Additional user account files that may have been created in the
related zip files in any of the subdirectories of the following
Detecting the Techniques
FireEye detects this activity across our platforms. The following
contains specific detection names that provide an indicator of
SonicWall ES exploitation or post-exploitation activities associated
with this adversary.
FireEye Endpoint Security
FireEye Network Security
FireEye Email Security
FireEye Detection On
FireEye Malware File Scanning
Additionally, SonicWall has deployed Intrusion Prevention System
(IPS) signatures to SonicWall firewalls to help detect and block
attacks that attempt to leverage the aforementioned vulnerabilities.
The following signatures have already been applied to SonicWall
firewalls with active security subscriptions:
IPS Signature: 15520 WEB-ATTACKS SonicWall Email Security
IPS Signature: 1067 WEB-ATTACKS Web Application Directory
Traversal Attack 7
IPS Signature: 15509 WEB-ATTACKS Web Application Directory
Traversal Attack 7 -c2
Mandiant Security Validation Actions
Organizations can validate their security controls using the
following actions with Mandiant
Malicious File Transfer –
Web Shell Activity – BEHINDER,
Malicious File Transfer – Zip
Phishing Email – Malicious
Mandiant disclosed the vulnerabilities CVE-2021-20021 and
CVE-2021-20022 to SonicWall Product Security Incident Response Team
(PSIRT) on March 26, 2021. The vulnerabilities were acknowledged and
validated on March 29, 2021 and a hotfix became available on April 9,
2021. The patch was communicated to impacted SonicWall customers and
partners on April 9, 2021.
Mandiant disclosed the
vulnerability CVE-2021-20023 to SonicWall PSIRT on April 6, 2021. The
vulnerability was acknowledged and validated on April 9, 2021 and a
patch became available April 19.
To mitigate the three CVEs, Mandiant and SonicWall recommend
upgrading Email Security to version 10.0.9.6173 (Windows) or
10.0.9.6177 (Hardware & ESXi Virtual Appliances). Organizations
using SonicWall Hosted Email Security (HES) products were
automatically updated and no action is required for those customers.
SonicWall PSIRT, Charles Carmakal, Ben Fedore, Geoff Ackerman and