Unknown Therat s: The Achilles Heel of Email Security€¦ · popular email security solutions. The...
Unknown Threats: The Achilles Heel of Email Security March 2020
Transcript of Unknown Therat s: The Achilles Heel of Email Security€¦ · popular email security solutions. The...
Unknown Threats: The Achilles Heel of
Email SecurityMarch 2020
Contents
Executive Summary
Motivation
Study Description
The Results
Discussion and Implications
Appendix: Technical Challenges Resolved During The study
About BitDam
2
3
4
6
10
12
17
Executive Summary
All enterprises rely on email security products to protect their email. Unfortunately, malicious files and links regularly bypass the leading email security products, which leaves enterprises vulnerable to email-based attacks including Ransomware, Phishing and malware leading to Data Breaches. Our experience at BitDam, and the proliferation of threat variants fueled by automation, led us to suspect that email security products struggle to detect threats they encounter for the first time (aka Unknown Threats). Therefore, we have conducted an empirical study aimed at measuring how well these products handle Unknown Threats at First Encounter.
Our findings show that email security systems, such as Microsoft’s Office365 ATP, G-Suite Enterprise and ProofPoint TAP, have a high miss rate of 20-40% for unknown threats at first encounter. In fact, up to 45% of threats bypass at least one of these leading products. Furthermore, it takes them 10-53 hours to start protecting against the threats they miss. This Detection Gap means that enterprises are continually unprotected against unknown threats, allowing successful Ransomware, Phishing and Data Breaches. We describe the root cause of this inherent limitation of email security systems, and suggest threat-agnostic protection technologies (that do not require knowledge about threats) as an effective remedy. BitDam’s Advanced Threat Protection (ATP) solution is threat-agnostic, has a very low miss rate of unknown threats, significantly reducing the risk of successful email-based attacks.
Current data from the continuous study is available here.
Motivation
Most cyber-attacks start with an email bearing a malicious file or link. Enterprises use an email security product to protect their email, but are still vulnerable to email-based attacks including Ransomware, Phishing and data breaches. As threat actors continue to develop their attacks to be stealthier, malicious files bypass the leading email security products on a regular basis, landing in mailboxes.
BitDam provides an Advanced Threat Protection (ATP) solution that blocks content-borne threats (files and links) on multiple enterprise collaboration channels (email, cloud drives, messaging). As part of our work with customers we have seen first-hand that email security is bypassed by malicious content. First, being installed as a last line of defense, our solution detects malicious attachments that slip through various Secure Email Gateways (SEGs). Additionally, when customers try our Breach & Attack Simulation (BAS) tool for simulating email attacks, the results are consistent – all the email security products score poorly and are unable to block the majority of the samples sent during the BAS tests.
It should also be noted that increased use of automation allows attackers to create many variants (mutations) for a malware or malicious file, potentially inundating email security products with new unknown threats. As these products rely on threat data for detection, a significant increase in the numbers of unknown threats may impair their efficacy.
Our own experience described above, together with the proliferation of threat variants, suggests that the problem with the leading email security products is that they are struggling when it comes to detecting Unknown Threats, i.e., threats they encounter for the first time. Thus, we decided to conduct an empirical study aimed at measuring just how well email security systems handle unknown threats at first encounter. The unknown threats we use are fresh, live samples of malicious files. We measure Miss Rate at First Encounter and Time To Detect (TTD) for unknown threats. This type of analysis is not generally available to potential customers of popular email security solutions.
The findings show that email security systems, such as Microsoft’s Office 365 ATP, G-Suite Enterprise and ProofPoint TAP, have a Miss Rate at First Encounter in the range of 20-40% for unknown threats. Their Time To Detect is in the range of 10-53 hours on average. This observed Detection Gap means that enterprises are unprotected against many specific unknown threats every day, explaining the increasing success of Ransomware, Phishing and Data Breaches. We believe that the handicap of email security with regard to unknown threats is inherent, and stems from their complete reliance on knowledge about a threat. If they already ‘know’ a threat than they can detect it; if not, then they go through a process designed to turn the Unknown into a Known. This process, described in the Discussion section takes a significant amount of time to run its course, creating a window of opportunity for the attacker and risk to the enterprise.
In this paper we will describe the study and how it is being conducted. We will present the data, and discuss its implications. Finally, we will suggest an explanation for what we see as a fundamental limitation of ‘knowledge-driven’ email security systems, and propose how ‘threat-agnostic’ technologies can overcome it to provide protection against all unknown threats.
Study Description
Our empirical study has been in progress since October 2019. Thousands of ‘fresh’ malicious file samples, taken from various sources, have been sent to mailboxes protected by Office 365 ATP, G-Suite enterprise, and ProofPoint TAP. The fresh samples are considered Unknown Threats, for which we want to measure Miss Rate and Time To Detect.
In this study we use real functional (live) email addresses in an internet environment that are fully protected by the security products mentioned above. The study uses the following process:
� Continuously Obtain fresh samples of malicious files
� Qualify the samples: validate that they are malicious; modify them sufficiently so that they will not be blocked when we send them out
� For each sample: Send an email containing it (the malicious item) to the mailboxes protected by the security products covered in this study
� Monitor all mailboxes to log which sample made it to the inbox
� If a sample is not detected the first time it is sent (i.e., at first encounter), Re-send it until it is eventually detected
� Collect all the data needed for measuring the Miss Rate at First Encounter and the Time To Detect (TTD)
� Analyze the data per email security product
� Qualifying the samples
The files we send are chosen from live traffic that we have detected and flagged as malicious. The file types cover any non-executable files that cause malicious code to run on the device (Ransomware, password stealers, data breaches, APT and so on). These are mainly Office files and PDF files.
Due to a concern that False Positives (FPs) might lead to private data being exposed, we first verify that the file is on a public source before sending it out. A small percentage of the files has to be verified manually by our research team.
We also verify that each file was uploaded to the public source recently. Our goal is to minimize the time between when the sample is first seen in the wild and the time it is sent to the target mailbox. The reasoning behind this is that vendors such as, Google, Microsoft, Proofpoint, can see trends before anyone else, and could potentially be able to examine them before we are able to receive them.
� Configurationoftargetmailboxes
In general, we do not control the exact configuration1 of the mailboxes we send the samples to, since they are configured by the customer. This ensures that our study is based on real-world scenarios.
An Office ATP mailbox is configured as appropriate for an organization with reasonable security control. We use instances in Europe and in the USA. All mailboxes are configured such that malicious attachments are replaced with a text message stating the attachment name and the reason for blocking it.
A G-Suite Enterprise mailbox is configured with all advanced options selected, including the sandbox option in pre-delivery mode. Although all mailboxes are configured to “pre-delivery and discard” of the email message, some messages do get through to the inbox, with the attachment marked as “virus found”. We handle these attachments by comparing our send log to the Google receive logs. These marked attachments constitute less than 10% of the attachments in the inbox.
� Measuring TTD (time to detect)
In order to measure how long it takes for an email security solution to start blocking a malicious file that it had previously missed, we re-send the sample until it is blocked, or a timeout expires. Our resend schedule is as follows:
� In the first 4 hours re-send the sample every 30 minutes
� For the next 20 hours re-send the sample every 2 hours
� For the next 7 days re-send the sample every 6 hours
� After 7 days we stop re-sending the sample
In the Appendix we discuss some technical challenges that we faced during this study, and how we managed to overcome them.
x1- Example of Office 365 ATP Safe Attachments policies
The Results
Our study of the efficacy of email security products in detecting unknown threats at first encounter is in progress. So far, we have been running the study for almost 5 months, targeting mailboxes protected by O365 ATP and G-Suite Enterprise. Running against ProofPoint TAP has started in late January. During this period, we have sent many thousands of ‘fresh’ malicious file samples to multiple mailboxes protected by each security product. Below we provide charts for the two main indicators, Miss Rate at First Encounter and TTD, based on the data collected during this period.
The results of the study are summarized in Figure 1. The detailed results for each email security product are given below.
0.0%
10.0%
5.0%
15.0%
20.0%
25.0%
30.0%
40.0%
35.0%
ProofPoint OATP G Suite
Miss Rate
0
10
30
20
50
40
60Time (h)
53.3
31.6
10.1
Average Time to Detect
Figure 1. Overall Study Results
� Office365ATP
The charts below cover the period October 23rd 2019 – March 11th 2020. The miss rate during this period is about 25%. TTD average is about 53 hours. About 20% of unknown threats take 4 days or more to be detected.
0
40
20
60
80
100
120
160
140
≤6 [6, 12] [12, 18] [18, 24] [24, 30] [30, 36] >36
O365 ATP Time To Detect (average 53.3h)#Threats
Time (h)
Figure 2. Office 365 ATP Miss Rate and Time To Detect for Unknown Threats
0%
10%
20%
30%
40%
50%
60%
10/20/2019-
O365 ATP Miss Rate (average 25.3%)
Miss Rate
11/03/2019-
11/17/2019-
12/01/2019-
12/15/2019-
12/29/2019-
01/12/2020-
01/26/2020-
02/09/2020-
02/23/2020-
03/08/2020-
� G Suite Enterprise
The charts below cover the period November 5th 2019 - March 11th 2020. The Miss rate during this period is 35.4%. TTD average is about 32 hours. About 10% of unknown threats take 3 days or more to be detected.
0%
10%
30%
20%
40%
50%
70%
60%
80%
11/03/2019-
G Suite Miss Rate (average 35.4%)
Miss Rate
11/17/2019-
12/01/2019-
12/15/2019-
12/29/2019-
01/12/2020-
01/26/2020-
02/09/2020-
02/23/2020-
03/08/2020-
0
100
200
300
400
500
≤6 [6, 12] [12, 18] (18, 24] [24, 30] [30, 36] >36
G Suite Time To Detect (average 31.6h)
Time (h)
#Threats
Figure 3. G Suite Miss Rate and Time To Detect for Unknown Threats
� ProofPointTAP
The charts below cover the period January 26th 2020 - March 11th 2020. The Miss rate during this period is 22.6%. TTD average is about 10 hours.
0%
10%
20%
30%
40%
50%
60%
ProofPoint Miss Rate (average 22.6%)
Miss Rate
01/26/2020-
02/02/2020-
02/09/2020-
02/16/2020-
02/23/2020-
03/01/2020-
03/08/2020-
0
5
10
15
25
30
≤6 [6, 12] [12, 18] [18, 24] [24, 30] [30, 36] >36
ProofPoint Time To Detect (average 10.1h)#Threats
Time (h)
Figure 4. ProofPoint Miss Rate and Time To Detect for Unknown Threats
� Exposure Time Intervals
The chart below shows time intervals when O365 ATP is ‘blind’ to selected unknown threats it does not detect at first encounter. Although this is a relatively small sample of threats, it is notable that seven different unknown threats breeze through email security on Nov. 4th, ten on November 6th, six on Nov. 12th etc.
Detection Gaps for Missed Threats
10/31/2019
YCIp.csvea2b.xlsm2964.doc
e3ef.xls23d4.docdbbd.doc
d940.xlsba2a.xls
5a98.doc3d7b.xls6491.xls6b98.xls2d2a.rtf
d791.xlsmf896.xlsm65f0.doc27d5.xls
254d.docInfoLeaker.doc
62ba.xlsm
11/2/2019 11/4/2019 11/6/2019 11/8/2019 11/10/2019 11/12/2019 11/14/2019
Figure 5. ‘No Detection’ time intervals for various samples
� Discussion and Implications
The findings described in this document suggest that leading email security products have a very high miss rate for unknown threats. Considering that email security products depend heavily on threat data, this should not come as a big surprise. Figure 6 is an abstraction of the process used by email security vendors to handle unknown threats. When they encounter the first sample of an unknown threat, they usually ‘notice’ it through a reputation service or threat hunting (GAP 1). It then takes them some time to qualify the sample as malicious (GAP 2), and then they may need to apply some changes to their product to be able to detect the sample (GAP 3). This abstraction applies to vendor products using signatures, sandboxing, as well as machine learning.
The following description by Palo Alto Networks ties this time-related data dependency to recent changes in attackers’ practices:
“Attackers use automation to move fast and deploy new threats at breakneck speeds… A next-generation security platform rapidly analyzes data, turning unknown threats into known threats, creating an attack DNA, and automatically creating as well as enforcing a full set of protections...”.
Currently there are many more unknown threats than in the past. The industry’s best bet for handling them is to use data to turn them into known threats as quickly as possible. The problem is that as a result, Time To Detect (TTD) becomes hours or even days, which presents an unacceptable risk for the customers.
Since data-driven threat detection technologies fail to provide protection against unknown threats due to their inherent dependency on data, they must be augmented by a different type of technology in order to provide better email security. We believe that the answer lies in threat-agnostic detection technologies. If a detection technology is independent of data about the threats it identifies, then it should potentially provide excellent protection against unknown threats. It should probably provide similar protection against known and unknown threats.
The BitDam ATP solution is built on top of a unique threat-agnostic detection engine. BitDam’s technology uses a model of ‘clean’ execution flows of the application used to open (render) a file or a link (e.g., Office apps, Acrobat Reader, Chrome, etc.). This model consists of whitelisted CPU-level code execution flows of the application. A model is built for each application by tracing the execution of the application on benign files. When a new file is scanned, BitDam compares the execution flows extracted while running the application opening the file to the whitelisted flows included in the model for this application to calculate a verdict.
BitDam’s model-driven threat detection technology at the heart of BitDam ATP allows it to reach extremely high detection rates for unknown threats at first encounter. It’s TTD is zero, so full protection power is available at all times. BitDam is able to correctly identify all the unknown threats missed by the email security products we cover in our study. This makes BitDam ATP a natural choice for augmenting current email security products and considerably reduce the risk customers face today from their incoming email.
GAP 1 GAP 2 GAP 3 Detection
Reputation | Threat hunting
First encounter Qualifying maliciousness (i.e. not a marketing campaign)
Training ML | Adding a mechanism to detect it
Figure 6. Common process for handling unknown threats
Appendix: Technical Challenges Resolved During The study
Challenge 1: Sending the emails containing the samples
Sending emails containing malicious attachments is not a trivial task. A naïve approach of simply trying to send them using common services such as Google, Yahoo and others, ends up being blocked fairly quickly when an attempt to attach the file is made. Using email services that are not scanning for malicious attachments (like SendGrid2), results in an account freeze in less than one day.
Since reputation services are effective against known attacks, and since BitDam isn’t obtaining the samples before Gmail or O365 do, we needed to apply some changes to the files. The changes don’t alter the essence of the attack - it includes the same IP addresses, dropped files, sandbox evasion techniques and so on.
There are two major methods we employ in order to modify the files, thus avoiding being blocked before even sending the email.
� Method #1
Changing the hash of the file by adding benign data to it. For example, for RTF files, adding spaces to some of the streams. The malicious content in the file is left intact.
This method alone is effective for logical exploits that one can find in RTF files, but is ineffective for macro attacks (due to entropy calculations done on the macro itself by the email security vendor).
0
5
10
15
20
03:00 05:00 07:00 09:00 11:00
g.th
omps
on...
Coun
t
0
5
10
15
20
0
5
10
15
20
davi
d.bs
@bi
...Co
unt
bitd
amlu
c...
Coun
t
Success
Failed
Figure 7. Sending files with hash changed
x2 - World’s largest cloud-based email delivery platform
� Method #2
Changing the static signature of a macro:
1. Adding comments consisting of random words. Changing the macro’s entropy usually results in the file not being blocked on sending. This method works well, and about 90% of the malicious files are sent successfully.
2. Converting the code of each macro function to a base64 string and adding a code to decompress this code. This method works for 99% of the samples, as static analysis can’t detect if there is any issue with the macro.
0
5
10
15
14:00 17:00 20:00 23:00
Success
Failed
g.th
omps
on...
...
bitd
amlu
c...
Coun
t
0
5
10
15
Figure 8. Sending files with comments in the macro
0
5
10
17:00 20:00 23:00
Success
Failed
g.th
omps
on...
...Co
unt
0
5
10
bitd
amlu
c...
Coun
t
Figure 9. Sending files with base64 function contents
Challenge 2: Measuring Miss Rate
We send a few hundred malicious files each day to each target mailbox. For an O365 ATP protected mailbox, due to the way it is configured, we can tell which emails were blocked and view their eml headers where O365 writes the end to end latency of the scanning. We noticed that there are mainly two ranges of scanning times, below 5seconds and above 2 minutes.
All the files with scanning time below 5 seconds, were detected. In addition, inside the text file that replaced them, those files had a signature stating the malware type, i.e. “a02b332b78ede5d03490d4db96ebad85fca7fc2f.rtf O97M/CVE-2017-11882.AX!eml”which means it is the logical exploit marked as CVE-2017-11882, that is known as triggering “eqendt32.exe”.
We can deduce that these files were already known because of two main reasons:
1. The scanning time was below 5 seconds, which shows it didn’t go through a dynamic analysis solution (since dynamic analysis seems to take 2 minutes or more)
2. The malware type is descriptive and accurate, which means some sort of a signature mechanism was applied.
Hence, we assume that files with the low scanning latency are known threats and were detected using a reputation service. Due to the strong mechanism of permutation we apply to macros, the modified files were almost never detected – they had long scanning time with detonation. On the other hand, RTFs with CVE-2017-11882 are almost always detected due to a weak permutation mechanism (only basic hash change).
For files with a high scan time, we can deduce that they were unknown threats, hence had to go through the complete scanning process that includes a dynamic analysis\detonation mechanism. Only these files were included in the calculation of detection rates.
In the next step, we looked at the macros in each file we sent and went through long scanning (aka – unknown). We noticed that there are many files with different hashes but with the exact same macro (Figure 10). Since the same macro will result in the same IOCs (IP addressed, dropped files hashes, etc.), it doesn’t make sense to consider these files as different, and once you are able to detect one of them you should be able to detect all these similar threats.
Lastly, we noticed that some macros are “similar” to others, meaning that they are the same macro but slightly modified. See Figure 10 which has a similar macro to the one in Figure 11. The changes we have seen include:
� Changes of strings\comments
� Changes in variable names
� Addition of benign macro functions\lines
As in the case of the copied macros, the similar macros will also result in the same IOCs, thus were not included in the calculation of the overall result.
Figure 10. File macro (hash: 516d29bed93c78cd03a2ba8f2704050572954371, 8ccf648775188207f47fc4b7e7758ad13837b6a8)
Figure 11. Similar macro (hash: c1883e56b58a2027d25ea6184f5de9393a140981)
About BitDam
BitDam is a pioneer in cyber defense, securing enterprise email (Office 365, G-Suite, MS Exchange), cloud drives (OneDrive, G-Drive, Dropbox, Box etc.) and other collaboration tools from ransomware, malware, and phishing.
Unlike the alternatives that give a “grace period” to unknown cyberthreats, BitDam’s patented attack-agnostic technology stops malicious files and links at first encounter with unprecedented detection rates. Independent of feeds, reputation and intelligence services, BitDam’s cloud-based Advanced Threat Protection (ATP) detects never-seen-before attacks of any type, providing a remarkably higher detection rate and empowering organizations to collaborate safely.
Recognized by Frost & Sullivan for its technology leadership, BitDam’s award-winning ATP solution is utilized by hundreds of thousands of end-users and deployed by leading organizations in Europe and the US, with a proven record of detecting threats that other security solutions fail to uncover.
