Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June...
Transcript of Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June...
![Page 1: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/1.jpg)
Compression Bombs Strike BackGiancarlo Pellegrino(1) , Davide Balzarotti(2)
OWASP AppSec Europe '16June 30, Rome
(1) CISPA, Saarland University, Germany(2) Eurecom, France
![Page 2: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/2.jpg)
June 30, 2016 2
Introduction
Modern applications rely on (core) network services, e.g., Web, email, and IM services
HTTP, json, XML, SOAP
XMPP
IMAP, POP3, SMTP
![Page 3: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/3.jpg)
June 30, 2016 3
Introduction
Modern applications rely on (core) network services, e.g., Web, email, and IM services
Amount of exchanged data continues to increase steadily● More data → more transfer time → unresponsiveness → user unhappiness
![Page 4: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/4.jpg)
June 30, 2016 4
Introduction
Modern applications rely on (core) network services, e.g., Web, email, and IM services
Amount of exchanged data continues to increase steadily● More data → more transfer time → unresponsiveness → user unhappiness● Avg web page size as Doom ~2.3MB [1]
[1] HTTP Archive: http://www.httparchive.org/interesting.php?a=All&l=Apr%201%202016
![Page 5: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/5.jpg)
June 30, 2016 5
Introduction
Modern applications rely on (core) network services, e.g., Web, email, and IM services
Amount of exchanged data continues to increase steadily● More data → more transfer time → unresponsiveness → user unhappiness
Solution 1: buy more bandwidth!
![Page 6: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/6.jpg)
June 30, 2016 6
Introduction
Modern applications rely on (core) network services, e.g., Web, email, and IM services
Amount of exchanged data continues to increase steadily● More data → more transfer time → unresponsiveness → user unhappiness
Solution 1: buy more bandwidth!➔ Bandwidth costs
![Page 7: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/7.jpg)
June 30, 2016 7
Introduction
Modern applications rely on (core) network services, e.g., Web, email, and IM services
Amount of exchanged data continues to increase steadily● More data → more transfer time → unresponsiveness → user unhappiness
Solution 1: buy more bandwidth!➔ Bandwidth costs
Another solution is ...
![Page 8: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/8.jpg)
June 30, 2016 8
Introduction
Modern applications rely on (core) network services, e.g., web, email, and IM services
Amount of exchanged data continues to increase steadily● More data → more transfer time → unresponsiveness → user unhappiness
Solution 1: buy more bandwidth!➔ Bandwidth costs
Another solution is ...
Data compression!Data compression!
![Page 9: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/9.jpg)
June 30, 2016 9
Data Compression
Reduces # of bits of a string by removing redundancy● lossless if decompr(compr(d)) = d or lossy if decompr(compr(d)) ~= d
Lots of algorithms (See [1]) Among the most popular: Deflate [RFC 1951]
● Implemented in libraries, e.g., zlib, or as a tool, e.g., gzip, and zip archive tool● Available in most of the programming languages
100KB 15KB
[1] SALOMON, D. Data Compression: The Complete Reference. Springer-Verlang, 2007.
![Page 10: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/10.jpg)
June 30, 2016 10
Compression in Protocols
Compression used by network protocols to reduce message size Mandated by protocol specifications
● e.g., HTTP (response!) compression, IMAP, XMPP, SSH, PPP, and others Or implemented as custom feature
● e.g., HTTP request compression
XMPP Compression [XEP-0138]
IMAP Compression [RFC 4978]HTTP Compression [RFC 7230]
![Page 11: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/11.jpg)
June 30, 2016 11
Compression in HTTP (RFC 7230)
GET / HTTP/1.1Host: wikipedia.org[...]
HTTP/1.1 200 OK[...]ContentLength: 82170ContentType: text/html; charset=UTF8
<!DOCTYPE html><html [...]
Retrieve defaultHTML page
~80Kb of page
HTTP Request
HTTP Response
![Page 12: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/12.jpg)
June 30, 2016 12
Compression in HTTP (RFC 7230)
GET / HTTP/1.1Host: wikipedia.orgAcceptEncoding: gzip, deflate[...]
HTTP/1.1 200 OK[...]ContentLength: 18879ContentType: text/html; charset=UTF8ContentEncoding: gzip
�%O 5 * #�� � ��� �Ԟ[...]
Select algorithm
Response size -70%
Compressed response bodyDecompress
HTTP Request
HTTP Response
![Page 13: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/13.jpg)
June 30, 2016 13
The Problem of Data Compression If not properly implemented, it can make application vulnerable to DoS Risks:
1)Intensive task ● Computationally intensive● If abused, it can stall an application
2)Data Amplification● Decompression increases the data to be processed (compression rate of zlib ~1:1024)● Internal components may not be designed to handle high volume of data
3)Unbalanced Client-Server Scenario● One party pre-compute compressed messages● The other one decompresses messages each time
Popular examples from the past...
![Page 14: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/14.jpg)
June 30, 2016 14
The Past: Zip Bombs (1996)
42 KB zip file → 4.5 PB uncompressed data
5 layers of nested zip files in blocks of 16, last layer with text files of 4.3 GB each
Cause Disk/Memory exhaustion
Sent as attachment to crash anti-virus software
0.dll 1.dll 15.dll...
page0.zip
doc0.zip
chapter0.zip
book0.zip
lib0.zip
page1.zip page15.zip
42.zip
lib1.zip lib15.zip...
doc1.zip doc15.zip...
...
chapter1.zip chapter15.zip...
book1.zip book15.zip...
4.3GB
AAAAAAAAAA ... A
0.dll 1.dll 15.dll...
4.5 PB
![Page 15: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/15.jpg)
June 30, 2016 15
The Past: Billion Laughs (2003) Resource exhaustion in libxml2 when processing nested XML entity definitions
810 bytes of XML document expanded to 3GB
<?xml version="1.0"?><!DOCTYPE lolz [ <!ENTITY lol "lol"> <!ELEMENT lolz (#PCDATA)> <!ENTITY lol1 "&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;"> <!ENTITY lol2 "&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;"> <!ENTITY lol3 "&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;"> <!ENTITY lol4 "&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;"> <!ENTITY lol5 "&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;"> <!ENTITY lol6 "&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;"> <!ENTITY lol7 "&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;"> <!ENTITY lol8 "&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;"> <!ENTITY lol9 "&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;">]><lolz>&lol9;</lolz>
![Page 16: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/16.jpg)
June 30, 2016 16
The Past: Zip Bombs and Billion Laughs
<?xml version="1.0"?><!DOCTYPE lolz [ <!ENTITY lol "lol"> <!ELEMENT lolz (#PCDATA)> <!ENTITY lol1 "&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;"> <!ENTITY lol2 "&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;"> <!ENTITY lol3 "&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;"> <!ENTITY lol4 "&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;"> <!ENTITY lol5 "&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;"> <!ENTITY lol6 "&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;"> <!ENTITY lol7 "&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;"> <!ENTITY lol8 "&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;"> <!ENTITY lol9 "&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;">]><lolz>&lol9;</lolz>
0.dll 1.dll 16.dll...
page0.zip
doc0.zip
chapter0.zip
book0.zip
lib0.zip
page1.zip page16.zip
42.zip
lib1.zip lib16.zip...
doc1.zip doc16.zip...
...
chapter2.zip chapter16.zip...
book2.zip book16.zip...
4.3GB
AAAAAAAAAA ... A
0.dll 1.dll 16.dll...
This was 1996-2003!Now we know better, right?
This was 1996-2003!Now we know better, right?
![Page 17: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/17.jpg)
June 30, 2016 17
The Present
Reviewed protocol specs, design patterns, and coding rulesUnawareness of the risks, guidelines on handling data compression are missing or misleading
![Page 18: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/18.jpg)
June 30, 2016 18
The Present
Reviewed protocol specs, design patterns, and coding rulesUnawareness of the risks, guidelines on handling data compression are missing or misleading
1. Protocol specifications: ➔ No data compression handling issues, redirects to SSL/TLS (concerned with leakage and packet limits, but
unexplained how they apply to other protocols)
![Page 19: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/19.jpg)
June 30, 2016 19
The Present
Reviewed protocol specs, design patterns, and coding rulesUnawareness of the risks, guidelines on handling data compression are missing or misleading
1. Protocol specifications: ➔ No data compression handling issues, redirects to SSL/TLS (concerned with leakage and packet limits, but
unexplained how they apply to other protocols)
2. Secure Design Patterns: ● Patterns to solve vulns. during design phase : DoS Safety, Compartmentalization, and Small Process➔ However, lack of the details to address implementation-level concerns
![Page 20: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/20.jpg)
June 30, 2016 20
The Present
Reviewed protocol specs, design patterns, and coding rulesUnawareness of the risks, guidelines on handling data compression are missing or misleading
1. Protocol specifications: ➔ No data compression handling issues, redirects to SSL/TLS (concerned with leakage and packet limits, but
unexplained how they apply to other protocols)
2. Secure Design Patterns: ● Patterns to solve vulns. during design phase : DoS Safety, Compartmentalization, and Small Process➔ However, lack of the details to address implementation-level concerns
3. Secure Coding Rules ● Only one, i.e., Anti-Zip Bomb coding rule➔ Sadly, incorrect
![Page 21: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/21.jpg)
June 30, 2016 21
The Present
Reviewed protocol specs, design patterns, and coding rulesUnawareness of the risks, guidelines on handling data compression are missing or misleading
1. Protocol specifications: ➔ No data compression handling issues, redirects to SSL/TLS (concerned with leakage and packet limits, but
unexplained how they apply to other protocols)
2. Secure Design Patterns: ● Patterns to solve vulns. During design phase : DoS Safety, Compartmentalization, and Small Process➔ However, lack of the details to address implementation-level concerns
3. Secure Coding Rules ● Only one, i.e., Anti-Zip Bomb coding rule➔ Sadly, incorrect
How does this lack of common knowledge andunderstanding affect implementations?
How does this lack of common knowledge andunderstanding affect implementations?
![Page 22: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/22.jpg)
June 30, 2016 22
Impact on Implementations
![Page 23: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/23.jpg)
June 30, 2016 23
HTTP (Response) Compression (RFC 7230)
GET / HTTP/1.1Host: wikipedia.orgAcceptEncoding: gzip, deflate[...]
HTTP/1.1 200 OK[...]ContentLength: 18879ContentType: text/html; charset=UTF8ContentEncoding: gzip
�%O 5 * #�� � ��� �Ԟ[...]
Select algoritm
Response size -70%
Compressed response bodyDecompress
HTTP Request
HTTP Response
![Page 24: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/24.jpg)
June 30, 2016 24
Compression Bombs against Web Browsers #1
GET / HTTP/1.1Host: attacker.fooAcceptEncoding: gzip, deflate[...]
HTTP/1.1 200 OK[...]ContentLength: 4000000ContentType: text/html; charset=UTF8ContentEncoding: gzip
HTTP Request
HTTP Response
4 GB of white spaces
Compression rate ~1:1000
See: Geoff Jones http://blog.cyberis.co.uk/2013/08/vulnerabilities-that-just-wont-die.html
![Page 25: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/25.jpg)
June 30, 2016 25
Compression Bombs against Web Browsers #1
GET / HTTP/1.1Host: attacker.fooAcceptEncoding: gzip, deflate[...]
HTTP/1.1 200 OK[...]ContentLength: 4000000ContentType: text/html; charset=UTF8ContentEncoding: gzip
Decompress
HTTP Request
HTTP Response
4 GB of white spaces
Compression rate ~1:1000
See: Geoff Jones http://blog.cyberis.co.uk/2013/08/vulnerabilities-that-just-wont-die.html
![Page 26: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/26.jpg)
June 30, 2016 26
Compression Bombs against Web Browsers #2
GET / HTTP/1.1Host: attacker.fooAcceptEncoding: gzip, deflate[...]
HTTP/1.1 200 OK[...]ContentLength: 4000ContentType: text/html; charset=UTF8ContentEncoding: gzip, gzip
HTTP Request
HTTP Response
4 GB of white spaces
2 layers of compression!
x1000 smaller
See: Geoff Jones http://blog.cyberis.co.uk/2013/08/vulnerabilities-that-just-wont-die.html
![Page 27: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/27.jpg)
June 30, 2016 27
Compression Bombs against Web Browsers #2
GET / HTTP/1.1Host: attacker.fooAcceptEncoding: gzip, deflate[...]
HTTP/1.1 200 OK[...]ContentLength: 4000ContentType: text/html; charset=UTF8ContentEncoding: gzip, gzip
HTTP Request
HTTP Response
4 GB of white spaces
2 layers of compression!
Decompress
x1000 smaller
See: Geoff Jones http://blog.cyberis.co.uk/2013/08/vulnerabilities-that-just-wont-die.html
![Page 28: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/28.jpg)
June 30, 2016 28
HTTP (Response) Compression Bombs
“Vulnerabilities that just won't die - Compression Bombs”by Geoff Jones
http://blog.cyberis.co.uk/2013/08/vulnerabilities-that-just-wont-die.html
Most are stillvulnerable!
Most are stillvulnerable!
![Page 29: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/29.jpg)
June 30, 2016 29
How about servers?
??
![Page 30: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/30.jpg)
June 30, 2016 30
Experiments
Case studies:● HTTP, XMPP, and IMAP servers
Testbed:
InternalMonitor
Implementation
Linux 3.8 Kernel
/proc
External monitor
Attackers
Compression bombs
![Page 31: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/31.jpg)
June 30, 2016 31
HTTP (request) Compression Bomb (SOAP)
Case studies:● HTTP, XMPP, and IMAP servers
Testbed:
InternalMonitor
Implementation
Linux 3.8 Kernel
/proc
External monitor
Attackers
Compression bombs~4 MB, ~1:1000 compr. ratio
POST /index.html HTTP/1.1ContentEncoding: gzip\r\n<soapenv:Envelope>
<soapenv:Body>[...]</soapenv:Body></soapenv:Envelope>\r\n
4 GB of white spaces
compressed
Same for JSON
Same for JSON
![Page 32: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/32.jpg)
June 30, 2016 32
XMPP Compression Bomb
Case studies:● HTTP, XMPP, and IMAP servers
Testbed:
InternalMonitor
Implementation
Linux 3.8 Kernel
/proc
External monitor
Attackers
Compression bombs~4 MB, ~1:1000 compr. ratio
<?xml version='1.0' ?><stream:stream to='server'
xmlns='jabber:client' Version='1.0'>
4 GB of white spaces
compressed
![Page 33: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/33.jpg)
June 30, 2016 33
IMAP Compression Bomb
Case studies:● HTTP, XMPP, and IMAP servers
Testbed:
InternalMonitor
Implementation
Linux 3.8 Kernel
/proc
External monitor
Attackers
Compression bombs~4 MB, ~1:1000 compr. ratio
From: sender@fooTo: receiver@fooSubject : I am a bomb!
4 GB of white spaces
compressed
![Page 34: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/34.jpg)
June 30, 2016 34
Compression Bombs Everywhere
Protocol Network ServiceXMPP OpenFire
Prosody
Tigase
Ejabberd, jabberd2HTTP Apache HTTPD + mod_deflate
+ mod-php, CSJRPC, mod-gsoap, mod-dav
Apache Tomcat + 2Way/Webutilities filter+ Apache CXF
+ json-rpc, lib-json-rpc
+ Axis2/ +jsonrpc4j
Axis 2 standalone
gSOAP standaloneIMAP Dovecot, Cyrus
![Page 35: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/35.jpg)
June 30, 2016 35
Compression Bombs Everywhere
Protocol Network ServiceXMPP OpenFire
Prosody
Tigase
Ejabberd, jabberd2HTTP Apache HTTPD + mod_deflate
+ mod-php, CSJRPC, mod-gsoap, mod-dav
Apache Tomcat + 2Way/Webutilities filter+ Apache CXF
+ json-rpc, lib-json-rpc
+ Axis2/ +jsonrpc4j
Axis 2 standalone
gSOAP standaloneIMAP Dovecot, Cyrus
CVE-2014-2741
CVE-2014-2746
CVE-2014-0118
Notif. devel
Notif. devels
CVE-2014-2744/ -2745
CVE-2014-0109/ -0110
Notif. devel
![Page 36: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/36.jpg)
June 30, 2016 36
Pitfalls
![Page 37: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/37.jpg)
June 30, 2016 37
Pitfalls
1. Implementation
2. Specification
3. Configuration
![Page 38: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/38.jpg)
June 30, 2016 38
Pitfalls
1. Implementation
2. Specification
3. Configuration
● Use of Compression before Authentication● Improper Input Validation during Decompression● Logging Decompressed Messages● Improper Inter-Units Communication● Unbounded Resource Usage (CPU and Memory)
● Erroneous Best Practice● Misleading Documentation● API Specs Inconsistency
● Insufficient Configuration Options● Insecure Default Values● Decentralized Configuration Parameters
![Page 39: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/39.jpg)
June 30, 2016 39
1. Implementation
2. Specification
3. Configuration
Pitfalls
● Use of Compression before Authentication● Improper Input Validation during Decompression● Logging Decompressed Messages● Improper Inter-Units Communication● Unbounded Resource Usage (CPU and Memory)
● Erroneous Best Practice● Misleading Documentation● API Specs Inconsistency
● Insufficient Configuration Options● Insecure Default Values● Decentralized Configuration Parameters
![Page 40: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/40.jpg)
June 30, 2016 40
1. Implementation
2. Specification
3. Configuration
Pitfalls
● Use of Compression before Authentication● Improper Input Validation during Decompression● Logging Decompressed Messages● Improper Inter-Units Communication● Unbounded Resource Usage (CPU and Memory)
● Erroneous Best Practice● Misleading Documentation● API Specs Inconsistency
● Insufficient Configuration Options● Insecure Default Values● Decentralized Configuration Parameters
Have a look at our paper!http://trouge.net/gp/papers/compr_usenix15.pdf
Have a look at our paper!http://trouge.net/gp/papers/compr_usenix15.pdf
![Page 41: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/41.jpg)
June 30, 2016 41
Pitfalls at Implementation level
Valid. Decompr. Parser
Logger
Appl.M
evt evt evt
Authn.
evt
Abstract message processing pipeline extracted from our case studies
![Page 42: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/42.jpg)
June 30, 2016 42
Compression before Authentication
Inconsistent best practice● Mandatory in SSL/TLS, recommended in XMPP, and undefined in IMAP and HTTP● Implementation may diverge from the specs, i.e., OpenSSH
Developers may underestimate the risk or overlook recommendations Prosody accepted compressed messages before user authentication
➔ DoS by unauthenticated attackers
Valid. Decompr. Parser
Logger
Appl.M
evt evt evt
Authn.
evt
CVE-2014-2744
![Page 43: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/43.jpg)
June 30, 2016 43
Improper Input Validation during Decompression
3 ways to validate a message:● Compressed message size
● mod-deflate: If (compr. size > LimitRequestBody) → Reject➔ However, hard to assess message size from its compressed form (1 MB compr → 1 GB decompr.)
Valid. Decompr. Parser
Logger
Appl.M
evt evt evt
Authn.
evt
CVE-2014-0118
mistake
![Page 44: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/44.jpg)
June 30, 2016 44
Improper Input Validation during Decompression
3 ways to validate a message:● Compressed message size
● mod-deflate: If (compr. size > LimitRequestBody) → Reject➔ However, hard to assess message size from its compressed form (1 MB compr → 1 GB decompr.)
● Decompression ratio● Patched mod-deflate: if (decompr ratio > threshold) → Reject➔ Problem of ratio selection
Valid. Decompr. Parser
Logger
Appl.M
evt evt evt
Authn.
evt
mistake
risky
CVE-2014-0118
![Page 45: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/45.jpg)
June 30, 2016 45
Improper Input Validation during Decompression
3 ways to validate a message:● Compressed message size
● mod-deflate: If (compr. size > LimitRequestBody) → Reject➔ However, hard to assess message size from its compressed form (1 MB compr → 1 GB decompr.)
● Decompression ratio● Patched mod-deflate: if (decompr ratio > threshold) → Reject➔ Problem of ratio selection
● Decompressed message size● mod-deflate + mod-dav: If (decompr. size > LimitXMLRequestBody) → Reject
Valid. Decompr. Parser
Logger
Appl.M
evt evt evt
Authn.
evt
mistake
correct
risky
CVE-2014-0118
![Page 46: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/46.jpg)
June 30, 2016 46
Improper Inter-Units Communication
Upon exception, the pipeline halts and rejects message mod-php and mod-gsoap limit the size of incoming (decompressed) message … but had no means to halt mod-deflate
➔ mod-deflate keeps on decompressing data● Problem addressed in
Valid. Decompr. Parser
Logger
Appl.M
evt evt evt
Authn.
evt
CVE-2014-0118
![Page 47: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/47.jpg)
June 30, 2016 47
Logging Decompressed Messages
Frequency and verbosity of log events can cause DoS If exception is caused by compressed data, the needed resources may be underestimated Upon invalid requests, Apache CXF logs first 100KB of incoming message
● However, first it decompresses the entire message on a file, then logs the first 100KB➔ DoS due to disk space exhaustion
Valid. Decompr. Parser
Logger
Appl.M
evt evt evt
Authn.
evt
CVE-2014-0109/ -0110
![Page 48: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/48.jpg)
June 30, 2016 48
Erroneous Best Practices (Spec. level) Only one code pattern specific for data compression
● Rule: “IDS04-J. Safely extract files from ZipInputStream”
.getSize() returns ZIP file header with uncompressed size
but ZIP headers not integrity protected!➔ DoS countermeasure bypass
// Write the files to the disk, but// only if the file is not insanely bigif (zipfile.getSize() > TOOBIG ) { throw new IllegalStateException("File to be unzipped is huge.");}
Notif. Authors
![Page 49: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/49.jpg)
June 30, 2016 49
Pitfalls
![Page 50: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/50.jpg)
June 30, 2016 50
Conclusion
![Page 51: Compression Bombs Strike Back - Global AppSec › wp-content › uploads › 2016 › 07 › ...June 30, 2016 14 The Past: Zip Bombs (1996) 42 KB zip file → 4.5 PB uncompressed data](https://reader033.fdocuments.us/reader033/viewer/2022060502/5f1bc2080248c134c86028d2/html5/thumbnails/51.jpg)
June 30, 2016 51
Conclusion/Takeaway
Compression bombs are back
~20 years after the zip bombs, developers still unaware of the risks of handling data compression
➔ Discovered 10 previously-unknown vulnerabilities in popular network services
➔ Presented 12 pitfalls which can be used by developers to build more secure services