High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov...

31
High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht Nikita Borisov University of Illinois at Urbana-Champaign * Department of Computer Science Department of Electrical and Computer Engineering {nschear2, dalbrech, nikita}@illinois.edu 16 September 2008

Transcript of High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov...

Page 1: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

High-Speed Matching of Vulnerability Signatures

Nabil Schear *

David R. Albrecht †

Nikita Borisov †

University of Illinois at Urbana-Champaign* Department of Computer Science

† Department of Electrical and Computer Engineering

{nschear2, dalbrech, nikita}@illinois.edu

16 September 2008

Page 2: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

2

Exploit vs. Vulnerability Signatures

• Exploit Signatures – Match a specific example of an exploit

• Vulnerability Signatures – Match the condition at which the program is vulnerable

+ fast to match

- imprecise, false positives

+ exploit generic, very precise

- expensive

Page 3: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

3

Example – CUPS/IPPHTTP/1.1 200 OKContent-Type: ippTransfer-Encoding: chunked

A05

headerattribute attribute

attribute

HTTP

IPP

extra dataattribute

attribute

Page 4: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

4

Example – CUPS/IPPHTTP/1.1 200 OKContent-Type: ippTransfer-Encoding: chunked

A05

headerattribute attribute

attribute

HTTP

IPP

extra dataattribute

attribute

tag name_len namevalue_len value

Page 5: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

5

Example – CUPS/IPPHTTP/1.1 200 OKContent-Type: ippTransfer-Encoding: chunked

A05

headerattribute attribute

attribute

HTTP

IPP

extra dataattribute

attribute

tag name_len namevalue_len value

Buffer overflow: uint16 name_len used to copy name into 8KB buffer without checks

Page 6: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

6

Example – CUPS/IPPHTTP/1.1 200 OKContent-Type: ippTransfer-Encoding: chunked

A05

headerattribute attribute

attribute

HTTP

IPP

extra dataattribute

attribute

tag name_lenvalue_len value

0xA190909090EB105B4B33C966B9960380340BFDE2FAEB05E8EBFFFFFF

Exploit Signature

alert tcp any any -> any 631(content: “|EB 10 5B 4B 33 C9 66 B9 96 03…|”)

Shell code stored in name field

Page 7: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

7

Example – CUPS/IPP

HTTP/1.1 200 OKContent-Type: ippTransfer-Encoding: chunked

E5

headerattribute attribute

attribute tag name_len0xA190909090EB105B4B33C966B99

920

extra dataattribute

value_len value60380340BFDE2FAEB05E8EBFFFFFF

HTTP Chunk 1

Chunk 2

attribute attribute

• Now split shell code across two HTTP chunks

Page 8: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

8

Example – CUPS/IPPHTTP/1.1 200 OKContent-Type: ippTransfer-Encoding: chunked

A05

headerattribute attribute

attribute

HTTP

IPP

extra dataattribute

attribute

tag name_len namevalue_len value

Vulnerability Signature

if(name_len > 8192) Exception!

Page 9: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

9

Motivation: Matching Performance

Protocol binpac hand-coded

CUPS/HTTP 5,414 20,340

DNS 71 2,647

IPP 809 7,601

WMF 610 14,013

Throughput (Mbits/s) of vulnerability matchers

• Hand-coded 3x to 37x faster!• Many vulnerabilities do not

require full protocol parsing

Page 10: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

10

Introducing VESPA

• A vulnerability signature and protocol parsing architecture

• Focus on performance– Hardware acceleration friendly design

• Future work: Offload to FPGA, network processor

– Target use in NIC or switch• 1 Gbps+• Low latency

Page 11: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

11

Outline

• Parsing Architecture Design– Text Protocols– Binary Protocols

• Vulnerability Specification Language

• Performance Evaluation

• Related Work

• Conclusions

Page 12: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

12

VESPA Design

• Couple protocol and vulnerability specifications– maximum parser optimization

• Design Principles– Fast matching primitives– Explicit State Management– Avoid parsing irrelevant message parts

• Basic Idea: Construct matching specs based on primitives and marry to state control functions

Page 13: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

13

Protocol State

• Core State– Example: HTTP Content-Length header– Define structure and semantics of the message

• Always parse

Page 14: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

14

Protocol State

• Core State– Example: HTTP Content-Length header– Define structure and semantics of the message

• Always parse

• Application State– Example: HTTP Accept-Charset header – Only relevant to the application

• Skip by default

Page 15: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

15

Text Protocols

• Often use explicit field labeling– e.g., RCPT TO: <[email protected]>

• multi-string matching primitive to flatten irrelevant protocol structure– e.g., search for “HTTP/1.”, “Content-Length:”,

“Transfer-Encoding:”, “POST”, and “\r\n\r\n” simultaneously

• Use control logic to drive matching primitive

Page 16: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

16

Binary Protocols

• Field meaning based on position in message

• Binary traversal primitive– Parses only core fields – No full in-memory representation– Parses vulnerability relevant fields when desired– Implemented with binpac language

Page 17: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

17

VESPA Language

• Stores each var as a member of generated C++ class

• Extraction function within %{…}%

bool is_post = str_matcher “POST” handler handle_post() %{ is_post = true; }%

handle_post() %{ if(is_post) deploy(content_length);}%

Handler SpecString Matcher Primitive Spec

• Embedded C++ code• deploy(var) function to

control match state• Check vulnerability

predicates here

Page 18: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

18

Binary Protocols

uint16 name_len = bin_matcher IPP.binpac:IPP_Attr_Data.name_len handler handle_name() default;

handle_name() %{ if(name_len > 8192) // throw exception}%

VESPA

• VESPA controls:– vulnerability state– predicate evaluation

Page 19: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

19

Binary Protocols

• binpac controls protocol binary traversal

uint16 name_len = bin_matcher IPP.binpac:IPP_Attr_Data.name_len handler handle_name() default;

handle_name() %{ if(name_len > 8192) // throw exception}%

type IPP_Attr_Data = record { name_len: uint16; name: bytestring &length = name_len &transient; value_len: uint16; value: bytestring &length = value_len &transient; };

binpac IPP specification

VESPA

Page 20: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

20

Modifying binpac for Binary Traversal

• Optimized binpac dynamic memory usage– Pre-allocate one of each object that could be

parsed in one object – Remove STL vector storage for all array elements

Page 21: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

21

Modifying binpac for Binary Traversal

• Optimized binpac dynamic memory usage– Pre-allocate one of each object that could be

parsed in one object – Remove STL vector storage for all array elements

• Use &pointer attribute to specify objects that must be dynamically created– e.g., DNS name pointers…

Page 22: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

22

Evaluation

• Focus on vulnerabilities difficult to match with exploit sigs• Tested raw vuln sig matcher/parser performance

– Network reassembly and reporting stages studied elsewhere

• Test System– 2.6 GHz AMD Athlon64– 4GB RAM– Ubuntu Linux 2.6.22-x86-64

Page 23: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

23

Tested Vulnerabilities

• HTTP/IPP– Negative Content-Length causes integer overflow– uint16 name_len used to store size of 8KB buffer

• DNS– Pointer cycle can cause denial of service

• WMF– Vulnerable feature: allows arbitrary abort

procedure to execute malicious code

Page 24: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

24

Memory Micro-benchmarks

• 6x to 40x reduction in number of calls to new• IPP and WMF call new 6x for any file• DNS proportional to num of DNS pointers

Protocol binpac traversal

DNS 15,812 2,296

IPP 1,360 432

WMF 3,824 312

Protocol binpac traversal

DNS 539 14

IPP 33 6

WMF 94 6

Bytes allocated per message

Calls to new/malloc per message

Page 25: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

25

Memory Micro-benchmarks

• 6x to 40x reduction in number of calls to new• IPP and WMF call new 6x for any file• DNS proportional to num of DNS pointers

Protocol binpac traversal

DNS 15,812 2,296

IPP 1,360 432

WMF 3,824 312

Protocol binpac traversal

DNS 539 14

IPP 33 6

WMF 94 6

Bytes allocated per message

Calls to new/malloc per message

Page 26: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

26

String Primitive Micro-benchmarks

• Multi-string matching dominates text performance

• VESPA approximates performance of pattern based IDS for simple signatures

Page 27: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

27

Parser Performance

• VESPA outperforms binpac by 3 to 5 times

Page 28: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

28

Parser Performance

• VESPA DNS considerably faster than binpac– Recall, hand-coded 9x faster than VESPA (2.6 Gbits/s)– Room for improvement in binary traversal

Page 29: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

29

Related Work

• Pattern Matching– Wu-Manber, Aho-Corasik, flex, pcre, XFA,

Protomatching

• Vulnerability Signatures– Shield, GAPA, binpac, NetShield, Prospector

• IDS/IPS– Snort, Bro, SafeCard

Page 30: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

30

Conclusions

• Key Insight: Vulnerability signatures often do not require full protocol parsing– Specialize protocol parser to signature matching

• Developed VESPA language and architecture– 3-5 times faster than binpac– Performance tied to speed of primitives

• Able to hardware accelerate multi-string matching• Improved performance of binary traversal

• Vulnerability signatures can be matched at 1 Gbps+– Suitable for server NICs, switches, inline IPS

Page 31: High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign * Department.

31

Thank you!

Questions?