CSE 6392Intrusion Detection Systems

Lecture #1Dr. Donggang Liu

CSE 6392 By Dr. Donggang Liu 2

About the Instructor

• Dr. Donggang Liu, Assistant @CSE– http://ranger.uta.edu/~dliu– dliu@cse.uta.edu– (817)272-0741– Office: NH330– Office hours: MW 5:00PM ~ 6:00PM

CSE 6392 By Dr. Donggang Liu 3

About the TA

• TBD

CSE 6392 By Dr. Donggang Liu 4

Course Description

• Comprehensive and In-Depth Introduction to theScience and Art of Intrusion Detection– What is intrusion?– Why need intrusion detection?– History of intrusion detection.– What techniques are available?

• Study Principles, Techniques for Intrusion Detection– Misuse detection– Anomaly detection– Hybrid model

CSE 6392 By Dr. Donggang Liu 5

Course Description (Cont’d)

• Case Study of Representative Intrusion Detection Systems– IDES (Intrusion Detection Expert System)– GrIDS (Graph Based Intrusion Detection System)– EMERALS (Event Monitoring Enabling Responses to Anomalous Live

Disturbances)– NetSTAT (A Network Based Intrusion Detection System)– Bro (Real-Time Network Intrusion Detection)– Snort

• Theoretical Background of Intrusion Detection– Intrusion detection models– Base rate fallacy and its implication

CSE 6392 By Dr. Donggang Liu 6

Course Description (Cont’d)

• Countermeasures against Intrusion Detection• Advanced Topics

– Intrusion Detection and Beyond– Forensic– Intrusion Tracing– Virus and Worm

• Limitations of Intrusion Detection• Open Problems in Intrusion Detection

CSE 6392 By Dr. Donggang Liu 7

Course Objectives

• Gain Understanding of Basic Issues, Concepts,Principles, and Techniques in Intrusion Detection.– Vulnerability, exploit– Intrusion– Intrusion detection– Intrusion response

• Be Able to Evaluate Intrusion Detection Systems forParticular Security Requirements– Root privilege compromise should be detected in real-time– False positive rate should be less than 1%

CSE 6392 By Dr. Donggang Liu 8

Course Outline

• Intrusions– Almost always come from network– Almost always against host

• Network Based Attacks– Passive: eavesdropping, unauthorized access– Active: break-in, modification, deletion, forgery of

confidential information, denial-of-service attack• Basic Security Concepts

– Confidentiality, integrity, identity, anonymity, availability– Vulnerability and exploit of vulnerability

CSE 6392 By Dr. Donggang Liu 9

Course Outline (Cont’d)

• Host Based Intrusion Detection– Pro & cons

• Network Based Intrusion Detection– Pro & cons

• Misuse Detection– Efficient– Lower false positive rate– Only effective against known attacks

• Anomaly Detection– Could potentially detect unknown attacks– High false positive rate

CSE 6392 By Dr. Donggang Liu 10

Course Outline (Cont’d)

• Intrusion Detection Techniques– Static and Dynamic Checking of Programs– Large-Scale (Internet-wide) Distributed Intrusion Detection– Early Sensing– Alert Correlation– Complex Attack Scenario Analysis

CSE 6392 By Dr. Donggang Liu 11

Course Outline (Cont’d)

• Intrusion Tracing– IP Spoofing– Stepping Stones– Reflector– Zombie

• Intrusion Response– Blocking?– Rate limiting?

• Advanced Topics– Countermeasures against intrusion detection– Survivable systems– Forensics– Virus, worms, Trojan horse

CSE 6392 By Dr. Donggang Liu 12

Prerequisites

• Familiar with Operating System Internals• Familiar with TCP/IP Protocol Suite and Its

Implementations (i.e. BSD, Linux)• Basic Knowledge and Skills in Discrete

Mathematics• Motivation!!!

CSE 6392 By Dr. Donggang Liu 13

Course Format

• No Textbook!– This is a research oriented course, no existing textbook on

intrusion detection is appropriate (good enough)– Course is based on recent papers in academic conferences

and journals• The Course Consists of Lectures and Projects and

Presentations– In the first half of the semester, for each topic, the

instructor will provide a list of papers and give an overviewof the research problems

– Students are required to research for more papers and sharetheir reports

• Research papers listed on the course website

CSE 6392 By Dr. Donggang Liu 14

Course Style

• Descriptive: what is out there• Critical: what is wrong with ...• Skill oriented: papers and projects

– Explore!• Interactive: discussion and questions encouraged and

considered in grade– Students are encouraged to present their findings– Active participation in class discussion is part of

requirement for students

CSE 6392 By Dr. Donggang Liu 15

On-line Resources

• WWW page:– http://ranger.uta.edu/~dliu/cse6392-ids-

spring2007.htm– For course materials, e.g., lecture slides,

homework files, papers, tools, etc.– Will be updated frequently. So check frequently.

CSE 6392 By Dr. Donggang Liu 16

Grading

• No Exams!• Participation 10%, Presentations 90%• The Final Grades Are Computed According to the

Following Rules:– A: >=85%– B: >=70%

CSE 6392 By Dr. Donggang Liu 17

Policies on Absences

• You may be excused from class without penalty onclass participation credits only with a universityapproved condition, with proof. For example, if youcannot take a class because of a sickness, we willneed a doctor's note.

CSE 6392 By Dr. Donggang Liu 18

Academic Integrity

• The university, college, and departmentpolicies against academic dishonesty willbe strictly enforced.– http://www.uta.edu/studentaffairs/judicialaffair

s

CSE 6392 By Dr. Donggang Liu 19

Term Paper/Project

• (Optional) Can Be:– Research Paper

• Work on original research problem with original technicalcontribution

– Survey Paper• Comprehensive summary of a particular topic

– Design of New Algorithms, Protocols or New Attacks!• Should justify the usefulness

– Analysis/Evaluation of Existing Algorithms, Protocols.• Provide new insights

– Implementation and Experimentation.• Better implementation of existing algorithm, protocols

CSE 6392 By Dr. Donggang Liu 20

Term Paper/Project (Cont’d)

• 30%• To Be Done Individually or Team of 2~3 Students• Two phases:

– Proposal– Presentation and final report

CSE 6392 By Dr. Donggang Liu 21

Security Problems on Internet ConnectedComputers

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

1997 1998 1999 2000 2001 2002 2003

Number of Security Incidents

Reported to CERT/CC

CSE 6392 By Dr. Donggang Liu 22

Network Security Problems

• Start From The Basics

a) Normal Flow

A B

c) Modification

A B

C

d) Fabrication

A B

C

b) Eavesdropping

A B

C

CSE 6392 By Dr. Donggang Liu 23

Network Security Problems (Cont’d)

• Start From The Basics

e) Drop

A B

g) Jam it!

A B

C

f) Replay

A B

C

CSE 6392 By Dr. Donggang Liu 24

Network Security Concepts

• Confidentiality– Prevent information from being exposed to unintended party

• Integrity– Assure that the information has not been tempered

• Authentication– Assure that the party of concern is authentic - it is what it claims to be

• Availability– Assure that unused service or resource is available to legitimate users

• Anonymity– Assure that the identity of some party is remain anonymous

• Non-Repudiation– Assure that authenticated party has indeed done something and it can

not deny it

CSE 6392 By Dr. Donggang Liu 25

Commercial Example

• Confidentiality– An employee should not come to know the salary of his manager

• Integrity– An employee should not be able to modify the employee's own salary

• Authentication– An employee should be able to uniquely authenticate himself/herself

• Availability– Paychecks should be printed on time as stipulated by law

• Anonymity– The manager should not know who had a critical review for him

• Non-repudiation– Once the employee has cashed out the paycheck, he/she can’t deny it

CSE 6392 By Dr. Donggang Liu 26

Real-World Network Based Attacks

• Unauthorized Access to Resources– Disclosure, modification, and destruction of resources

• Distributed Denial of Service (DDOS) Attacks• Worm and Virus Attacks (e.g., worm sasser)• Monitoring and Capture of Network Traffic

– User IDs, passwords, and other information are often stolen on Internet

• Exploitation of Software Vulnerability (MS-Windows)• Compromised System Used as Stepping Stone• Masquerade as Authorized User or End System• Data driven attacks

– Importation of malicious or infected code

• E-Mail Forgery

CSE 6392 By Dr. Donggang Liu 27

Attack Family Interdependency

who toimpersonate

sniff forcontent

traffic analysis- who is talking

jam/cut it

capture &modify

pretend

I want tobe Bill

Passive attacks Active Attacks

CSE 6392 By Dr. Donggang Liu 28

Contributing Factors

• Lack of Awareness of Threats and Risks of From theNetwork– Security measures are often not considered until an

Enterprise has been penetrated by malicious users• Wide-Open Network Policies

– Many Internet sites allow wide-open Internet access• Vast Majority of Network Traffic is Unencrypted

– Network traffic can be monitored and captured

CSE 6392 By Dr. Donggang Liu 29

Contributing Factors (Cont’d)

• Lack of Security in TCP/IP Protocol Suite– Most TCP/IP protocols were not built with security in mind– Work is actively progressing within the Internet

Engineering Task Force (IETF)• Complexity of Management of Network Security• Exploitation of Software (e.g., Protocol

Implementation) Bugs– Example: Sendmail bugs

• Attacker’ Skills Keep Improving

CSE 6392 By Dr. Donggang Liu 30

Existing Internet Security Mechanisms

• Prevention– Firewall– Authentication, authorization– IPSEC/VPN– Access control– Encryption

• Detection– Auditing– Misuse detection– Anomaly detection

• Survivability• Response

Can we prevent all the intrusions from happening?

CSE 6392 By Dr. Donggang Liu 31

Existing Internet Security Mechanisms

• Security mechanisms implement functions that helpto prevent, detect, tolerate, respond to security attacks

• Prevention is ideal, but...– Detection seeks to prevent by threat of punitive action– Detection requires that the audit trail be protected from

alteration• If we can’t completely prevent attack from

happening, detection is the only option• There could be attacks we can’t detect, then live with

it - survivable system• Once detect the attack, then what? Active response!!!

CSE 6392 By Dr. Donggang Liu 32

Existing Internet Security Mechanisms

Prevent Detect Survive/Response

CSE 6392 By Dr. Donggang Liu 33

Unique Aspects of Intrusion Detection Problem

• The Whole System is as Strong as Its Weakest Point• The Root Cause of Intrusion Problem is Not

Computer, But Human Being• Ever Changing - Moving Target

– countermeasures by adversary

• Conflicting Requirements– Identity/authentication– Anonymity

CSE 6392 By Dr. Donggang Liu 34

Key Concepts

• Vulnerability– Flaws in system and/or networks that could be exploited to violate the

security policy of system or network– Examples

• strcpy() could result buffer overflow• 3-way handshake of TCP could result denial-of-service

• Intrusion– A specific execution of planed exploits of vulnerabilities to attempt to

• Access unauthorized information• Manipulate unauthorized information• Render system unreliable or unavailable

– Example• Break-in server of payroll department…• Crash the traffic control computer system

CSE 6392 By Dr. Donggang Liu 35

Key Concepts Cont’d

• Intrusion Detection (ID)– The art and science of identify attempted intrusions– Could be real-time or post-mortem

• ID usually involves– Monitoring and analyzing both user and system activities– Analyzing system configurations and vulnerabilities– Assessing system and file integrity– Ability to recognize patterns typical of attacks– Analysis of abnormal activity patterns– Tracking user policy violations

• Can Intrusion Detection Detect “Sniffering”?

CSE 6392 By Dr. Donggang Liu 36

Taxonomy of Intrusions

• Taxonomy – a way to classify and refer to threats(and attacks) by names/categories– Benefits – avoid confusion– Focus/coordinate development efforts of security

mechanisms• No standard yet• One possibility: by results/intentions first, then by

techniques, then further by targets, etc.– Associate severity/cost to each threat

CSE 6392 By Dr. Donggang Liu 37

Intrusion Taxonomy Example

• By results then by (high-level) techniques:– Illegal root

• Remote, e.g., buffer-overflow a daemon• Local, e.g., buffer-overflow a “root” program

– Illegal user• Single, e.g., guess password• Multiple, e.g., via previously installed back-door

– Denial-of-Service• Crashing, e.g., teardrop, ping-of-death, land• Resource consumption, e.g., syn-flood

– Probe• Simple, e.g., fast/regular port-scan• Stealth, e.g., slow/”random” port-scan

CSE 6392 By Dr. Donggang Liu 38

Brief History of Intrusion Detection

• In The Beginning…– Manual Intrusion Detection in practice

• System administrator manually monitor user’s activity• Ad hoc and non-scalable

• The Study of Intrusion Detection– Was started by James P. Anderson's 1980 technical report

• “Computer Security Threat Monitoring and Surveillance”

• Anderson– Introduced the notion of audit trails– Suggested that audit trails contain vital information that could be

valuable in tracking misuse and understanding user behavior– Formed foundation of host-based intrusion and IDS in general

CSE 6392 By Dr. Donggang Liu 39

Brief History of Intrusion Detection

• Dr. Dorothy Denning at SRI International– Developed Intrusion Detection Expert System (IDES) in early 80’s– Published “An Intrusion Detection Model” in 1987

• The first general intrusion detection model

• DIDS from UC Davis ~1990– DIDS (Distributed Intrusion Detection System) - Motivation,

Architecture, and An Early Prototype

• Network Security Monitor (NSM) ~1990– UC Davis's Todd Heberlein introduced the idea of network intrusion

detection in 1990

CSE 6392 By Dr. Donggang Liu 40

Brief History of Intrusion Detection

• GrIDS – Graph-Based Intrusion Detection from UC Davis1996

• EMERALD – Event Monitoring Enabling Responses toAnomalous Live Disturbances from SRI 1997

• NetSTAT from UC Santa Barbara 1998• Bro from International Computer Science Institute (ICSI) 1998• …

CSE 6392 By Dr. Donggang Liu 41

Taxonomy of Intrusion Detection

• Based on Detection Technique– Misuse detection

• Assumes that intrusions can be represented by a pattern or signature• Low false positive rate• Can only detect known intrusions

– Anomaly detection• Assumes that all intrusive activities are necessarily anomalous• Could potentially detect new intrusions• High false positive rate

• Based on Source of Audit Trail– Host based– Network based– Hybrid

CSE 6392 By Dr. Donggang Liu 42

Taxonomy of Intrusion Detection

• Based on Analysis Technique– Expert systems

• Primarily used for misuse detection• But could be used in anomaly detection as well

– Signature analysis– Petri nets– State transition analysis– Statistics– Neural networks– Machine learning– …

CSE 6392 By Dr. Donggang Liu 43

Evaluation Criteria of Intrusion Detection• Accuracy

– If an alert really reveals an intrusion?– Can be quantitatively measured by false positive rate (FPR)

• Completeness– Whether the IDS could detect all intrusions?– Can be quantitatively measured by true positive rate (TPR) or false negative

rate (FNR)• Scalability

– Whether the intrusion detection can keep up with the growth of the network ortraffic volume

• Robustness or fault tolerance– Whether the IDS itself is resistant to attacks?– If IDS is running on vulnerable host …

• Timeliness– How soon can the IDS detect the intrusion?– Real-time or post-mortem?

CSE 6392 By Dr. Donggang Liu 44

What’s Next After Successful IntrusionDetection?• You have discovered that there is an intrusion• You might want to find out

– How it happened– What vulnerability has been exploited– How to fix the problem

• What about the intruders themselves?– Will IDS tell you where the attack come from?

CSE 6392 By Dr. Donggang Liu 45

New Form of Intrusions

• Virus• Worm• Spyware• Logic Bomb• …

CSE 6392 By Dr. Donggang Liu 46

Open Problems in Intrusion Detection

• Does There Exist Undetectable Intrusion?