Hit ‘em Where it Hurts: A Live Security Exercise on Cyber Situational Awareness
-
Upload
adam-doupe -
Category
Technology
-
view
677 -
download
2
description
Transcript of Hit ‘em Where it Hurts: A Live Security Exercise on Cyber Situational Awareness
Hit ‘em Where it Hurts:A Live Security Exercise on
Cyber Situational Awareness
Adam Doupé, Manuel Egele, Benjamin Caillat, Gianluca Stringhini, Gorkem Yakin, Ali Zand, Ludovico
Cavedon, and Giovanni Vigna
University of California, Santa Barbara
ACSAC 2011 – 7/12/11
What Are Live Security Competitions?
• AKA Hacking Competitions• Useful educational tool for teaching computer security• Born as a way to showcase security skills
– DefCon’s CTF
• Various forms– Challenge set (DefCon quals, iCTF challenges, CMU’s
competition, DIMVA competition, RuCTF)– Capture the flag (DefCon, iCTF 2003-2007, CIPHER)– Other designs
• Attack-only (e.g., iCTF 2008)• Defense-only (e.g., Cyber Defense eXercise)
Doupé - 7/12/11
Why Live Security Competitions?
• Real-time factor enhances understanding• Forces teams to:
– Analyze unknown services/binaries– Defend systems from attack– Utilize different security skills– Work as a team– Create novel tools
Doupé - 7/12/11
Key Insight
• Security competitions can be designed to generate datasets for research
• In the 2010 international Capture The Flag (iCTF), we structured the competition to create a Cyber Situational Awareness dataset
Doupé - 7/12/11
Situational Awareness
• By putting perceived events into the context of the currently executing mission, one can improve decision making
• Mission– Series of tasks that an organization wishes to carry
out
• Task– Discrete step that is carried out using a service
• Service– Provided to users to accomplish a task
Doupé - 7/12/11
Cyber Situational Awareness
• Situational awareness extended to the cyber domain
• Large organizations constantly under attack– Which attacks are important?– Which assets are important?
• “What if” scenarios
Doupé - 7/12/11
Overview
• Live Security Competitions• Situational Awareness• Design of the 2010 iCTF• Cyber Situational Awareness Metrics• Lessons Learned• Conclusion
The 2010 iCTF: A Cyber SA Competition
• Introduced the concept of cyber-mission• “Not all attacks are created equal”• Participants must be aware of cyber-
missions and cyber-assets• Attackers must time their attacks to cause
the maximum amount of damage
The Setting
• Teams are part of a coalition to bring down the rogue nation of Litya
• LityaLeaks site used to leak description of Litya’s cyber-missions
• Litya’s network protected by a firewall and an IDS– If an attack is detected, nation’s access is shut off– Nations can bribe network administrator
• Litya has a botnet in each nation, stealing their money– If botnet is disabled, nation’s access shut off
• Money made by solving side challenges.
CARGODSTR-TQ-1442
COMSAT-WK-1127
SEDAFER-GOT-BKT-8217
DRIVEBY-DEPLOY-QFK-9751
Doupé - 7/12/11
Petri-net Representation of Mission
Service 1 Service 2 … Service 10
.. .
.
Service 1 Service 2 … Service 10
The Bank
ScoreBot
.. .
.
InternalNetwork
…
VPN server
Botnet C&C
The Bank
Service 1 Service 2 … Service 10ScoreBot
.. .
.
InternalNetwork
…
VPN server
Firewall/IDS
Botnet C&C
The Bank
Briber
Flag Submission
Service 1 Service 2 … Service 10ScoreBot
.. .
.
InternalNetwork
…
VPN server
Firewall/IDS
Botnet C&C
The Bank
Briber
Flag Submission
Service 1 Service 2 … Service 10ScoreBot
.. .
.
Challenges
ScoreBoard
LityaLeaks
Service 1
InternalNetwork
…
VPN server
Firewall/IDS
Service 2 … Service 10 Botnet C&C
The Bank
ScoreBot
Briber
Flag Submission
.. .
.
Challenges
ScoreBoard
LityaLeaks
Doupé - 7/12/11
Competition Overview
• December 3rd 2010 ~8 hours• 72 teams• ~900 participants (largest at the time) • 7 of 10 services compromised• 39 teams submitted 872 flags• 69 of 72 teams solved at least 1 challenge• 37 GB of traffic
Doupé - 7/12/11
Analysis of iCTF Data
• Use the data to validate models and theories
• We introduce two Situational Awareness metrics:– Toxicity
• Capture the amount of damage an attacker has caused
– Effectiveness• Capture how effective the attacker was at causing
damage
Analysis – CAD - Criticality
• C(s, t): service criticality [0,1]– Expresses the criticality of service s at time t– Function can have any shape
• iCTF: 1 when service active, 0 otherwise
Service: MostWanted
Analysis – CAD - Attacker
• A(a, s, t): attacker activity [0, 1]– Represent the attacker’s activity with respect
to a service– Can have any shape
• iCTF: 1 when team attacked a service, 0 if no attack
Team: PPP Service: MostWanted
Analysis – CAD - Damage
• D(s, t): Damage to the attacker [0, 1] – Represents the penalty for performing an
attack against service s at time t– Can have any shape
• iCTF: 1 when service is inactive, 0 when active
Service: MostWanted
Analysis – Toxicity
Analysis – Effectiveness
Analysis – Toxicity of PPP
Team: PPP Service: OvertCovert
Analysis – Toxicity and Effectiveness
Doupé - 7/12/11
Overview
• Live Security Competitions• Situational Awareness• Design of the 2010 iCTF• Cyber Situational Awareness Metrics• Lessons Learned• Conclusion
Doupé - 7/12/11
Lessons Learned
• The Good– Pre-competition information prepared teams who
took advantage– Winning team automatically qualified for DefCon
• The Bad– Structure of the competition was complex and was
understood by a subset of the teams– Services too hard
• The Ugly– Intentionally put a root backdoor into bot– Losing points sucks
Conclusions
• Live security exercises great for learning and security education
• They can be designed to create a research dataset
• Designed the 2010 iCTF to produce the first publically available dataset on CSA
• Presented SA metrics: toxicity and effectiveness
Doupé - 7/12/11
Questions?
Data: http://ictf.cs.ucsb.edu/data/ictf2010/
Email: [email protected]: @adamdoupe
Service Exploitation