Converge ppt

40
Check Yo’self What Ice Cube taught me about security metrics

Transcript of Converge ppt

Page 1: Converge ppt

Check Yo’selfWhat Ice Cube taught me about security metrics

Page 2: Converge ppt

DisclaimerAll opinions and thoughts in this presentation are my own and do not represent my employerAll use of Ice Cube’s image, lyrics, movies, and music are for storytelling, not for profitThe data used in this presentation comes from my employer, but is anonymized to protect the guilty and innocent

Page 3: Converge ppt

OverviewSpeedQualityCoverageCharts & Takeaways

Quality

Coverage

Speed

Page 4: Converge ppt

SpeedIf you're foul, you better run a make on that license plate

You coulda had a V8Instead of a tre-eight slug to the cranium

I got six and I'm aimin em

Page 5: Converge ppt

SpeedHow fast did you find the breach?How fast did you stop the breach after it happened?How fast did you clean it up?

How fast did you go from What? to So What? to Now What?

Page 6: Converge ppt

Speed

Comp Find Alert Give a s&*t?

Taking action

Stop the s&*t

Clean up the s&*t

You better check yourself before you wreck yourselfCause I'm bad for your health, I come real stealthDropping bombs on your moms, f*** car alarmsDoing foul crime, I'm that fool with your Alpine

- Check Yourself – Ice Cube

Page 7: Converge ppt

Intellectual Honesty

Time’s are all in the same time zone – goes without sayingThe time of compromise is when something changed in the system – not when you or your system found itMissing that key fact means you miss

Quality of intelligenceCoverage of intelligence

Time dropper hit the file tableTime A/V reported finding the backdoor

Difference = 7 months, 8 days, 13 hours, 34 minutes, 7 seconds

Page 8: Converge ppt

Trusted sources of truth

HostEvent logsMFTs

NetworkFirewall logsNetflow logsSMTP logs (for phish)Proxy logs (for watering-holes)

Page 9: Converge ppt

Comp-to-Find

Speed of intelligence deployment to your tools

How fast did you get it?How fast did you know it?How fast did you use it?

Frequency of scansAlertness of users

Collection

Processing

Exploitation

Dissemination

Tasking

Comp Find Alert Give a s&*t?

Taking action

Stop the s&*t

Clean up the s&*t

Page 10: Converge ppt

How to find?Host

AV logsEvent logsNagiosTripwire

NetworkIDS/IPS alertsFirewall logsProxy logsEmail gateway logs

Page 11: Converge ppt

Find-to-Alert

Speed of the sensorAre your alerts backing up on a DB somewhere?How often are sensors reporting back to their console?

Knowledge of user (protein-based sensor)

Do they know how to report shadiness?

Comp Find Alert Give a s&*t?

Taking action

Stop the s&*t

Clean up the s&*t

Page 12: Converge ppt

Alert-to-Give a s&*t

How long do alerts linger?How long do emails about incidents bounced around inboxes?SIEM logs

When analyst acknowledges the alert

Comp Find Alert Give a s&*t?

Taking action

Stop the s&*t

Clean up the s&*t

Page 13: Converge ppt

Give a s&*t-to-taking action

Speed of triage & initial analysisKnowledge of internal organization

Do your responders know who to call?

Comprehensiveness of response plans and SOPs

Comp Find Alert Give a s&*t?

Taking action

Stop the s&*t

Clean up the s&*t

I found the APT !!!

Page 14: Converge ppt

Taking Action-to-Stopping the s&*t

HostEvent log (shutdown)DHCP logAV log (deleted malz)Phish deleted

NetworkACL in switchIPS rule change logIP block added to routerFirewall block addedProxy log

Not when the rule was added, but when it was confirmed to be working

Comp Find Alert Give a s&*t?

Taking action

Stop the s&*t

Clean up the s&*t

Page 15: Converge ppt

Stopping-to-cleaning up the s&*t

How long the business was impacted by the breach?Did the containment strategy conflict with or support recovery?How fast did you find other breaches?How effective was your recovery?

The fed’s preferred recovery method

Comp Find Alert Give a s&*t?

Taking action

Stop the s&*t

Clean up the s&*t

Page 16: Converge ppt

QualityI hate motherf**kers claimin that they foldin bank

But steady talkin s&*t in the holding tankFirst you wanna step to me

Now you’re a** screamin for the deputy

Page 17: Converge ppt

QualityIt’s great that you’re fast, but are you any good at it?Easy to confuse quality with forensic soundnessEasy to confuse quality with expensive blinking boxesQuality really measures

Are you focusing on what’s really important (customer)?Are you focusing on what really works (performance)?Do you track failures as much as you do successes (defects)?Do you learn from mistakes and do you repeat them (improvement)?

Page 18: Converge ppt

Comp Find Alert Give a s&*t?

Taking action

Stop the s&*t

Clean up the s&*t

First time rightIn this process, how often were mistakes made

Do you track and categorize mistakes and misfires?

How many times did you

miss the breach?

Did the alerts go to the

right place the first time?

Did the person

viewing the alert make the right

call?

Did the person who gives a s&*t do the right

thing?

Did the actions

actually stop the breach?

Was your cleanup

effective?

Page 19: Converge ppt

Measuring QualityGet granularAvoid “other” or “unknown”

If given an option, analysts will choose “other” two out of every three times.

Set goalsWhat’s acceptable performance?

Page 20: Converge ppt

Forensics & Kill Chain

Reconnaissan

ce

Weaponizatio

n

DeliveryExploitationInstallation

C&C

Actions on

Objectives

Increasing ferocity of Ice Cube movie characters

Increasing cost of response and recovery

Page 21: Converge ppt

NetworkReconnaissanceDeliveryC2AoO

HostExploitInstallationAoO

Forensics & Kill Chain

Page 22: Converge ppt

ReconnaissanceWeaponizationDeliveryExploitationInstallationC2AoO

Forensics & Kill Chain

Know every system/person involved in the incident and how they performed – relative to the Kill Chain

PLAYING WITH MY KILL CHAIN

IS LIKE PLAYING WITH MY EMOTIONS

Page 23: Converge ppt

CoverageTricks wanna step to Cube and then they get played

Cause they b&*ch made pullin out a switchbladeThat's kinda trifle, cause that's a knife-o

[here’s an] AK-47, assault rifle

Page 24: Converge ppt

CoverageAre you looking for the right things in the right places?

Filenames in IDS?IP addresses in AV logs?

What percentage of your install base are you monitoring?First, check yo’self

Use the Kill ChainFind your gaps

Page 25: Converge ppt

Check Yo’selfHow do you get got?

Phishing?Watering holes?Thumbdrives?Websites getting popped?

For one thing, you don’t know how the f**k my company be muthaf**king

owned.

Page 26: Converge ppt

Check Yo’self

Recon Weapon Deliver Exploit Install C2 AoO

#1

#2

#3

#4

#5

Attacks stopped by Kill

Chain

#6

Page 27: Converge ppt

Check Yo’self

Recon Weapon Deliver Exploit Install C2 AoO

$$$

$$$$

$

$$ $

$$

$$$

Cost of the intrusion

Page 28: Converge ppt

Check Yo’self

Recon Weapon Deliver Exploit Install C2 AoO

$$

$$$$$$

$$$

Cost of countermeasur

es

$

Page 29: Converge ppt

Finding GapsLack of process

Misapplying IntelBad deployment of web applications

Lack of TrainingDevelopers building insecure apps

Lack of technologyBuy only when you have a clear blind spot

Not every gap in yo’ security needs to be filled with cash

money

Page 30: Converge ppt

Check yo ‘netDo you have every network ingress/egress point monitored?

3rd Parties/SuppliersVPNMobile/BYOD

Do you have monitoring on every network service?

FTP, SFTP, Web, SMTP, Telnet (yes, telnet)Cloud services (*aaS)

Gary’s manager found an un-instrumented PoP on the

network

Page 31: Converge ppt

Check yo ‘boxesWhat is your host logging policy?Do your logs go to a central location? Do you have a method to search the endpoints and servers for IOCs?How agro are your patching policies?

Will a Java patch f’ your network? http://bit.ly/1pTiodM - for other

derp-ables referring to “the APT”

Page 32: Converge ppt

TakeawaysHere to let you know boy, oh boy

I make dough but don't call me DoughBoyThis ain't no f**kin motion picture

A guy or b^*ch-a, my fool get wit'chaAnd hit ya, takin that yack to the neck

So you better run a check

Page 33: Converge ppt

Telling your story to management

Know the real cost of your breach

Your timeYour team’s timeCost of recoveryClient’s lost productivityData loss

Cost of R&DProfit Margin

Know the real cost of countermeasures

Training costs should include time away and travelProcess improvements requires good data, discipline, and expertiseIf you’re buying a new tool, double the cost of deployment and add 50% to annual O&M

Page 34: Converge ppt

Telling your story to management

6/19/1

0 11:4

5

4/8/10

21:52

3/7/10

4:00

1/3/10

7:55

12/16/

09 22

:32

8/29/0

9 1:28

5/23/0

9 16:3

2

5/20/0

9 15:2

9

4/25/0

9 14:0

3

4/9/09

16:28

3/13/0

9 2:28

3/6/09

14:38

1/29/0

9 20:0

9

1/19/0

9 7:48

12/28

/08 4:0

7

11/4/0

8 8:00

9/10/0

8 10:3

3

6/4/08

9:47

4/19/0

8 7:39

4/4/08

7:25

3/22/0

8 16:0

3

3/11/0

8 2:25

2/6/08

16:11

1/10/0

8 20:5

5$ K

$20 K

$40 K

$60 K

$80 K

$100 K

$120 K

$140 K

$160 K

$135 K

$99 K

$39 K

$119 K

$100 K$97 K

$144 K$152 K

$47 K

$17 K

$110 K

$7 K$6 K$6 K

$45 K$56 K

$42 K

$114 K

$142 K

$122 K$119 K

$64 K

$113 K

$7 K

Per-event cost of our large-scale intrusions (Jan ‘12 – Jul ’14)

(# of days of full-scale response) x (daily rate of employee) x (# of employees involved in the response)

Page 35: Converge ppt

What point in the Kill Chain are attacks being stopped?

Does it cost more to respond to events higher in the KC?

Telling your story to management

Recon Deliver Exploit Install C2 AoO0

100

200

300

400

500

600

700

800

900

1000

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

Num

ber

of In

cide

nts

Day

s

Page 36: Converge ppt

What systems are catching attacks from “the APT”

Telling your story to management

IDS29%

Host-Based Scanner12%

AV12%

Proxy Logs7%

User Report6%

Email Scanner6%

Frequency Analysis5%

Monthly Host Checker4%

IP/Domain Hotlist4%

SIEM Correlations4%

Event Logs3%

Other2%

Netflow2%

3rd Party Notification2%

Cloud-based Proxy1%

IPS1%

Commercial Malware Analysis appliance1%

Registry Scanner1%

Email Logs1%

Page 37: Converge ppt

Don’t buy me another chirping box

Telling your story to management

IDS Crmcl Malware Analysis Device

McAfee User Report Email Scanner

3rd Party (Other)

Event Logs Proxy Logs0

1

2

3

4

5

6

7

8

9

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

Detection Tool

# o

f Fa

lse

Posi

tives

Day

s of

Inve

stig

atio

n

Page 38: Converge ppt

5 10 15 20 25 30 35 400

5

10

15

20

25

f(x) = − 0.0957872009672683 x + 12.2788185508248R² = 0.0181875582165211

# of Analysts on IR Team

# o

f Day

s of

Ful

l-sca

le R

espo

nse

More people, more problems

Practically no correlation between having more people and being able to responding faster

Page 39: Converge ppt

Training vs. Tools

Cost of Training an Analyst for a small network – 10K hosts

SANS Course & Certification = ~$5,500Travel & Meals = ~$1,500Time Away from office = ~$1,750Cost of OS IDS appliance(s) & management servers = $20,000Total = $28,750

Cost of a commercial IDS solution = ~$50,000 - $150,000

Cost of a commercial SIEM product = ~$150,000-$200,000

Annual cost of MSSP services = ~$60,000-$120,000

Page 40: Converge ppt

Questions?@DaveTrollman

(since Jul 10, 2014 – 245PM)