Butterflies, Black Swans, and Beautiful Security Metrics

Curtis ColemanVisiting Assistant Professor

Cybersecurity Program DirectorOffice: PEC-230B

Phone: 405.425.5472Cell: 405-990-1842

Week 1

DAY 1

- Who am I?

- The Butterfly Effect

- The Black Swan Metaphor

- 10 Lessons from Last Year’s Black Swans

- My Black Swan - 2015

- Beautiful Security Metrics

Who am I?

The Butterfly Effect

• Tiny changes within a complex system lead to results that are impossible to predict

• The flapping of a butterfly’s wings could create tiny changes in the atmosphere that lead to violent weather conditions elsewhere on the planet.

• In 1961 due to weather prediction modeling performed by meteorologist Edward Lorenz. He found that changes that should have been statistically insignificant led to completely different weather scenarios.

Image Source: http://www.intekworld.com/butterfly-effect/

Imagine . . .

• Sr. Network Engineer Connor gets an email with an attachment from a LinkedIn friend telling him about an exciting Network Engineering Conference in Las Vegas. He opens the attachment.

• The attachment is an Adobe PDF flyer of an upcoming Software Defined Network Forum offered at the conference. Connor is already planning to attend the forum. He closes the PDF flyer and goes back reading his emails.

• His computer is now running a Gh0st RAT application that connects to a site on the Internet that is used by the bad guys to control his computer.

APT . . . Stealthy, Data Focused, Targeted

Top 10 Personal Cyber Hygiene

1. Choose your password carefully2. Regularly update your software3. Make regular backups4. Secure your WiFi access (use VPN on public WiFi)5. Secure your mobile device (password/biometric,

encryption)6. Protect your Personal Information and Data (2FA, SSL,

AES-256 encryption)7. Be care when using email; know how to spot

phishing, don’t click on unknown links/files8. Download software from official sites9. Use a personal firewall10. Install anti-Malware protection

The Black Swan Metaphor

The Black Swan: The Impact of the Highly Improbable, by Nassim Taleb

• Prior to the discovery of the Australian

black swan, everyone in the world

assumed all swans were white

• The importance of the black swan -

something that falls outside the realm of

regular expectations -- is that a single such

sighting can invalidate a general statement

(i.e. "All swans are white.").

• What we don't know is frequently more

important than what we do know (or think

we know). There is compelling evidence

that people -- especially so-called experts -

- tend to overestimate what they know and

underestimate the uncertainty that is

derived from those things they don't know.

Thus the need for metrics!

Image Source: https://www.amazon.com/dp/B00139XTG4/ref=dp-kindle-redirect?_encoding=UTF8&btkr=1

My Black Swan - 2015

Image Source: https://krebsonsecurity.com/2016/03/seagate-phish-exposes-all-employee-w-2s/

Top 10 Lessons of 2017-2018 Cybersecurity Black Swans

1. System patches were not properly maintained2. Poor vetting of 3rd Party security3. Need limitations and alerts on bulk download of data4. Did not monitor databases for abnormal behavior and

violations of Segregation of Duties5. Sensitive data was poorly handled6. Poor change management control resulted in

accidental exposure7. Password policy was not enforced8. Did not close terminated accounts9. Poor management of service accounts and shared

password accounts10. Poor Cyber Situational Awareness Program

Cyber Situational Awareness Program

October is National Cyber Security

Awareness Month

1. Week 1: Oct. 1–5: Make Your Home a Haven for Online Safety

2. Week 2: Oct. 8–12: Millions of Rewarding Jobs: Educating for a Career in Cybersecurity

3. Week 3: Oct. 15–19: It’s Everyone’s Job to Ensure Online Safety at Work

4. Week 4: Oct. 22–26: Safeguarding the Nation’s Critical Infrastructure

How do we discover what we don’t know?

How do we discover the unknowns before they become Black Swans?

Image Source: https://lexfun4kids.com/iceburg

The Power of Measuring

How do we discover the unknowns before they become Black Swans?

When you can measure what you are speaking about, and

express it in numbers, you know something about it; but when

you cannot measure it, when you cannot express it in numbers,

your knowledge is a meager and unsatisfactory kind; it may be

the beginning of knowledge, but you have scarcely, in your

thoughts, advanced to the state of science.

—William Thomson, Lord Kelvin, 1883

Possible Operational Metrics

How do we discover the unknowns before they become Black Swans?

Summary and Trends

• Number of exposures fixed

• Number of Incidents prevented

• Systems down due to security incidents (and time to fix)

• Security operating efficiency

• Technical Compliance

• Successful attacks (not initially detected)

• Unsuccessful attacks (detected and blocked)

Incidents

• Total number of incidents per reporting period

• Total number of incidents per fiscal year

• Total time to detect and mobilize

• Impact of sever incidents (time and cost to repair, impact on business)

• Systems un-available due to security incidents

Risk and Compliance• Top 10 exposures• Top 10 emerging threats• Likelihood of impact• Projected cost of impact• Projected Remediation Costs• Time since last assessment• Time since last content update

Projects• Projects % complete & period target

Financial• Cost of all controls (capitol and

operational)• Cost of doing nothing• Efficiency / Modified ROI

What the CISO wants to know –

These are high-level metrics that

show security performance against

desired levels of service. Each

security manager will need to

formulate what they need to measure.

Conceptual Security Balance Scorecards

How do we discover the unknowns before they become Black Swans?

Conceptual Security Balance Scorecards

How do we discover the unknowns before they become Black Swans?

Conclusion

• You must look beyond conventional modes of defense to achieve a security posture that is dynamic, not static.

• This requires:

1. Cyber Situational Awareness

2. Measurements

3. Information Sharing with Management & Employees

• The audience take-away includes:

1. Use National Cyber Security Awareness Month to launch your own company Cyber Situational Awareness program

2. A Pragmatic Operational approach to Cybersecurity Metrics

3. 2 Model Security Balance Scorecards for information sharing

“Praise be to the LORD my rock, who trains my hands for war, and my fingers for battle.” Psalm 144:1