Using Automation to Enhance Cyber Workforce Development · • Presentation Ask an interesting...

© 2018 Noblis, Inc.

Using Automation to Enhance Cyber Workforce DevelopmentMike Cameron30 Jan 2018

2© 2018 Noblis, Inc.

Okay, so there’s a problem…


A lot of people are working on it


At the heart of all cybersecurity is a Curious Human

“What’s happening?”

“Why did it happen?”“Who is doing it?”

“What does it signify?”

“What action should I take?”


Cyber Workforce Development 101

Train them in the knowledge and skills they need to do their workFind enough curious humans

1. 2.


If everything goes well, it should work like this

MissionMission

TIME


“For every complex problem there is a solution that is simple, obvious, and wrong”

‐ H.L. Mencken

But…


Why is it wrong (or why is Mencken right)?

■ Everyone is fishing in the same pond!

■ Intel, DoD, Civil, and private sector are all looking for the same people…

■ …and there aren’t enough of them.


It usually ends up more like this

MissionMission

TIME


Let’s look at it a little differently

In words, the sum of all the people you have, with all their knowledge and skills, applied across all the work to be done

Workforce Effectiveness

� = ü(� + �)�


This view points out that what we’ve been dealing with is this…

This gets bigger slowly

This gets bigger very quickly02468

101214161820

1 2 3 4 5 6 7 8 9 10

Workforce Gap

Workload People

� = ü(� + �)�

…a race we can’t win


The solution is to reduce the work load (W ) while still growing the labor force

0

5

10

15

20

1 2 3 4 5 6 7 8 9 10

Chart Title

Workload People

Keep growing this

Reduce this!

� = ü(� + �)�


“The goal is to entirely automate what can be automated and to improve the performance of human analysts where automation is not possible – moving them away from data handling tasks and into higher‐level reasoning and analysis.”

Technology can reduce the effective workload on each human


We can’t actually reduce the TOTAL workload, but…

…we can segment the work and let machines do the heavy lifting on the data and give humans more time for higher-level reasoning and analysis


The collision of Technology and Tradecraft

■ Fast■ Good at frequent, high-volume tasks■ Machine Learning

• Classification• Prediction

■ Prone to False Positive errors

Machines

■ Slow ■ Superb at novel tasks

• Understanding incomplete data• Draw meaning from what’s missing• Relating similar past experience

■ Prone to False Negative errors

Curious Humans


Technology should help humans do what only humans can do…

Needle in a Haystack

Machine Learning = Needle in a

Stack of NeedlesHumans =


Not everyone agrees…

Two results from the same query…


Technology contributes to the workforce in three ways

1. Data Science

• Data cleansing• Data modeling• Characterization

and tagging• Analysis• Presentation

Ask an interesting question

GETthe data

EXPLOREthe data

MODELthe data

Communicateand visualize

the results.

The Data Science Process



2. Machine Learning

• Frequent, high-volume tasks• Classification (“spam” or “not

spam”)• Prediction (if this, then…”

InputHidden

Output

Neural Network



3. Workflow Automation

• Alerting• Dissemination• Rule-based Audit• Archiving


Integrating Technology—Human-Computer Collaboration

• Deal with novel things• Examine the outliers• Draw conclusions• Make decisions

• Alert• Audit• Report• Archive

• Look for known things• Find relationships between things• Find the outliers• Eliminate the unnecessary data

• Understand the nature of the data• Develop algorithms to examine

the data• Clean it and prepare it for analysis

Data Handling Reasoning Data Handling

Workflow AutomationTradecraftMachine LearningData ScienceUnstructured Data


The collaboration involves many disciplines

Engineering and Data ScienceHigh Performance

Computing

Data Analytics

Security Engineering

Software Development

Algorithm Development

Machine Learning

Neural Networks

AI

Python

R

Cyber Tradecraft

Threat Intelligence

Forensic Analysis

Insider Threat

Threat Hunting

Incident Response


The key is to define the KSAs, and then perfect the collaboration

and mission integration

So what does the future cyber workforce look like?

Data Science/ Statistics

Cyber Analytics

Machine Learning/ Computer Science


Human-Computer collaboration is already showing up

■ Similar patterns• Machines sort data• Humans review the output and tag legitimate threats• Tagged data is fed back into the ML algorithms

■ Two examples■ CSAIL-PatternEX

• 40 million log lines per day• 85% detection rate

■ F-Secure• Threats normally live on networks ~90 days before detection• Working toward goal of detection within 30 minutes


In practice…Analytic Tool Development

■ Primary objective is to develop analytic tools…migrate them to the Ops floor

■ Primary skill sets are software development and data science

■ Work in a tight loop with the Threat Hunters

■ 650% improvement in threat searches


In practice…Cyber Insider Threat Analysis

■ Primary mission is to detect and prevent harm from insider threats

■ Primary skills sets are forensic analysts and counter intelligence

■ Embed machine learning and data science into the operational team


To enhance the workforce, technology must…

■ Be low friction…fit easily into the operational environment

■ Be granular...tools should perform simple tasks, but can be combined to perform more complex tasks.

■ Conform to existing workflows and work the way the analysts work


?Mike CameronDirector, Cyber SolutionsNoblis(571) [email protected]

Using Automation to Enhance Cyber Workforce Development · • Presentation Ask an interesting...

Documents

Transcript of Using Automation to Enhance Cyber Workforce Development · • Presentation Ask an interesting...