CYBERSECURITY DECISION PATTERNS AS ADAPTIVE …...May 02, 2014 · SDSF 2016 November 16, 2016...

SDSF2016 November16,2016

CYBERSECURITYDECISIONPATTERNSASADAPTIVEKNOWLEDGEENCODINGINCYBERSECURITYOPERATIONS

Sponsor:StevensIns>tuteofTechnologySchoolofSystemsandEnterprises

ByMr.KeithD.WilleL

4thAnnualSERCDoctoralStudentsForumNovember16,2016

20FStreetNWConferenceCenter20FStreet,NWWashington,DC

www.sercuarc.org

SDSF2016 November16,2016 2

BoLomLineUpFront

• Uniqueresearchcontribu<on:― CybersecurityDecisionPaLerns(CDPs)o  Captureandreuseanomalyprocessingknowledge

― …forprocessinganomaliesin11workflowphases― …toincreasequan<tyofknown-knownanomaliesprocessed

― Acybersecurityopera>onsframeworkconducivetoiden<fytherole,fit,func<on,andimpactofanycybersecuritysolu<on

• UseofhumansubjectmaPerexperts(e.g.,“clinicaltrialapproach”)notviable― Controlgroupv.treatedgroupo  #peopleavailabletoosmallo  Abilitytocontrolvariabilityinpeopleverydifficult

• Alterna<vetrialapproach:modelingandsimula<on― Systemdynamicsmodeling(SDM)asasurrogateforpeople― UseSDMtotesthundredsofthousandsofdatapoints


CDPAbbreviatedTemplate1of3

Descrip>on Name Evoca<vename thatemerges fromnatural language to reference thisproblem/

solu<onpairing. Context The situa<on, the circumstances in which the problem is solved. The context

imposesconstraintsandhelpsiden<fytherela<veimportanceoftheforces.Thecontextmayincludetac<calandstrategicperspec<ves,forexample:•  Strategic

o  Organiza<on:<organiza<oniden<fier>(e.g.,accoun<ng)o  StrategicFunc<on:<func<on>(e.g.,collec<onsworkflowmanagement)o  Capability:<details>(i.e.,descrip<onofdesiredresults)o  Ac<vity: <details> (i.e., formal collec<onof ac<vi<es producing desired

results;may use the genericworkflow as a standardway to representac<vity)

o  Task:<details>•  Tac<cal (design note:wedonotwant to turn thepaPern repository into a

<cke<ngsystem,thedetailsbelowaretoodetailedforapaPern.Theyhavetheirplaceandmaybereferencedviasome<cket#intheExamplesessen<alelement,buttheybelonginacomplementary,separatesystem)o  Physicalloca<on(s):<details>o  Networkiden<fica<on:<details>o  Tool:<details>



Descrip>on

Problem Thespecificproblemthatneedstobesolved.Describetheproblem,therootneedas a coarse abstrac<on and the specific need in less coarse terms. The problemdescribesthewhatandshouldnotincludethesolu<on(thehow).Attheleast,theproblemdescrip<onshouldinclude:•  Therootoftheproblemis…<therootofthecause…maybeanarchetype>to

help frame the problem and to iden<fy exis<ng approaches to exis<ngarchetypeproblems

•  The desired result is… < express desired result agnos<c of solu<on thatproducestheresult>

Spocngtheproblem:•  Observables(indirect,symptoms,indicators):<details>;•  Observables(direct,problemsource):<details>



Descrip>on Solu>on Describehowtosolvetheproblem;howtoproducethedesiredresultexpressed

intheproblem.Theremaybemul<plepoten<alsolu<ons;thebestisrelevanttothecontextandresolvesthehighestpriorityforces.Thesolu<ondescrip<onmayreadlikeaninstruc<on/impera<ve.No<onalsolu<onstructure:•  Monitor:for<details>•  Detect:observable<details>•  Characterize:known-known,known-unknown,unknown-unknown,unknown-

known•  No>fy:who<details>accordingtodetectdetails•  Triage:priority<details>accordingtodetectdetailsandtac<calandstrategic

mission•  Escalate:tolevelofexper<se<details>perdetectandtriagedetails•  Isolate:containment<details>accordingtotac<calandstrategicdetails•  Restore: achieve interim opera<ons <details> according to tac<cal and

strategicmission•  RootCauseAnalysis(RCA):<details;or,explicitreferencetoexternalreport>•  Recover:achievenormalopera<ons<details>•  Feedback:systemicfeedback<details>;CDPfeedback<details>


StrategicDriver

K n o w y o u r e n e m y a n d k n o w y o u r s e l f a n d y o u n e e d n o t f e a r i n a h u n d r e d b a t t l e s . – S u n T z u


People

Process

Technology

Environment

Results

Consumption

DesiredOutcome

People

Technical

Natural

SecurityServices

SecurityMechanisms

VulnerabilitySpace

Mission RiskTolerance

MissionPriorities

Risk Posture

DesiredSecurityPosture

ContinuousMonitoring

CurrentSecurityPosture

GapAnalysis

CorrectiveAction Resource

Constraints

OrganizationalEfficacy

OrganizationalVision Strategy

Marketplace

Customers

Vendors

LocalEnvironment

Statistics

Etc.

OrganizationalMission

Cyber

Legislative

EtcEtc

Policy

Regulatory ProcedureEtc

Contracts

SLAs

ThreatSpace

Workflow(Asset Space)

Safeguards

ExternalThreat

Information

Compliance Drivers

ExternallyImposed Self-ImposedNegotiated

perform

using

within an

to produce

for

to bringabout a

potential disuptions to

protect workflow

weaknesseswithin

tolerance for workflowdisruption drives

drive

influenceshelps define

drives

monitors

captures

providesactual for

drives

remediates risk in

providestarget for

limit

definesapplicable

preserve

withindrivesdrives

drives drives drive

monitors

fulfills

capacity to fulfill

influencepartiallyaddress

guides tactical execution

informsinfluences

guides strategic execution

StrategicContext


CybersecurityOpera>ons11PhaseWorkflow

1.  Monitor: ongoing observation with intent to raise awareness 2.  Detect: indicator of anomaly 3.  Characterize: known-known, known-unknown, unknown-unknown, unknown-known 4.  Notify: first tier support 5.  Triage: determine priorities 6.  Escalate: send to subject matter expert(s) 7.  Isolate: contain threat or threat effects 8.  Restore: restore effective operations even at diminished efficiency 9.  Root Cause Analysis: identify root cause of problem 10.  Recover: recover effective & efficient operations to desired performance level

11.  Feedback: minimize recurrence and effects of recurrence


Quan>fiedTac>calContext(Quan>ty)

Monitor

Detect

Notify

Triage

Qd is quantity of anomalies detected

t0 tfinal

Escalate

Isolate

Restore

RCA

Feedback

Recover

Characterize

Indicator

å (Qd)

Qc is quantity of anomalies characterizedå (Qc)

Qn is quantity of anomalies notifiedå (Qn)

Qt is quantity of anomalies triagedå (Qt)

Qe is quantity of anomalies escalatedå (Qe)

Qi is quantity of anomalies isolatedå (Qi)

Qc is quantity of anomalies restoredå (Qr)

QRCA is quantity of anomalies RCA’då (QRCA)

Qv is quantity of anomalies recoveredå (Qv)

Qf is quantity of anomalies fedbackå (Qf)

Qprocessed = S (Qd, Qn, Qe, QRCA, Qf)Qt, Qi,Qr, Qv,Qc,


ResearchHypothesis

CDPsEffectonQuan>tyofKnown-KnownAnomalies

H0UsingCDPsinthecybersecurityopera<onsworkflowtoprocessknown-knownanomaliesresultsinthesameorlessnumberofknown-knownanomaliesprocessedthanwhenusingthecurrentmethod.Ho:Quan<tyμNewProcess<=Quan<tyμOldProcess+X1;or,CDPsdonotprovideimprovementinquan<tyofknown-knownanomaliesprocessed.

Ha UsingCDPsinthecybersecurityopera<onsworkflowtoprocessknown-knownanomaliesresultsinaquan<tyofknown-knownanomaliesprocessedgreaterthanthequan<typrocessedwhenusingthecurrentmethod.Ha:Quan<tyμNewProcess>Quan<tyμOldProcess+X1;or,CDPsdoprovideimprovementinquan<tyofknown-knownanomaliesprocessed.

1X = (Quantity µ OldProcess * 25%) Assumption: quantity of anomalies processed will increase by 25% without CDPs; therefore, CDP affect must be at least 25% greater than baseline.


ExperimentalDesign

• KnowledgeManagementinCybersecurityOpera<ons― Role,fit,func<on,andimpactofCDPsonsecurityautoma<on

PeopleEnhanced

Cognition(PEC)

MachineEnhanced

Cognition(MEC)

+Security

Automation

++

-BPeople

RoteTasks

PeoplePerformance(UniqueExperienceOpportunities)

+

-

WhatNeedstobeDone

+B

R

+

B : balancing loop

R : reinforcing loop


SDMDevelopmentandResultsGenera>on

1JohnsHopkinsUniversityAppliedPhysicsLab(JHUAPL)IntegratedAdap<veCyberspaceDefense(IACD)

1

• Controlgroup:anomalyprocessingwithoutCDPs• Treatedgroup:anomalyprocessingwithCDPs


Quan>fyingManualProcessing(ControlGroup)

Ponemon2014DataBreachReport•  350respondentsonanomalyresponse<mesinvariousphases•  Fromreportdetails,derived:

–  Anomalyprocessingscale–  Rangesofdata–  Specificdatawithinranges–  Manualprocessingprobabilitydistribu/onfunc/ons(PDFs)

•  Perworkflowphase

∴PDFsarebasedonreal-worldexperience


Quan>fyingMachineProcessing(Influence)

JHUAPLIACDPilotResults•  Introducingsecurityautoma<on•  Manualprocessing(pre-pilot):

–  Detec>ontodecision:bestcase:10minutes;worstcase:11hours–  Ac>on:averageof45minutes–  Capacity:1perperson;notscalable

•  Machineprocessing(post-pilot):–  Detec>ontodecision:bestcase:1second;worstcase:10minutes–  Ac>on:30to60seconds–  Capacity:24to96percomputer;veryscalable

•  Pilotresults:machineprocessing98%+faster


Quan>fyingMachineProcessing(TreatedGroup)

•  JHUAPLPilotResults–  Breadth:1anomalytype(whitelis<ng)–  Speed:98%+fasterthanmanualprocessing–  Capacity:24to96<mesthatofmanualprocessing

•  SDMReflects–  Breadth:mul<pleanomalytypesrepresentedviaNORMAL(.05,.005)–  Speed:80%fasterthanmanualprocessing–  Capacity:24<mesthatofmanualprocessing–  1machineperworkflowphase

•  Ra<onale–  Conserva<vees<matesbasedonempiricalevidencetodate

•  Un<laddi<onalempiricalevidencejus<fiesgreateres<mates


Quan>fyingAnomalyArrivals

•  JHUAPLPilotResults–  Whitelistanomalyarrival

•  Upto1,000,000,000perday…yes,1billion

•  SDMReflects–  NORMAL(2600,866)anomaliesarrivingperminute

•  Between2.9thousandand7.5millionperday•  Runmodelperminutefor525,600minutes(1year)

•  Ra<onale–  Simulatethefilteringofanomalies–  Simulateerra<canomalyarrivalperminute–  Actualanomalyarrivalisvastlydifferentamongenvironments

•  E.g.,targetofinteresttoastatesponsoredadversaryv.incidentalaPack

–  Showinternalconsistencyofmodel–  EasySDMadjustmenttoreflectactualenvironment


DataCollec>onforHypothesis

•  TwoSDMvaria<ons,ninerandomseedvaria<ons;18runs:1.   WithoutCDPs

i.  Reflecttoday’senvironmentofnoCDPs

2.   WithCDPs:i.  CDPinfluencetointroducesecurityautoma<onandeffectsecurityautoma<onover<me


HypothesisTestforKKCharacterize

Conclusion: CDPs do significantly improve the quantity of known-known (KK) anomalies processed within the characterize workflow phase.

WithCDPs WithoutCDPs

336,038 169,214

358,791 161,785346,572 184,316344,126 167,211364,762 176,738312,140 151,389321,606 178,181355,741 173,074354,992 151,473

t-Test:PairedTwoSampleforMeans

WithCDPs WithoutCDPsMean 343863.1 168153.382Variance 311866647.5 132689659.5Observations 9 9PearsonCorrelation 0.184966816HypothesizedMeanDifference 42038df 8tStat 20.86745531P(T<=t)one-tail 1.4587E-08tCriticalone-tail 1.859548038P(T<=t)two-tail 2.91739E-08tCriticaltwo-tail 2.306004135

Hypothesis:Ho: Quantity µ NewProcess <= (Quantity µ OldProcess + Hypothesized Mean Difference)Ha: Quantity µ NewProcess > (Quantity µ OldProcess + Hypothesized Mean Difference)One-Tail Test:If P(T<=t) one-tail value is smaller than the selected alpha value, THEN we reject Ho and conclude Ha.The P-value 1.4587E-08 is less than the selected alpha value 0.05 , therefore we reject Ho.The data does not support Ho. The data does support Ha.IF the t Stat is larger than the t Critical one-tail value, THEN we reject Ho and conclude Ha.The t Stat of 20.86745531 is greater than the t Critical one-tail value 1.859548 , therefore we reject Ho.The data does not support Ho. The data does support Ha.


Conclusions

• SDMdemonstratesanomalyprocessingperminutefor1year― 525,600itera<onsofanomalyarrivalsandanomalyprocessing

• Datashowssta<s<callysignificantCDPimpact― Onmostphases,butnotallphases

• TheSDMprovidesauniqueframework― Examinerole,fit,func<on,andimpactofanycybersecuritysolu<on― Applicabletootherdomains(e.g.,hospitalpa<entprocessing)


CommiLee

• CommiPeeChair:― Dr.MarkBlackburn,SchoolofSystems&Enterprises

• CommiPeeMembers:― Dr.RobertClou<er–SchoolofSystems&Enterprises― Mr.RickDove,SchoolofSystems&Enterprises― Dr.GeorgePortokalidis,DepartmentofComputerScience― Dr.MukeshRohatgi,TheMITRECorpora<on


Ques>ons?

SDSF2016 November16,2016

BackupSlides


UniquenessofResearch

• StrategicContextforCybersecurityOpera>ons

• Tac>calContextforCybersecurityOpera>ons― FrameworkforCybersecurityOpera>onsWorkflow

• CybersecurityDecisionPaLernsStructureandContent

• CybersecurityDecisionPaLernLanguageStructure

• Quan>fica>onofTac>calContext― Quan>fyingcybersecurityopera>onalperformance

• CybersecurityOpera>onsSystemDynamicsModel― Basedonthetac>calcontext― Isolateandanalyzecons>tuentparts― Isolateandanalyzeprospec>vesolu>oneffects(e.g.,CDPs)


Quan>fiedTac>calContext(Time)

Monitor

Detect

Notify

Triage

∆td

∆tn

∆tt

∆tooda1

∆te

∆td = td – t0

∆tn = tn – td

∆tt = tt – tn

∆te = te – tac

∆ti = ti – tooda4

∆tr = tr – tooda5

∆tRCA = tRCA – tooda6

∆tf = tf – tooda8

∆td is duration of detection process

td is initiation of detection process

t0 is initiation of indicator

∆tn is duration of notify process

tn is initiation of notify process

∆tt is duration of triage process

tt is initiation of triage process

∆te is duration of escalate processte is initiation of escalate process

∆ti is duration of isolate process

ti is initiation of isolate process

∆tr is duration of restore process

tr is initiation of restore process

∆tRCA is duration of RCA process

tRCA is initiation of root cause analysis (RCA)

∆tf is duration of feedback process

tf is initiation of feedback process

t0 tfinal

OODA4

Escalate

Isolate

Restore

RCA

Feedback

∆ti

∆tr

∆tRCA

∆tv

∆tf

Recover

OODA1

OODA5

OODA7

(∆tob+ ∆tor + ∆tac)4+ ∆tde



Note: t0 may not be known until after the fact

OODA4 ∆tooda4 = tooda4 – te∆tooda4 is duration of OODA process

tooda4 is initiation of OODA process

∆tooda5 = tooda5 – ti∆tooda5 is duration of OODA process


∆tv = tv – tooda7∆tv is duration of recover process

tv is initiation of recover process

OODA5

OODA7

Characterize

OODA3

OODA2

OODA2

(∆tob+ ∆tor + ∆tac)3+ ∆tde OODA3

∆tc = tc – tde∆tn is duration of characterize process

tn is initiation of characterize process

∆tooda2 = tooda2 – te∆tooda2 is duration of OODA process


∆tooda3 = tooda3 – te∆tooda3 is duration of OODA process


OODA1

∆tooda1 = tooda1 – td∆tooda1 is duration of OODA process


∆tooda2

OODA6


OODA8

OODA9

∆tooda6 = tooda6 – tr∆tooda5 is duration of OODA process


∆tooda8 = tooda8 – tv∆tooda8 is duration of OODA process


∆tooda7 = tooda7 – tRCA∆tooda7 is duration of OODA process



∆tooda9 = tooda9 – tv∆tooda9 is duration of OODA process


trespond = å (∆td,∆tn, ∆te,trespond = tfinal – t0 ∆tRCA, ∆tf)∆tt, ∆ti,∆tr, ∆tv,or + å (∆toooda1...∆toooda9)


Indicator (t0)

∆tc




Quan>fiedTac>calContext(Quan>ty)

Monitor

Detect

Notify

Triage

Qd is quantity of anomalies detected

t0 tfinal

Escalate

Isolate

Restore

RCA

Feedback

Recover

Characterize

Indicator

å (Qd)

Qc is quantity of anomalies characterizedå (Qc)

Qn is quantity of anomalies notifiedå (Qn)

Qt is quantity of anomalies triagedå (Qt)

Qe is quantity of anomalies escalatedå (Qe)

Qi is quantity of anomalies isolatedå (Qi)

Qc is quantity of anomalies restoredå (Qr)

QRCA is quantity of anomalies RCA’då (QRCA)

Qv is quantity of anomalies recoveredå (Qv)

Qf is quantity of anomalies fedbackå (Qf)

Qprocessed = S (Qd, Qn, Qe, QRCA, Qf)Qt, Qi,Qr, Qv,Qc,


Quan>fiedTac>calContext(Accuracy)

Monitor

Detect

Notify

Triage

ad

an

at

aooda1

ae

ad is accuracy of detection process

t0 is initiation of indicator

tn is accuracy of notify process

tt is accuracy of triage process

te is accuracy of escalate process

ti is accuracy of isolate process

tr is accuracy of restore process

tRCA is accuracy of root cause analysis (RCA)

tf is accuracy of feedback process

t0 tfinal

OODA4

Escalate

Isolate

Restore

RCA

Feedback

ai

ar

aRCA

av

af

Recover

OODA1

OODA5

OODA7

OODA4 tooda4 is accuracy of OODA process

tooda5 is accuracy of OODA process

tv is accuracy of recover process

OODA5

OODA7

Characterize

OODA3

OODA2

OODA2

(aob+ aor + aac)3+ ade OODA3

tn is accuracy of characterize process



OODA1


aooda2

OODA6

OODA6

OODA8

OODA9




OODA8


Indicator (t0)

arespond = å (∆ad,∆an, ∆ae, ∆aRCA, ∆af)∆at, ∆ai,∆ar, ∆av, + å (∆aoooda1...∆aoooda9)

ac

(aob+ aor + aac)4+ ade





(aob+ aor + aac)9+ ade OODA9

tf is completion time of anomaly process


CDPLStructure1of2

Phase Problem Helps Answer Operational Questions (Knowledge Types)

Monitor Via situational awareness, I heard to expect something…

•  When should I expect it? (conditional) •  How do I keep it out? (procedural) •  How do I find it? (procedural)

Detect I see something… •  What is it? (declarative) •  What does it do? (declarative) •  How does it do it? (declarative) •  Where does it come from? (declarative, relational)

Notify I need to raise awareness… •  Who needs to know? (declarative) •  What do they need to know? (declarative) •  How do I notify them? (procedural)

Triage What are the priorities… •  How do I determine the priorities? (procedural) •  What incident is most critical to address first? (relational) •  How do I determine the effects to the tactical mission?

(relational, procedural) •  How do I determine the effects to the strategic mission?

(relational, procedural)

Escalate I need to engage the appropriate expertise…

•  Have we seen this before and know what to do about it (i.e., known knowns)? (declarative)

•  Have we seen this before and still not characterized it (i.e., known unknown)? (declarative)

•  Have we not seen this before (i.e., unknown unknown)? (declarative)

•  Posteriori, did we see it before but failed to characterize it correctly (unknown known)? (declarative)


CDPLStructure2of2

Phase Problem Helps Answer Operational Questions (Knowledge Types) Isolate I need to stop it from

proliferating… I need to stop its effects from spreading…

•  How do I contain it? (procedural) •  How do I contain its effects? (procedural)

Restore I need to continue operations… I need to continue to fulfill [tactical | strategic] mission…

•  What is the tactical implication to mission? (declarative) •  What is the strategic implication to mission? (declarative) •  How do I continue the mission? How do I fight through the

attack? (procedural)

Root cause analysis (RCA)

I need to find the root problem… I need to find the root cause… I need to define the root cause…

•  What is the root cause? (relational) •  How do I get rid of it? (procedural) •  How do I reduce the probability of recurrence? (procedural) •  How do I stop it from happening again? (procedural)

Recover I need to resume normal operations…

•  How do I get rid of it? (procedural) •  How do I modify the operating environment to accommodate

knowledge of what I found and what to do about it? (procedural)

Organizational Feedback

I need to disseminate incident details and lessons learned to others…

•  How can I capture and encode incident details, the problem, and the solution? (procedural)

•  How do I provide details to the organization for preventive or preemptive activity to minimize recurrence? (procedural)


HypothesisTestforKKNo>fy

Conclusion: CDPs do significantly improve the quantity of known-known anomalies processed within the notify workflow phase.


336,051 136,593

358,804 151,294346,574 136,141344,130 141,360364,769 135,871312,151 127,774321,616 137,924355,734 159,259354,973 127,517





HypothesisTestforKKTriage

Conclusion: CDPs do significantly improve the quantity of known-known anomalies processed within the triage workflow phase.


336,045 102,985

358,798 110,813346,557 88,845344,111 124,995364,753 109,881312,141 75,788321,591 102,967355,717 98,576354,942 100,738


WithCDPs WithoutCDPsMean 343850.4432 101732Variance 311684916.3 192905930.3Observations 9 9PearsonCorrelation 0.516998385HypothesizedMeanDifference 25433df 8tStat 41.02719754P(T<=t)one-tail 6.85804E-11tCriticalone-tail 1.859548038P(T<=t)two-tail 1.37161E-10tCriticaltwo-tail 2.306004135



HypothesisTestforKKEscalate

Conclusion: CDPs do not significantly improve the quantity of known-known anomalies processed within the escalate workflow phase.


83,965 102,985

82,368 101,74583,287 72,75687,474 110,28081,457 88,96875,394 72,11780,553 74,73169,699 75,85381,724 99,430


WithCDPs WithoutCDPsMean 80657.95234 88762.77778Variance 27104541.59 230560537.4Observations 9 9PearsonCorrelation 0.657673822HypothesizedMeanDifference 22191df 8tStat -7.331445878P(T<=t)one-tail 4.06898E-05tCriticalone-tail 1.859548038P(T<=t)two-tail 8.13797E-05tCriticaltwo-tail 2.306004135

One-Tail Test:If P(T<=t) one-tail value is smaller than the selected alpha value, THEN we reject Ho and conclude Ha.The P-value 4.06898E-05 is less than the selected alpha value 0.05 , therefore we reject Ho.The data does not support Ho. The data does support Ha.IF the t Stat is larger than the t Critical one-tail value, THEN we reject Ho and conclude Ha.The t Stat of -7.331445878 is less than the t Critical one-tail value 1.859548 , therefore we retain Ho.The data does support Ho. The data does not support Ha.


HypothesisTestforKKIsolate

Conclusion: CDPs do significantly improve the quantity of known-known anomalies processed within the isolate workflow phase.


22,592 7,804

22,253 7,92121,359 7,33722,287 7,05122,756 7,52119,792 6,46220,801 7,99519,772 7,48021,818 7,596


WithCDPs WithoutCDPsMean 21492.23557 7463Variance 1308605.136 227254Observations 9 9PearsonCorrelation 0.420501057HypothesizedMeanDifference 1866df 8tStat 35.15727451P(T<=t)one-tail 2.34412E-10tCriticalone-tail 1.859548038P(T<=t)two-tail 4.68823E-10tCriticaltwo-tail 2.306004135



HypothesisTestforKKRestore

Conclusion: CDPs do significantly improve the quantity of known-known anomalies processed within the restore workflow phase.


22,577 4,648

22,242 6,33421,348 5,04622,275 6,20622,746 5,23019,780 3,78720,790 6,44819,760 6,57821,803 6,614





HypothesisTestforKKRCA

Conclusion:

CDPs do not significantly improve the quantity of known-known anomalies processed within the root cause analysis workflow phase because RCA is a manual only phase.


HypothesisTestforKKRecover

Conclusion: CDPs do not significantly improve the quantity of known-known anomalies processed within the recover workflow phase.


194 150

186 150187 150191 150193 150200 149178 150185 150189 150


WithCDPs WithoutCDPsMean 189.4184179 149.8888889Variance 39.51912789 0.111111111Observations 9 9PearsonCorrelation -0.627380712HypothesizedMeanDifference 37df 8tStat 1.167342853P(T<=t)one-tail 0.138342531tCriticalone-tail 1.859548038P(T<=t)two-tail 0.276685062tCriticaltwo-tail 2.306004135

Hypothesis:Ho: Quantity µ NewProcess <= (Quantity µ OldProcess + Hypothesized Mean Difference)Ha: Quantity µ NewProcess > (Quantity µ OldProcess + Hypothesized Mean Difference)One-Tail Test:If P(T<=t) one-tail value is smaller than the selected alpha value, THEN we reject Ho and conclude Ha.The P-value 0.138342531 is greater than the selected alpha value 0.05 , therefore we retain Ho.The data does support Ho. The data does not support Ha.IF the t Stat is larger than the t Critical one-tail value, THEN we reject Ho and conclude Ha.The t Stat of 1.167342853 is less than the t Critical one-tail value 1.859548 , therefore we retain Ho.The data does support Ho. The data does not support Ha.


HypothesisTestforKKFeedback

Conclusion:

•  Indeterminateeffect

• Feedbackdura<onextendsbeyondrun<meparameters― Valida<onofthemodelrunningasdesigned

• Feedbackphaseismanualonlyandbydesignofthehypothesistest,thereisnoexpectedimpactonthisphasefromCDPs.


HypothesisTestforAnomaliesinEnvironment

Conclusion: CDPs do not significantly improve the quantity of known-known anomalies entering the environment. Validation of model design.

Data: WithCDPs WithoutCDPs

13,108,559 13,108,559

11,139,356 11,139,35612,060,014 12,060,01411,698,123 11,698,12310,816,298 10,816,29811,070,812 11,070,81212,367,548 12,367,54811,312,129 11,312,12912,043,553 12,043,553


HypothesisTestforDetect

Conclusion: CDPs do not significantly improve the quantity of known-known anomalies detected in the environment. Validation of model design.

Data: WithCDPs WithoutCDPs

171,139 171,139

161,851 161,851184,825 184,825171,978 171,978176,950 176,950153,214 153,214178,255 178,255174,714 174,714151,531 151,531

CYBERSECURITY DECISION PATTERNS AS ADAPTIVE …...May 02, 2014 · SDSF 2016 November 16, 2016...

Documents

Transcript of CYBERSECURITY DECISION PATTERNS AS ADAPTIVE …...May 02, 2014 · SDSF 2016 November 16, 2016...