Data-Driven Threat Intelligence: Metrics on Indicator … · 2018-05-11 · Data-Driven Threat...

Data-DrivenThreatIntelligence:MetricsonIndicatorDisseminationandSharing

(#ddti)

AlexPintoChiefDataScientist

MLSec Project@alexcpsec

@MLSecProject

AlexandreSieiraCTONiddel

@AlexandreSieira@NiddelCorp

• Previouslyon#ddti• ChallengesatTISharing• MeasuringTISharing• TheFutureofSharing

Agenda

Thisisadata-drivenwebinar!Pleasecheckyouranecdotesatthedoor

Previouslyon#ddti• UsefulMethodsandMeasurementsforHandlingIndicators• AnalysisofThreatIntelligenceFeeds• Indirectly,amethodologyforanalyzingTIProviders

• Combine(https://github.com/mlsecproject/combine)• GathersTIdata(ip/host)fromInternetandlocalfiles

• TIQ-Test(https://github.com/mlsecproject/tiq-test)• RunsstatisticalsummariesandtestsonTIfeeds

TIQ-TEST- TonsofThreat-yTests

• NOVELTY – Howoftendothefeedsupdatethemselves?• AGING – Howlongdoesanindicatorsitonafeed?• POPULATION – Howdoesthispopulationdistributioncomparetomydata?

• OVERLAP– Howdotheindicatorscomparetotheonesyougot?

• UNIQUENESS – Howmanyindicatorsarefoundonlyononefeed?

Puttingthisthreatinteldatatowork

OverlapTestMoredataisfine,butmakesure

itisdifferent

OverlapTest- Outbound

UniquenessTestCanwetellifweareclosetofinding*all*thethreats?

Ihatequotingmyself,but…

KeyTakeaway#1

MORE!=BETTERThreatIntelligenceIndicatorFeeds

ThreatIntelligenceProgram

ConstructiveFeedbackfromtheInternet:

“TISharingisTOTALLYgoingtosolvethis”

Right,folks?Right?

TISharingSolutionPlan:

1. ThebestThreatIntelligenceistheonethatyouanalyzefromyourownincidents(homegrown/organicintelligence)

2. Thereisstrengthinnumbers– verticalherdimmunity!

3. ????????

4. PROFIT!!(oratleastSECURITY!!)

Oratleastaroughstrawman

IfCONSUMINGisforthe1%,whatisthepercentageoforganizationsabletoPRODUCE?

Issue1- BYOTI

Issue2- HerdImmunity

Source:www.vaccines.gov

• Wemaybeabletodetectmore”virusstrains”togetherbutweare*terrible*atinoculation.

• Thethingswedetectthemostmutatetoofast(PyramidofPain)

• Whodidn’tgetimmunized,stillgetssick(FOMO-TI)

TheCognitiveDissonancesofTISharing

Everybody shouldshare! TheCIRCLEOFTRUST

Doyoutrustthegroupenoughtoconsume?

TheTwoSidesoftheTrustCoin

Doyoutrustthegroupenoughtoshare?

Howaboutmeasuringit?WewouldliketothankthekindcontributionofdatafromthefinefolksatFacebookThreatExchange andThreatConnect

…andalsothesharingcommunitiesthatchosetoremainanonymous.Youknowwhoyouare,andwe❤ youtoo.

OVERLAPSLIDE

UNIQUENESSSLIDE

Lookslikewewouldgetsimilarqualityona”good”ThreatIntelligenceSharingPlatformaswewouldon

a”paidfeed"

ActivityMetricIsthereanyactualsharinggoing

Updatefrequencychart

High10saverage Low100saverage

Large– 10.000smembers Small– High10smembers

LargeCommunityisroughly36x

biggerthanSmallCommunity

DiversityMetricCheckyoursharingprivilege

Roughly10%oftheorganizationsshare

dataintothecommunity

Someorganizationsareclearlyinabetterpositionoperationallyandlegallytoshare.Andthatis

expectedduetoourpremises.

FeedbackMetricButisthedataanygood?

FeedbackMetric• Almostnosupportonautomation-drivenplatforms• Someallowyoutoleave”comments”or”newdescriptors”fortheIOCs– evenbycountingthoseverylow%inrelationtonewshareddata

• Analyst-drivenenvironmentsallowforcollaborationone-mailsandforumpoststodescribeandrefinestrategiesandbestpractices.

• Howcanwemakethiscollaborationworkonautomation-drivenplatforms?

TrustMetricArewehelpingallthecommunity

orjustafeworgsatatime?

TrustMetric• Theroughestimateseemstobethatmorethan50%of”sharing”(IOCs,messages,etc)happensin”privategroups”insidetheinfrastructureofthesharingplatform

• Allcommunitieshavethem:• PartoftheDNAoftheIC/clearedcommunity• Offsetsthetrustequation,butdefeatsthe”herdimmunity”argument• UsuallyMANDATORYoncollaborationwithLEA

• Butthenthe”good”dataisnothelping”thecommunity”isthereanywaywecanreconcile?

TheFutureofSharingAttheveryleastmyhumble

opinionJ

#squadgoalsIncreasetheTRUST

amongpeers

ReducetheTECHNICALBARRIERforsharinguseful

information

TRUST:Anonymity+GoodCuration

Somesharingcommunitiesacceptanonymoussubmissionsthattheythencurateanddisseminate

toallorganizations

Feedback

TelemetryLESSMATURE

MOREMATURE

❤ andapologiesto@DavidJBianco

TECHNICALBARRIER:”PyramidofSharing”

Takeaways

• IntelligenceSharingisaveryanalyst-centricactivitythatwehavebeentaskedwithscalingout

• Datacanbeasgoodasapaidfeed,butyouhavetobeintherightcirclesoftrust

• Doesnotsolveanalystshortageandmakingtheindicators/strategiesoperationalintoyourenvironment

Thanks!

• Q&A?• Feedback!

”Themeasureofintelligenceistheabilitytochange."- AlbertEinstein

AlexPinto@alexcpsec

@MLSecProject

Alexandre Sieira@AlexandreSieira@NiddelCorp

Data-Driven Threat Intelligence: Metrics on Indicator … · 2018-05-11 · Data-Driven Threat...

Documents

Transcript of Data-Driven Threat Intelligence: Metrics on Indicator … · 2018-05-11 · Data-Driven Threat...

Metrics, Metrics, Metrics (and tips) 2014 JobG8

WEAPON SYSTEM PERFORMANCE INDICATORS - OSD SUSTAINMENT KPPs · WEAPON SYSTEM PERFORMANCE INDICATORS - OSD SUSTAINMENT KPPs 29 Oct 2008 Roy E. Rice, ... • Wartime metrics, threat

Standardized Threat Indicators Tenable Formatted Indicator Export Adversary Analysis (Pivoting)

chemistry.ucsd.educhemistry.ucsd.edu/.../aerotekopenings9-5-17.docx · Web view$17-30/hr DOE. From the hiring ... summarize and report key performance indicator data and metrics

Introduction to Altmetric for Institutions Training Nov 15 Schulich.pdfalt + metrics Complementary to traditional citation metrics Score is an indicator and the underlying, qualitative

BSC Key Performance Indicator Metrics Guide · PDF fileTotal Parts Purchase ... Second Level Key Performance Indicator Metrics ... BSC Key Performance Indicator Metrics Guide:

1 OS II: Dependability & Trust Threat Modeling & Security Metrics Dependable Embedded Systems & SW Group Prof. Neeraj.

Yair Amir 1 2 Nov 07 The Insider Threat in Scalable Distributed Systems: Algorithms, Metrics, Gaps Distributed Systems and Networks lab Johns Hopkins University.

LOGRHYTHM LABS 2017 THREAT INTELLIGENCE REPORT · 46 Sample: d0fb.eml 47 Sample: cleaner.exe 47 Sample: example_powershell_payloads.txt 49 Appendix B: Consolidated Indicator List

Working Smarter: Job Performance Metrics For the … Smarter - Job...Working Smarter: Job Performance Metrics For the Smart Grid SANS SCADA Conference Presentation ... - Advanced Threat

Standardized Threat Indicators Indicator Export Adversary Analysis (Pivoting)

Citizenship Likely an Unreliable Indicator of Terrorist ... › irp › eprint › dhs-7countries.pdfCilizclIship Likely an Unreliable ו,ndicator ofl'errorj!lt Threat r() t וfe United

Webroot Perspective · 4 Webroot Perspective The statistics in this report are based on threat intelligence metrics automatically captured, analyzed, and contextualized through the

Irish Maternity Indicator System - HSE.ie · Introduction This second annual report of the Irish Maternity Indicator System (IMIS) provides national results for 30 metrics from all

Performance Metrics Central User Guide - · PDF fileOperational Indicator Metrics List ... Your Performance Metrics Central User Guide provides detailed information about how to use

Software Metrics - ida.liu.seTDDC88/theory/12Metrics-before.pdf · Software metrics • Usage-based metrics • Verification & Validation metrics • Volume metrics • Structural

Spc Indicator Metrics Framework

Module 2 Developing Metrics and Indicators · Developing Metrics and Indicators . End of Module Objective ... Key performance indicator (KPI) is a business metric used to evaluate

MHA Consulting BCM Metrics – Resiliency Through …...Critical Success Factor (CSF) – Element critical to the service and maps to department objectives. Key Performance Indicator

1 SOFTWARE PROCESS AND PROJECT METRICS. 2 Process, project and measurement Process Metrics or indicator:- Are collected across all projects and over long.