Machine Learning + Analytics

Copyright©2016SplunkInc.

MachineLearning+AnalyticsinSplunkBeauMorganSeniorSalesEngineer– SecuritySME

1of1billion

2

DisclaimerDuringthecourseofthispresentation,wemaymakeforwardlookingstatementsregardingfuture

eventsortheexpectedperformanceofthecompany.Wecautionyouthatsuchstatementsreflectourcurrentexpectationsandestimatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthose

containedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentationarebeingmadeasofthetimeanddateofitslivepresentation.Ifreviewedafteritslivepresentation,thispresentationmaynotcontaincurrentoraccurateinformation.Wedonotassumeanyobligationtoupdateanyforwardlookingstatementswemaymake.Inaddition,anyinformationaboutourroadmapoutlinesourgeneralproductdirectionandissubjecttochangeatanytimewithoutnotice.Itisforinformationalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.Splunkundertakesnoobligationeithertodevelopthefeaturesor

functionalitydescribedortoincludeanysuchfeatureorfunctionalityinafuturerelease.

3

MachineLearningandYou

4

Agenda

• MachineLearning101&Splunk

• DemooftheMachineLearningToolkit&Showcase

• HowtobesuccessfulwithML+Splunk

5

WhydoweneedMachineLearning?

- ImproveDecisionMaking,ImproveFutureActions

- ForecastorPredictKPIs,AlertonDeviation

- Uncoverhiddentrendsorrelationships

AllofthisrequiresDiverseDatafromacrossManySilos.LotsofUnstructured,RealTimeData.

6

RunTheBusinessinReal-time

DataFromthePast Real-timeData StatisticalForecastT– afewdays T+afewdays

SecurityOperationsCenter

ITOperationsCenter

BusinessOperationsCenter

Predictive(Models)

Descriptive(BITools,DataLakes) Greyspace

7

What is “Learning”?[Prediction]• When we see thick clouds and an overcast sky, we

predict that it’s likely going to rain

[Estimation/ Regression]• Estimate how much an apartment costs based on its

location, condition and prices of properties in that neighborhood

[Classification/ Clustering]• Determine the gender of a person based on her/his

features, hair style and the way s/he dresses

[Anomaly Detection] • Identify the odd one out

[Reinforcement Learning]• If I made a mistake this time, can I do better next time?

Allofushavehadsomeexperienceinlearning.But…what’sbehindourexperience?Howdowetranslatethatknowledgetocode?

8

MachineLearning101:Whatisit?MachineLearningisaprocessforgeneralizingfromexamples

Examples=exampleor“training”dataGeneralizing=building“statisticalmodels”tocapturecorrelationsProcess=neverquitedone,wekeepvalidating&refittingmodelsforincreasingaccuracy

SimpleMachineLearningworkflow:ExploredataFITmodelsbasedondataAPPLYmodelsinproductionKeepvalidatingmodels

“Allmodelsarewrong,butsomeareuseful.”

9

ML101:ExistingApplicationsRecall:EXPLORE>FIT>APPLY>VALIDATE>REPEAT

• Facedetection:findfacesinimages

• Spamfiltering:identifySPAMmessages

• ShoppingRecommendations:predictwhatcustomerswouldliketobuy

• Frauddetection:identifycreditcardtransactionswhichmaybefraudulentinnature

• Weatherforecast:predictwhetherornotitwillraintomorrow;estimatedailymax/min

10

ThreeTypesofMachineLearning1.Supervised Learning:generalizingfromlabeled data

OR?

Gatherdata:• Dimensions• StemLength• Color• Etc.

11

ThreeTypesofMachineLearning2.Unsupervised Learning:generalizingfromunlabeled data

Willmyhomesell?Gatherdata:• Squarefeet• Levels• Parksnearby• Schools• Zipcode• Etc.

12

ThreeTypesofMachineLearning3.ReinforcementLearning:generalizingfromrewards intime

RecommendationEngines

13

OverviewofMLatSplunk

13

CorePlatformSearch PackagedPremiumSolutions

CustomML

PlatformforOperationalIntelligence

14

SearchIncludesMachineLearningCorePlatformSearchisapowerfulandhighlyflexibleinterfacebuiltwithML

anomalydetection

15

SplunkITServiceIntelligence

GetData Defineservices,entitiesandKPIs

Monitorandtroubleshoot

Analyzeanddetect

Data-Defined,Data-DrivenServiceInsights

PackagedML:AdaptiveThresholdsandAnomalyDetection

OneofseveralPremiumSolutions

16

SplunkMachineLearningToolkit

Assistants: Guidemodelbuilding,testing,&deploymentforcommonobjectivesShowcases: Interactiveexamplesforover25typicalIT,security,business,IoTusecases

Algorithms: 25+standardalgorithmsavailableprepackagedwiththetoolkitSPLMLCommands:Newcommandstofit,testandoperationalizemodelsPythonforScientificComputingLibrary:300+opensourcealgorithmsavailableforuse

Buildcustomanalyticsforanyusecase

ExtendsSplunkplatformfunctionsandprovidesaguidedmodelingenvironment

17

What’sNewin2.0?

17

• Newnameandabbreviation• Noeventlimits(removalof50Klimitonfittingmodels)

• Configurableresourcecapsviamlspl.conf

• Searchheadclusteringsupport• Distributed/streamingapply• Scheduledfit• Newalgorithms(seeSlide7)

– Featureengineeringandselection– Stochasticgradientdescent(e.g.)– ARIMA

• Multi-algorithmsupportacrossAssistants

• Scatterplotmatrixviz• Alerting• Tooltips• In-apptours• ClusterNumericEventsassistant

18

SplunkMLAlgorithms(v2.0,.Conf2016)• ARIMA• SGDClassifier• SGDRegressor• DecisionTreeClassifier• DecisionTreeRegressor• AdaBoostRegressor• BernoulliNB• Birch• DBSCAN• ElasticNet• FieldSelector• GaussianNB• KMeans

• KernelPCA• KernelRidge• Lasso• LinearRegression• LogisticRegression• OneClassSVM• PCA• RandomForestClassifier• RandomForestRegressor• Ridge• SVM• SpectralClustering• TFIDF• StandardScaler


ToTheDemo!


SuccessWithML:TheProcess

DomainExpertise(IT,Security,…)

DataScienceExpertise

SplunkExpertise

CustomMachineLearning– SuccessFormula

Identifyusecases

Drivedecisions

Setbusiness/opspriorities

SPL

Dataprep

Statistics/mathbackground

Algorithmselection

Modelbuilding

SplunkMLToolkitfacilitatesandsimplifiesviaexamples&guidance

Operationalsuccess

22

Summary:TheMLProcess

1. Getallrelevantdatatoproblem

2. Exploredata,andfitpredictivemodelsonpast/real-timedata

3. Apply&validatemodelsuntilpredictionsareaccurate

4. ForecastKPIs&notableeventsassociatedtousecase

5. SurfaceincidentstoXOps,whoINVESTIGATES&ACTS

Problem:<Stuffintheworld>causesbigtime&moneyexpense.ValueHypothesisSolution:BuildMLmodeltoforecast<possibleincidents>,actpre-emptively&learn

Operatio

nalize

23

MachineLearningProcesswithSplunk

23

CollectData

Explore/Visualize

Model

Evaluate

Clean/Transform

Publish/Deploy

props.conf,transforms.conf,DatamodelsAdd-onsfromSplunkbase,etc.

Pivot,TableUI,SPLMLToolkit

Alerts,Dashboards,Reports

24

StepstoBuildingYourOwnMLAppPrioritize&solvethebigproblems:

DatacenterorcriticalinfrastructurefailingHard-to-find,high-riskbehaviors

UseALLdatatohelpsolveproblems:E.g.,can’tidentifyappcrasheswithoutappdataEnrichmachinedatawithtickets,appdata,DB,etc.

Findthestakeholders:Whoownstheseproblems?Whowillinvestinyoutobuildasolution?

Solutionsnotscienceprojects:Ifit’smission-critical,treatitassuch(Dev->QA->Prod)Prototype:buildsimpleMVPs,showvalue,iterate

25

Fit,Apply&ValidateModelsMLSPL – NewgrammarfordoingMLinSplunkfit – fitmodelsbasedontrainingdata– [training data] | fit LinearRegression costly_KPI from

feature1 feature2 feature3 into my_model

apply – applymodelsontestingandproductiondata– [testing/production data] | apply my_model

ValidateYourModel (TheHardPart)– Whyhard?Becausestatisticsishard!Also:modelerror≠realworldrisk.– Analyzeresiduals,mean-squareerror,goodnessoffit,cross-validate,etc.– TakeSplunk’sAnalytics&DataScienceEducationcourse

26

MLCommandsinCoreSPL

Cluster – groups events together based on how textually similar they are to each other.

Anomalies – finds events or field values that are unusual or unexpected

Predict - forecasts values for one or more sets of time-series data

Kmeans - Kmeans clustering on events.

Anomalousvalue - anomaly score for each field of each event, relative to the values of this field across other events.

Anomalydetection - identifies anomalous events by computing a probability for each event and then detecting unusually small probabilities.

• X11 - exposes seasonal trend in your time series.

Associate – Change in entropy between two fields.

Findkeywords - Given a set of numbered groups (from say cluster) calculates the common words found in each cluster.

Analyzefields - What is the ability of a set of fields to predict a single field. Univariate analysis.

And lots more

ReferenceDocs->http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/ListOfSearchCommands

27

Don’tForget:80%ofDataScienceisDataMunging

Trendline – Moving Averages of fields

Erex- Use the erex command to extract data from a field when you do not know the regular expression to use

Correlation – Co-occurence NOT correlation as per Pearson et., don’t mix this up.

Autoregress – Copies one or more previous values for a field into an event.

Contingency - Frequency distribution matrix.

Cofilter - find how many times field1 and field2 values occurred together.

ReferenceDocs->http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/ListOfSearchCommands

Stats, Eventstats, Streamstats,Timechart, Chart – stats reporting

Eval - evaluate new fields

And lots more

28

WhatNow?

28

• GettheMachineLearningToolkitfromSplunkbase• GowatchMachineLearningVideosonSplunkYoutube Channelhttp://tiny.cc/splunkmlvideos• GotoMachineLearningstalks:https://conf.splunk.com/

– AdvancedMachineLearninginSPLwiththeMachineLearningToolkitbyJacobLeverich– ExtendingSPLwithCustomSearchCommandsandtheSplunkSDKforPythonbyJacobLeverich

• SeveralCustomersandPartnerTalks– Cisco,Scianta Analytics,AsianTelco,etc.

• EarlyAdopterAndCustomerAdvisoryProgram:[email protected]• ProductManager:[email protected]• FieldExpert:[email protected]

http://tiny.cc/splunkmlapp


Thankyou!


Appendix


CustomerStories

32

MachineLearningCustomerSuccess

NetworkIncidentDetectionServiceDegradationDetection Security/FraudPrevention

PrioritizeWebsiteIssuesandPredictRootCause

PredictGamingOutagesFraudPrevention

MachineLearningConsultingServices AnalyticsAppbuiltonMLToolkit

Optimizingoperationsandbusinessresults

CellTowerIncidentDetectionOptimizeRepairOperations

Entertainment Company

15

33

MLToolkitCustomerUseCases

33

Speedingwebsiteproblemresolutionbyautomaticallyrankingactionsforsupportengineers

Reducingcustomerservicedisruptionwithearlyidentificationofdifficult-to-detectnetworkincidents

Minimizingcelltowerdegradationanddowntimewithimprovedissuedetectionsensitivity

Improvingcelltoweruptimeandreducingrepairtruckroleswithanomalydetectionandrootcauseanalysis

Predictingandavertingpotentialgamingoutageconditionswithfiner-graineddetection

EnsuringmobiledevicesecuritybydetectinganomaliesinIDauthentication

PreventingfraudbyIdentifyingmaliciousaccountsandsuspiciousactivitiesEntertainment Company

34

DetectNetworkOutliersReduceddowntime+increasedserviceavailability=bettercustomersatisfaction

34

MLUseCase Monitornoiserisefor20,000+celltowerstoincreaseserviceanddeviceavailability,reduceMTTR

Technicaloverview • Acustomizedsolutiondeployedinproductionbasedonoutlierdetection.• Leveragepreviousmonthdataandvotingalgorithms

“TheabilitytomodelcomplexsystemsandalertondeviationsiswhereITandsecurityoperationsareheaded…SplunkMachineLearninghasgivenusaheadstart...”

35

ReliablewebsiteupdatesProactivewebsitemonitoringleadstoreduceddowntime

35

“SplunkMLhelpsusrapidlyimproveend-userexperiencebyrankingissue severitywhichhelpsusdeterminerootcausesfasterthusreducingMTTRandimprovingSLA”

• Veryfrequentcodeandconfig updates(1000+daily)cancausesiteissues• Finderrorsinserverpools,thenprioritizeactionsandpredictrootcause

• CustomoutlierdetectionbuiltusingMLToolkitOutlierassistant• BuiltbySplunkArchitectwithnoDataSciencebackground

MLUseCase

Technicaloverview


Example:EnergyData

Sensordatadeliveringmillionsofdollarsinenergysavings.

RobotAnalyticstoReduceCostsintheSupplyChain

4% IncreasedThroughputperDistributionCenter

Aggregatemachinedatafromrobots

Failurepatterndetectionandreporting

Preventativemaintenancescheduling

39

EnergyData

39

ProcessData

86348;24.03.1523:59:59;140808,297;140746,031;140919,500;

24-03-201501:00:59;EPIP02-03-A;SB;PPR;PR;PRODUCTION;PR;aRTC:accountedtransaction(equip02_evnt_job_unit01);;;;;;;0,014;753,000

correlationovertime(join)

ProductionstatusEnergyconsumption

- Transparencyofequipmentonshopfloorlevel- Discoverprocessweaknesses- Conditionbasedandpredictivemaintenance- Optimizationofenergyefficiencyofequipment- Optimizationofenergypurchasingprocess(forecast/predictive)…etc ….

IncreasedefficiencySavedenergySaved$$$

Usecases

40

TypicalWorkflowforAnalyzingSensorData

40

COLLECT ENRICH ANALYZE

lookupdata

dataanalytics

feedbackloop

sensordata

middleware

41

Energyandprocessdataformaintenance

Energynotjustaoptimizationtarget– butalsoaninfluencingfactorfor

maintenancescenarios(rapidimpactfactor)

Maplowlevelprocessstatustoparticularenergyconsumptionprofilesand

learnnormalstatesandboundariesfromrawsignalA B C D

42

ConditionMonitoring&AlertingAnomalydetectionandproactivemonitoring

42

43

PredictiveMaintenance

43

• Predictanomaliesforaparticularprocessstep

Heatmap shows(recommend)timespaninwhichanerrormighthappen

PredictprocessstepsInwhichanerrormighthappen

Extrapolationofprocessstepswithintegratedpredictfunctionorotherregressionsmodels

Machine Learning + Analytics

Technology

Transcript of Machine Learning + Analytics