Copyright © 2015 Splunk Inc.

Innovate Better Through Machine Data AnalyticsHal Rottenberg, Global ITOA PractitionerSplunk

1

Or…

How to Splunk your DevOps

Or…

Hal really didn’t like the first two titles, thought way too long on a third one, then gave up and decided that what really mattered was the next slide anyway

This is what I’m selling

“DEVOPS IS A SUPERSET, NOT A SUBSET”

7

Is this DevOps?

Development Operations

DEVOPS?

8

Is this DevOps?

Development Operations

DEVOPS?NO!

9

This is DevOps!DevOps

Development Operations

10

How Shall We Get There?

Culture

Data

Analysis

Continuous Improvement

Defining DevOps

INTEGRATION

COLLABORATION

COMMUNICATION

BETWEEN DEV AND OPS

METHODS FOR IMPROVING

CAMS – as close to prescriptive as DevOps gets

CultureAutomationMeasurementSharing

BUT WHAT SHOULD

YOU MEASURE?

I’m working super hard!!

That’s my stapler.

Activity?

Yeah, but … … what are you

achieving?

I’m gonna need you to come in Sunday.

Outcomes?

Some DevOps Metrics that Might Matter

Culturee.g.• Retention• Satisfaction• Callouts

Processe.g.• Idea-to-cash• MTTR• Deliver time

Qualitye.g.• Tests passed• Tests failed• Best/worst

Systemse.g.• Throughput• Uptime• Build times

Activitye.g.• Commits• Tests run• Releases

Impacte.g.• Signups• Checkouts• Revenue

From every tool, every process, every component, on-prem or off

Machine Data Is A Critical Source Of DevOps Metrics

Industry Leading Platform for Machine DataAny Machine Data

Online Services Web

Services

ServersSecurity GPS

Location

StorageDesktops

Networks

Packaged Applications

CustomApplicationsMessaging

TelecomsOnline

Shopping Cart

Web Clickstreams

Databases

Energy Meters

Call Detail Records

Smartphones and Devices

RFID

Datacenter

Private Cloud

Public Cloud

Enterprise Scalability

Search and Investigation

Proactive Monitoring

Operational Visibility

Real-time Business Insights

Operational Intelligence

Visibility Across the Ops Environment

APISDKs UI

Server, Storage. N/W

Server Virtualization

Operating Systems

Infrastructure Applications

Mobile Applications Cloud Services

Other ToolsTicketing/Help

Desk

No rigid schemas – add in data from any other source.

Custom Applications API Services

Visibility Across the Dev Lifecycle

APISDKs UI

Other ToolsEscalation/

Collaboration

No rigid schemas – add in data from any other source.

Plan Code Build Test/QA Stage Release Config Monitor

INCREASE VELOCITY

IMPROVE QUALITY

DRIVE IMPACT

Improve the Impact of Application Delivery

What DevOps Data Can You Splunk?

Machine Data for Provisioning and Config

Machine Data from Pre-Prod/Staging

Machine Data From Testing and QA

Machine Data from Release Servers

Machine Data from Infrastructure Systems

Machine Data from Database Servers

Machine Data from Customer-Facing Systems

CI / Build Server

Code Review

Task Tracking

What Data Can You Splunk?

Which code has already been reviewed for this release/sprint? Who has completed the most code reviews? What code has NOT been reviewed?

Who is changing files? What kinds of files are being changed? What branches are most active? What types of activities are occurring for a branch?

Version Control

How many builds completed today/this week/this month? Which check-in kicked off this build? Which tests ran against this failed build?

Which tasks are assigned to which developers? What progress is being made to complete assigned tasks? What tasks remain for this release/sprint?

How Can You Splunk DevOps?

Industry Leading Platform for Machine DataAny Machine Data

Online Services Web

Services

ServersSecurity GPS

Location

StorageDesktops

Networks

Packaged Applications

CustomApplicationsMessaging

TelecomsOnline

Shopping Cart

Web Clickstreams

Databases

Energy Meters

Call Detail Records

Smartphones and Devices

RFID

Datacenter

Private Cloud

Public Cloud

Enterprise Scalability

Search and Investigation

Proactive Monitoring

Operational Visibility

Real-time Business Insights

Operational Intelligence

Splunk Add-On for Jira

Github Modular Input

Puppet Enterprise App for Splunk

Chef App for Splunk

Splunk App for AWS

Splunk Add-on for Google Cloud Platform

Splunk Logging Driver for Docker

curl -k https://<host>:8088/services/collector -H 'Authorization: Splunk <token>' -d '{"event":"Hello Event Collector"}'

Applications IoT Devices

Agentless, direct data onboarding via a standard API

HTTP Event Collector

Scales to Millions of Events/Second

AWS Lambda for HTTP Event Collector

Splunk App for Stream

Enables real-time insights into private,

public and hybrid cloud infrastructures

Delivers rapid deployment, easy

scale out and efficient wire data capture

Capture and analyze critical events not

found in logs or with other collection

methods.

1 2 3

Enhance Operational Intelligence With Wire Data Capture

Splunk MINT for Mobile Data

Deliver Better Performing, More

Reliable Apps

Deliver Real-Time Analytics

Achieve End-to-End Visibility

PagerDuty App for Splunk

Copyright © 2015 Splunk Inc.

Why Use Splunk for DevOps?

Machine Data To Enable Continuous Improvement

Defect Information

CapacityPlanning

Quality Standards

Enhancement Requests

Integration Requirements

Acceptance Metrics

Service Levels and KPIs

Application Development Test and Acceptance Production

BuildCodePlan Test/QA Stage Release Config Monitor

InfrastructureDependencies

Increase Delivery Velocity

DevOps Teams Iterate with Continuous Insights

Product Managers

identify new opportunities

Code Continuously delivered to market

Auditorshave visibility

Customersare happy

Improve Code Quality

Code quality scans Static security scans

White BoxDevelopers check in code

Automated Acceptance Tests

Dynamic Security Scans

Black Box

“Chaos Monkey” tests

Test Fail: Return

Test Fail: Return

X

X

Production

QA Prod Pattern

QA Pattern Library

Test Pass: Promote

Test Pass: Promote to Production

Pattern library used for test and

QA

Align With Business Impact

In Closing…

INCREASE VELOCITY

IMPROVE QUALITY

DRIVE IMPACT

Improve the Impact of Application Delivery

A Reminder

“DEVOPS IS A SUPERSET, NOT A SUBSET”- HAL ROTTENBERG, MAY 2016

Where to go for more Info● DevOps Videos, Customer Stories, Whitepapers

‣ http://splunk.com/DevOps– Developer Tutorials, Code Samples, Downloads– http://dev.splunk.com

● Splunk Apps and Plugins– https://splunkbase.splunk.com

● Blogs for Dev, Ops, and DevOps– http://blogs.splunk.com

53

54

Bonus Content

55

Splunk for Developers

Real-time dashboards show error rate in production and impact of pushing

new builds

Developers can search and visualize web logs, Java logs, eventlogs etc;

trace tx without complex instrumentation

Alerts notify developers as soon as a problem arises

57

Find and Fix Issues Faster

Gain end-to-end visibility to make informed decisions

Analytics insights without the need for additional analytics tools

Ask questions while exploring and collecting data

58

Push Better Code Using Analytics

Powerful Platform for Enterprise Developers

59

REST API

Build Splunk Apps Extend and Integrate Splunk

Simple XML

JavaScript/CSS Extensions C#JavaScriptPython

RubyJavaPHP

Data Models

Search Extensibility

Modular Inputs

SDKs

KV Store

Splunk Developer Guidance

Splunk Reference AppsComplete, working real-world Splunk solutions built together with partners (Conducive; Auth0)̶U 2 (pseudo-) production releases̶U entire code & test repos on GitHub̶U under Apache 2.0

Associated GuidanceI. Start-to-Finish Journey Documentary II. Essentials

dev.splunk.com/goto/devguide

Splunk in Real World DevOps Use Cases

Successful Businesses Use Splunk for DevOps

63

Improved DevOps Agility

63

Key Customer Benefits

• Increased success rate of deployments• Detect issues before they affect broad production• Monitoring deployment process several times per day

-Robert Gonsalves,Web Operations

“It’s like we were working without peripheral vision before and now we have it.”

64

Deliver Better Code Quality

Key Customer Benefits

-Principal Engineer,Apollo Group

“Developers are now able to look for errors and troubleshoot issues five to ten times faster by having all their event data centralized in Splunk.”

• Provide full visibility into QA sanity and load testing before production

• Exceed SLA thresholds with full visibility and benchmark key infrastructure metrics and errors

• Easier troubleshooting if tests do not contain the expected results

65

Enable Data-driven Continuous Delivery

-Alison Perkins, Senior Systems Engineer

“ Dump all the logs into Splunk, and it starts looking like one big system, instead of a bazillion teeny ones that hate each other.”

Key Customer Benefits • Quickly validate and troubleshoot code pushes to

production• Ensure that new code does not negatively impact

performance or user experience • Reduced one application’s error rate by 2 orders of

magnitude in a matter of weeks