© 2014 VMware Inc. All rights reserved.

Uncovering the Hidden Truth In Log Data with vCenter Insight™ April 2014

VMware vForum Istanbul 2014 Serdar Arıcan

VMware Strategy

2

Software-Defined Data Center Virtual Datacenter:

The ideal infrastructure for private,

public, and hybrid clouds.

Hybrid Cloud Computing Extend your data center into the cloud.

End User Computing Connectivity & Control:

Desktop, App, & Mobile Virtualization

& Provisioning

To help customers realize the promise of the Software-Defined Data Center, VMware introduced the Cloud Suite

VMware is “Going Bigger on Management!”

Business momentum

• 40% bookings growth YoY

• Accelerating Sphere installed base

penetration

• ~2.5x more customers in last year!

Partner commitment

• 3,000+ selling Mgmt (200% YoY growth)

• 2,500+ engaged in Mgmt competency

• Partners invested in Mgmt have grown

bookings 200% more than non-

competency partners

Market leadership

• Ranked #1 by IDC, 451 Research and

Info-Tech

Automation

Service Catalog

Governance

Release Automation

Operations

Service Health

Capacity Optimization

Configuration Standards

IT Business

Cost Transparency

Benchmarking

Service Quality

VMware Cloud Management

4 Compute

Physical

Hardware

Private

Clouds

Public

Clouds

Hybrid Cloud

VMware &

vCloud Datacenter Partners

Virtualized Infrastructure Abstract & Pool

Compute Abstraction

= Server Virtualization

Storage

Storage Abstraction =

Software-Defined

Storage

Network

Network Abstraction

= Virtual Networking

Applications Modern SaaS Traditional

The Control Plane for the Software-Defined Data Center and the Hybrid Cloud

Most complete, innovative management portfolio purpose-built for cloud

• Scalable solutions for dynamic, hybrid cloud environments

• Better automation driven by analytics, policy and vCloud / SDDC integration

• Converged disciplines of performance, capacity and configuration management

Fast time to value

• Easy. Fast deployment and administration

• Unencumbered by huge and complex legacy management frameworks

Best for vSphere, yet open and heterogeneous

• Co-developed with vSphere team with deep insight into platform

• Multi-hypervisor/platform management. Works across hundreds of public cloud

providers

From the trusted market leader

• Virtualization pioneer and Cloud Systems Management market leader*

• The only cloud era management vendor with the vision, scale, portfolio breadth and

track record to deliver comprehensively to the enterprise

1

2

3

* Per IDC based on 2012 Cloud Systems Management market share

4

Why VMware Cloud Management is Better

VMware Cloud Management Leadership

1st in four of six Cloud

Management and Automation

categories:

• Self-service Catalog

• Unified Cloud

Management Console

• Cloud Governance

• Metering and Billing

- 451 Research

1st in Cloud Systems

Management Software

Market Share

- IDC “Champion” in CMP

Vendor Landscape

- Info-Tech

Log Insight Overview

What Could You Do If You Had Insight?

8

Solve problems

faster—

from days to hours vCenter™

Log

Insight™ Find problems you

didn’t know you had

Get actionable

insight into what

it means

Our Vision

Create an intuitive and fast log management platform for cloud operations that delivers proactive

analytics through machine learning

9

vCenter Log Insight 2.0 Overview

Intelligent Operations

• Predictive analytics/Intelligent Grouping for faster problem

resolution

• Faster analytical queries than the leading solution1

• Improved analytical visualizations

Built with vSphere in Mind

• Powerful Log Management for VMware Products

• Support for Vmware products incl. NSX, vCAC, Horizon View

• Built in 2 way alerting with vC Ops

Unified Management

• Open and extensible platform/marketplace for content packs

What’s New

Overview

Delivers the best real-time log management

for VMware environments, across physical,

virtual, and cloud environments.

• Elastic Scale – up to 6 node clusters

• Improved Query Performance – Up to 6X faster than leading

solution

• Improved Data Collection – 30% improvement on ingestion

• Predictive Analytics –Machine Learning based Intelligent

Grouping

• Better visualization – New Analytic Visualizations

• Integration with vC Ops 2-way alert visualization

• Built by VMware Experts – vSphere analytics built in

• Predictable Pricing No surprises on storage costs

New!

Log Insight Technical Overview

Cloud / Data Center

Log

Management

OS

Logs

VC

Logs

App

Logs

System

Stats

Security

Logs

API Syslog

Analyze

• Can analyze any unstructured time-series data,

configuration etc.

• Automatically identifies structures in the data

• No need for ETL or databases

Scale

• Central, scale-out store (no-SQL) for all collected logs

• Configurable retention and archiving

• Maintenance free

Best for vSphere

• Queries, alerts, fields, charts

in the vSphere Content Pack

Log Insight UI - Interactive Log Analytics

Interactive Visualization of

Query Results, Plus Easy

Drop-Down Menu Options

Log Insight Features

• Proactive Analytics

– Significantly reduces manual parsing effort

– Automated data summarization

• Logs are clustered by event type, so rare messages can be easily spotted

– Intelligent schema detection adds structure to unstructured data

– Smart fields to aid in extraction

• Interactive Analytics

– Google-like search query capability

– Filter events on any criteria, on any field

– Flexible event views

– On-the-fly visualizations

13

vCenter Operations and Log Insight Leverage all your IT data for comprehensive visibility in one place

• Intelligent operations through predictive analytics across all machine data

• Policy-based automation enables proactive management and automated remediation

• Unified management for comprehensive visibility in one place, from vSphere to Hyper-V, AWS and physical infrastructure

Structured Data

Metrics Alerts Events

VMware vCenter

Operations

Capacity, Performance and

Configuration Management Events

Launch in Context

Unstructured Data

Logs Messages

VMware vCenter Log

Insight

Log analytics, aggregation,

and search

Public

Cloud

Scale-out Architecture With Ingestion High Availability

Load Balancer (UDP & TCP)

Log messages (Syslog:514 and HTTP:9000)

Log Insight Worker Nodes

Master Node

Web UI/Query (HTTP:80)

Syslog (TCP/UDP:514)

CFAPI (HTTP:9000)

Proactive Analytics

Log Insight Proactively Discovers Structure

Log Insight proactively learns:

from:

Through Machine Learning!

Then you can query it like a database!

Dashboard Enhancements Greatly Improve Effectiveness as Debugging Tools

• Dashboard Filters

– Can now apply a filter to all charts on a dashboard page

– Speeds up investigation of individual entities (hosts) or groups (clusters)

• Dashboard Linking

– Hyperlink from a value in one chart to a dashboard constraint in another

– Log Insight automatically discovers appropriate links

• Both Features Will Significantly Increase Content Pack Capabilities

– Richer workflows

Windows Agent

Collection Framework & API

• REST-based HTTP API

– For sending logs to Log Insight

– For clients to request configuration data

– Has all the benefits of an HTTP API (compression, proxies, easy to use, etc)

• Windows Agent

– VERY SMALL : 3.5MB RAM and 0.4% of 1 CPU core @ 100 events/s

– Deployable via standard Windows management (.msi)

– Monitors Windows Events

– Monitors text file logs (like “tail –f” on Unix/Linux)

Where to Go Next

• Communities

– https://communities.vmware.com/community/vmtn/vcenter/vcenter-log-insight

– Experts: http://loginsight.vmware.com

• Download Link:

– http://www.vmware.com/products/vcenter-log-insight

Questions?

Log Insight is the Right Solution For:

• Gathering, Analyzing, & Searching large amounts of unstructured data

• Real-time troubleshooting that requires quick answers

• VMware environments

– VMware experts have curated troubleshooting dashboards

– Native support for vCOps & vSphere

• Taming a large, complex infrastructure

Log Insight Features

• Deploys as a Virtual Appliance

– Scale-out architecture with high availability

– One deployment of 6 nodes can handle approx 45K events/second

• Microsoft Active Directory support

• Accepts logs over Syslog or RESTful API

– Any syslog agent can forward logs to Log Insight

– Lightweight Windows agent for forwarding logs is available

24

Primary Use Cases

• Troubleshooting and Root Cause Analysis

– Follow the trail from vCOps to logs to get to root cause to an observed problem

– I observed a problem (e.g. slowness), try to troubleshoot the problem and identify the

part of the stack that is responsible (e.g. network delay vs storage)

– Identify the needle in the haystack in real time when troubleshooting a problem

• Monitoring

– Monitor metrics and events (performance & change) that are visible only in logs

– Escalate alerts to on-call staff or via vCOps

– Identify problems proactively, ensure SLAs and comply to IT policies

• Unstructured Data Warehouse

– Collect all the data in one place without the need for custom parsing, transformation

of data

– Get full visibility across all your IT environment from a single place

25

Log Insight Features

• Archiving for long-term data storage

• Super-Powered Dashboards

– Dashboards called Content Packs are curated by experts for top applications & devices

– Add custom dashboards & share them with colleagues

– Beautiful visualizations

– Easily add new data filters on the fly

– Interact between dashboard widgets

• Native support for VMware products

– Simple to configure ESXi hosts

– vCenter Operations Manager interactive capabilities

26

Log Insight + vCenter Operations Manager

27

Automated correlation of performance and log data (Requires vCenter Operations Advanced or Enterprise)

Log Insight + vCenter Operations Manager

28

Launch Log Insight from vCenter Operations in-context

Escalate alerts & events directly to vCenter Operations

Much more on its way!