High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

16
High-performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware Jonas Rosland [email protected] @virtualswede Solutions Architect, EMC Office of the CTO Co-sponsored by Intel®

TAGS:

description

 

Transcript of High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Page 1: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

High-performance Splunk onEMC Scale-Out Storage,Cisco USC and VMware

Jonas [email protected]@virtualswede

Solutions Architect, EMC Office of the CTO

Co-sponsored by Intel®

Page 2: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 2© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 3: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 3© 2013-2014 Cisco and/or its affiliates. All rights reserved.

What is Splunk?

Page 4: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 4© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Engine for Collecting, Indexing, Analyzing, and Visualizing: Machine Data Logs Application Queues Records (Billing, Call Detail, Events) Click Streams Performance Metrics Packet Data

What is Splunk?

Page 5: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 5© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Better understanding of your environment to drive decisions

Search and Visualization Tools analyze data(Question Focused Data Set)

Automated Reporting

Why use Splunk?

Page 6: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 6© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Splunk indexers receive data from various sources

Those indexes are then searchable

For increased performance and scalability multiple Indexers can be deployed

A Search Head searches through all servers for specific data in an index

Clients connect to a Search Head for searching

How do you leverage Splunk?

Page 7: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 7© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Why Cisco together with EMC and VMware for Splunk?

Page 8: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 8© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Indexes are divided into different pieces: Hot – Newly indexed data, open for writing, searchable Warm – Data rolled from hot, searchable Cold – Data rolled from warm, searchable Frozen – Data rolled from cold, unsearchable, usually deleted or

archived Thawed – Restored Frozen data, searchable

Splunk handles ILM well

Page 9: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 9© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Indexes are divided into different storage solutions: Hot & Warm – Put on Cisco UCS servers with EMC ScaleIO, seen as

”Hot Edge” in 3rd platform, usually up to 10 – 100 TB. Cold, Frozen & Thawed – Put on EMC Isilon, seen as ”Cold Core”, for

longer keeping of data, up to PBs or EBs, up to you!

Customers will benefit from Splunk’s ILM

Page 10: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 10© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Example deployment of Splunk with Cisco UCS, EMC ScaleIO and Isilon

Page 11: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 11© 2013-2014 Cisco and/or its affiliates. All rights reserved.

As indexes are divided into different storage solutions, so are the performance and cost tiers

Scale-out storage architectures gives more control over the collected data without sacrificing the simplicity of Splunk

SPOF in Splunk are minimized as storage is spread over multiple nodes instead of standard DAS deployments

Storage teams can be given control over Splunk storage, server/Splunk admins need not to worry anymore (also called going home at 5PM)

Customer benefits of this solution

Page 12: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 12© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Splunk is usually deployed on bare metal servers

Customers can benefit from consolidating these into virtual servers

Customers just need to make sure that they set the virtual hardware to be properly sized for their Splunk environment

With VMware, we also remove a large Splunk SPOF which is the bare metal server itself

Benefits of using VMware for Splunk

Page 13: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 13© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 14: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Cisco Confidential 14© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Whitepaper: https://community.emc.com/docs/DOC-34096

Performance metrics: http://purevirtual.eu/2014/01/28/splunk-on-vmware-and-emc-scaleio

-a-quick-index-performance-test/

Documentation available!

https://community.emc.com/docs/DOC-34096
http://purevirtual.eu/2014/01/28/splunk-on-vmware-and-emc-scaleio-a-quick-index-performance-test/
http://purevirtual.eu/2014/01/28/splunk-on-vmware-and-emc-scaleio-a-quick-index-performance-test/
Page 15: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Questions and/or comments?

Send them to:

Jonas [email protected]@virtualswede

mailto:[email protected]
Page 16: High-Performance Splunk on EMC Scale-Out Storage, Cisco USC and VMware

Thank you.

Co-sponsored by Intel®


Program Overview - Splunk...mission-critical services. None. Splunk Enterprise System Administration Splunk Enterprise Data Administration Splunk Cloud Administration Implementing

Program Overview - Splunk...mission-critical services. None. Splunk Enterprise System Administration Splunk Enterprise Data Administration Splunk Cloud Administration Implementing

Splunk user group - automating Splunk with Ansible

Splunk user group - automating Splunk with Ansible

VMware and Arista Network Virtualization Reference Design ...€¦ · Taking Advantage of Abstraction ... Splunk, and several other log management and rotation/indexing tools and

VMware and Arista Network Virtualization Reference Design ...€¦ · Taking Advantage of Abstraction ... Splunk, and several other log management and rotation/indexing tools and

Splunk .conf2011: Splunk for Fraud and Forensics at Intuit

Splunk .conf2011: Splunk for Fraud and Forensics at Intuit

Industrial Data Internet of Things splunk (REST API, SDKs) splunk . TESCO splunk 500 Y—Jt1-3— 41328 . SHANON 1/5 splunk . htt . (500MB/B) s lunk splunk . Created Date:

Industrial Data Internet of Things splunk (REST API, SDKs) splunk . TESCO splunk 500 Y—Jt1-3— 41328 . SHANON 1/5 splunk . htt . (500MB/B) s lunk splunk . Created Date:

SCADA*Coordinator,*SCADA*Infrastructure*&*Cyber*Security ...€“ Palo*Alto,*AcJve*Directory,*iLO,*VPN,*RDP,*etc.* Splunk*for*VMware* **50*GB*Daily* Splunk*IT*Service*Intelligence

SCADA*Coordinator,*SCADA*Infrastructure*&*Cyber*Security ...€“ Palo*Alto,*AcJve*Directory,*iLO,*VPN,*RDP,*etc.* Splunk*for*VMware* **50*GB*Daily* Splunk*IT*Service*Intelligence

Rivium Splunk Windows · o Splunk Enterprise Security * o uberAgent* o Splunk App for Web Analytics Common Splunk Apps & Add-ons 15 SplunkingWIndows o Splunk Add-on for Microsoft

Rivium Splunk Windows · o Splunk Enterprise Security * o uberAgent* o Splunk App for Web Analytics Common Splunk Apps & Add-ons 15 SplunkingWIndows o Splunk Add-on for Microsoft

Our experiences throughout the 3 year journey · Splunk – Hardware is now available for Splunk expansion • Splunk begins to fill monitoring gaps, acts as “glue” • Splunk

Our experiences throughout the 3 year journey · Splunk – Hardware is now available for Splunk expansion • Splunk begins to fill monitoring gaps, acts as “glue” • Splunk

stoQ’ing your Splunk - SANS · PDF filestoQ’ingyour Splunk Ryan Kovar, Splunk Marcus LaFerrera, PUNCH SANS DFIR 2016

stoQ’ing your Splunk - SANS · PDF filestoQ’ingyour Splunk Ryan Kovar, Splunk Marcus LaFerrera, PUNCH SANS DFIR 2016

Hybrid Cloud Runtime Reference Architecture - VMware vForum...vRealize Operations/WaveFront/Nagios vRealize Network Insight/Network Insight vRealize Log Insight/ Log Intelligence/Splunk

Hybrid Cloud Runtime Reference Architecture - VMware vForum...vRealize Operations/WaveFront/Nagios vRealize Network Insight/Network Insight vRealize Log Insight/ Log Intelligence/Splunk

SPLUNK® ENTERPRISE - CDWwebobjects.cdw.com/.../Splunk/Product-Brief-Splunk-Enterprise-The... · Splunk Enterprise collects data from any source, including logs, clickstreams, sensors,

SPLUNK® ENTERPRISE - CDWwebobjects.cdw.com/.../Splunk/Product-Brief-Splunk-Enterprise-The... · Splunk Enterprise collects data from any source, including logs, clickstreams, sensors,

FireEye Splunk InterFireEye + Splunk: Intermediate Guidemediate Guide

FireEye Splunk InterFireEye + Splunk: Intermediate Guidemediate Guide

Tenable and Splunk Integration · Splunk Splunkreceivesvulnerabilitydatacollectedby SecurityCenter Nessus NessusHost Scans,Nes-susPlugins Splunk SplunkreceivesvulnerabilitydatacollectedbyNes-

Tenable and Splunk Integration · Splunk Splunkreceivesvulnerabilitydatacollectedby SecurityCenter Nessus NessusHost Scans,Nes-susPlugins Splunk SplunkreceivesvulnerabilitydatacollectedbyNes-

Referent / Redner Benjamin Tiggemann · 2015-07-08 · Splunk for Exchange Splunk for Enterprise Security* Splunk for Vmware Splunk for PCI Compliance* Splunk for Windows Splunk for

Referent / Redner Benjamin Tiggemann · 2015-07-08 · Splunk for Exchange Splunk for Enterprise Security* Splunk for Vmware Splunk for PCI Compliance* Splunk for Windows Splunk for

Admission Control using Splunk Enterprise Deployment Guide€¦ · Configuring Splunk Enterprise ... For more information on Splunk configuration, see Splunk documentation. Troubleshooting

Admission Control using Splunk Enterprise Deployment Guide€¦ · Configuring Splunk Enterprise ... For more information on Splunk configuration, see Splunk documentation. Troubleshooting

Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Splunk Deployment

Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Splunk Deployment

ReversingLabs Explainable Threat Intelligence Enriches ... Data Sheets/RL-Splunk...sent to Splunk. • In Splunk, the TitaniumScale report is correlated with other available Splunk

ReversingLabs Explainable Threat Intelligence Enriches ... Data Sheets/RL-Splunk...sent to Splunk. • In Splunk, the TitaniumScale report is correlated with other available Splunk

USING VMWARE HORIZON - USC Dana and David … · VMware Horizon software provides the ability to log into a virtual desktop environment and utilize full oper ating system functionality.

USING VMWARE HORIZON - USC Dana and David … · VMware Horizon software provides the ability to log into a virtual desktop environment and utilize full oper ating system functionality.

Splunk in Rakuten: Splunk as a Service for all

Splunk in Rakuten: Splunk as a Service for all

Splunk Validated Architectures · Splunk Engineers Responsible for managing the Splunk lifecycle. Managed Service Providers Entities that deploy and run Splunk as a service for customers.

Splunk Validated Architectures · Splunk Engineers Responsible for managing the Splunk lifecycle. Managed Service Providers Entities that deploy and run Splunk as a service for customers.