Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Splunk Deployment
Google Cloud & Splunk Hybrid Environments with Operating ...
Transcript of Google Cloud & Splunk Hybrid Environments with Operating ...
Alex CainSenior Product Manager | Splunk
Operating & Securing Hybrid Environments with Google Cloud & Splunk
Nic StoneSolutions Engineer | Splunk
© 2019 SPLUNK INC.
Senior Product Manager | Splunk
Alex CainSolutions Engineer | Splunk
Nicolas Stone
Use this if there will be two speakers for your session.
© 2019 SPLUNK INC.
During the course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC.
The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release.
Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. © 2019 Splunk Inc. All rights reserved.
Forward-Looking Statements
THIS SLIDE IS REQUIRED, DO NOT DELETE
© 2019 SPLUNK INC.
1. Google Anthos and Splunk Connect for Kubernetes
2. Stackdriver logging and VPC Flow logs
3. Cloud Security Command Center and Cloud Asset Inventory
Operating & Securing Hybrid Environments with Google Cloud &
Splunk
Agenda
© 2019 SPLUNK INC.
GDI Graph
Splunk Enterprise
DBX GCP TAHEC
BigQuery Cloud Storage
Stackdriver Monitoring
Stackdriver Logging
Cloud Security Command Center
Cloud Pub/Sub
GKE +GKE On-Prem
Cloud Asset Inventory
Compute, Storage, DB, Networking
Services
All GCP-MonitoredServices & Resources
Cloud Dataflow
© 2019 SPLUNK INC.
Google Anthos and Splunk Connect for KubernetesWhat are they? How do they work together?
© 2019 SPLUNK INC.
Hybrid Management, Monitoring,and Visibility
Anthos Anthos Splunk Connect for Kubernetes
Monitor/InvestigateManageBuild Send
Splunk App for Infrastructure
Cloud, on-prem, anywhere
© 2019 SPLUNK INC.
Build and manage modern hybrid
applications across on-premises and cloud
environments.
Anthos
© 2019 SPLUNK INC.
Import and search your Kubernetes logging, object, and metrics
data in Splunk
Splunk Connect for Kubernetes
© 2019 SPLUNK INC.
A seamless experience for infrastructure monitoring and troubleshooting
SplunkApp for
Infrastructure
© 2019 SPLUNK INC.
Hybrid Management, Monitoring,and Visibility
Google Kubernetes Engine
Splunk Connect for Kubernetes
Monitor/Investigate
Cloud, on-prem, anywhere
Manage Send
Splunk App for Infrastructure
Demo
© 2019 SPLUNK INC.
Insert your own screenshot here.For best results, use an image sized at 1450 x 850
© 2019 SPLUNK INC.
Insert your own screenshot here.For best results, use an image sized at 1450 x 850
© 2019 SPLUNK INC.
Insert your own screenshot here.For best results, use an image sized at 1450 x 850
© 2019 SPLUNK INC.
1. Deployment is simplified when using the SAI setup helper
2. Monitoring and investigation using the Splunk App for Infrastructure as a starting point
3. Essentially the same process for GKE, and Anthos
Anthos, GKE, Splunk App for Infrastructure
Demo
Key Takeaways
© 2019 SPLUNK INC.
Stackdriver logging and VPC Flow logsWhat, why, and how.
© 2019 SPLUNK INC.
▶ GCP service logs (audit, etc.) end up in Stackdriver Logs• Also referred to as GCP Logging
▶ Stackdriver logs can be configured to have a Pub/Sub topic as a sink destination
▶ Select which logs are routed to sink destinations using export query filters
Stackdriver logs – WhatWhat is Stackdriver logging
© 2019 SPLUNK INC.
The Stackdriver cheatsheet (AKA Query Library)Advanced (copy/paste-able) log export filtersBuilding Stackdriver logging exports isn’t hard, when you’ve got a query library cardhttps://cloud.google.com/logging/docs/view/query-library
© 2019 SPLUNK INC.
Insert your wn screenshot here.For best results, use an image sized at 1450 x 850
© 2019 SPLUNK INC.
GCP Pub/Sub – WhatWhat is this Pub/Sub thing?
▶ Cloud Pub/Sub is a fully-managed real-time messaging service that allows you to send and receive messages between independent applications
© 2019 SPLUNK INC.
Collect Google Cloud Platform events, logs, performance metrics
and billing data
Splunk Add-on for
Google Cloud Platform
© 2019 SPLUNK INC.
Push data from Pub/Sub to Splunk
HTTP Event Collector using this streaming
template
GCP Pub/Sub to Splunk Dataflow Template
© 2019 SPLUNK INC.
VPC Flow logs – What What are VPC flow logs?
Think of VPC Flow Logs like NetFlow, but with additional features.
• VPC Flow Logs provide responsive flow-level network telemetry for GCP environments
© 2019 SPLUNK INC.
▶ Network monitoring:• Monitor the VPC network and perform network diagnosis
• Understand traffic growth for capacity forecasting
▶ Understanding network usage and optimizing network traffic expenses:• Traffic between regions and zones and to specific countries
• Top talkers
▶ Network forensics:• Which IPs talked with whom and when
• Any compromised IPs by analyzing all the incoming and outgoing network flows
VPC Flow logs use cases – WhySecurity, monitoring, forecasting, and more…
© 2019 SPLUNK INC.
▶ VPC Flow logs (once turned on) are delivered to Stackdriver• Start by exporting to Pub/Sub
• Pull: Splunk Add-on for Google Cloud Platform Pub/Sub input− OR
• Push: GCP Pub/Sub to Splunk Dataflow template
▶ Log export filter? Let’s take a look at the cheat sheet. Copy pasta anyone?
Bring it inDid you go to the first session? You may have seen this before…
© 2019 SPLUNK INC.
Demo
© 2019 SPLUNK INC.
© 2019 SPLUNK INC.
© 2019 SPLUNK INC.
1. Standard Push or Pull processes for getting data from Pub/Sub to Splunk
2. Stackdriver to Pub/Sub is a common pattern and can be used to send all sorts of GCP data to Splunk
3. VPC flow logs can be a valuable data source for a variety of use cases
Stackdriver, Pub/Sub, and VPC flow logs
Key Takeaways
© 2019 SPLUNK INC.
Cloud Security Command Center and Cloud Asset InventoryWhat, why, and how.
© 2019 SPLUNK INC.
GCP Cloud Security Command Center Security and datarisk database
Cloud SCC enables you to understand your security and data attack surface by providing asset inventory, discovery, search, and management.
© 2019 SPLUNK INC.
© 2019 SPLUNK INC.
▶ The Splunk Connector app uses the Cloud SCC API to export an organization's assets and findings to Cloud Pub/Sub• Remember this?
• It’s a cool story, let me tell it again.
• Pull: Splunk Add-on for Google Cloud Platform Pub/Sub input
• OR
• Push: GCP Pub/Sub to Splunk Dataflow template
Bring it inConsistency
© 2019 SPLUNK INC.
Push assets and security findings to Pub/Sub for Splunk pickup or delivery
GCP Cloud Security
Command Center Splunk
Connector
© 2019 SPLUNK INC.
GCP Asset Inventory API GCP resourcesinventory service
Asset Inventory is a GCP API that can generate snapshots of assets in a GCP project/organization
• Snapshots are written to a GCS bucket
© 2019 SPLUNK INC.
▶ Entire Asset Inventory Snapshot? (Do this once a day/week/etc.)• Use the API to write the snapshot to GCS
• GCS -> Pub/Sub -> Splunk (Using the Splunk GCP Add-on or a Dataflow job)− OR
• GCS -> Dataflow -> Splunk
▶ Real time changes?• The Asset Inventory API has a feature that writes inventory changes directly to Pub/Sub
Bring it inTwo types, many approaches
© 2019 SPLUNK INC.
1. Leverage Google Cloud SCC for simplified GCP security monitoring
2. Asset Inventory is a powerful complement to audit logs
3. Getting data from the services into Splunk reuses all the same concepts you are now familiar with
Google Cloud Security Command Center and
the Asset Inventory API
Key Takeaways
© 2019 SPLUNK INC.© 2019 SPLUNK INC.
You!Thank
RATE THIS SESSION
Go to the .conf19 mobile app to
© 2019 SPLUNK INC.
Q&AAlex Cain | Senior Product ManagerNicolas Stone | Solutions Engineer