Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4!...
Transcript of Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4!...
![Page 1: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/1.jpg)
Copyright © 2014 Splunk Inc.
Rupak Pandya | OI Prac>ce Manager, Func>on1
Speeding up Dashboards with Pivot
www.function1.com
![Page 2: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/2.jpg)
Disclaimer During the course of this presenta>on, we may make forward-‐looking statements regarding future events or the expected performance of the company. We cau>on you that such statements reflect our current expecta>ons and
es>mates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-‐looking statements,
please review our filings with the SEC. The forward-‐looking statements made in the this presenta>on are being made as of the >me and date of its live presenta>on. If reviewed aRer its live presenta>on, this presenta>on may not contain current or accurate informa>on. We do not assume any obliga>on to update any forward-‐looking statements we may make. In addi>on, any informa>on about our roadmap outlines our general product direc>on and is subject to change at any >me without no>ce. It is for informa>onal purposes only, and shall not be incorporated into any contract or other commitment. Splunk undertakes no obliga>on either to develop the features or func>onality described or to
include any such feature or func>onality in a future release.
2
![Page 3: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/3.jpg)
Agenda
3
! Background ! Splunk Search ! Overview of Data Models ! Overview of the Pivot Search Command ! Demonstra>on
![Page 4: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/4.jpg)
Meet Func>on1
4
! Founded in 2007, Func>on1 is an enterprise technology solu>on firm and has been a Preferred Splunk Partner since 2011
! We have 11 consultants in our Opera>onal Intelligence group that specialize in delivering Splunk Professional Services
![Page 5: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/5.jpg)
Meet Rupak Pandya
5
! Prac>ce Manager of the Func>on1 OI Group
! Joined the Opera>onal Intelligence team as an experienced consultant from a large, global consul>ng company
! Avid follower of all of the Washington, D.C. area sports teams
rupak@func*on1.com 301.452.2475 ext 24
![Page 6: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/6.jpg)
Anatomy of a Splunk Search index=_internal sourcetype=splunkd group=per_sourcetype_thruput | eval host_series = host + "_" + series | stats sum(ev) by host_series | rename sum(ev) AS "Total Events" | sort - "Total Events"
> Search And Filter
> Enrich
> Report
> Format
6
![Page 7: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/7.jpg)
Challenges
7
index=warum sourcetype="ri:pas:applica>on" | lookup employee_details empusername as user OUTPUT empshiRstarbme empshiRend>me empshiRdays | eval hour = tonumber(strRime(_>me,"%H")) | eval overnight = if(empshiRstarbme>empshiRend>me,1,0) | eval valid_>me=if((overnight==1 AND (hour >= empshiRstarbme OR hour <= empshiRend>me)) OR (overnight==0 AND hour>=empshiRstarbme AND hour<empshiRend>me),1,0) | eval shiRday=if(overnight==1 AND hour < empshiRend>me,rela>ve_>me(_>me,"-‐1d"),_>me) | eval weekday = strRime(_>me,"%a") | eval shiRday=strRime(shiRday,"%a") | eval valid_day=if(match(empshiRdays,shiRday),1,0) | search valid_>me=0 OR valid_day=0 | bucket _>me span=5m | stats count by _>me user command object
! Complicated searches get very verbose ! Searchers need to understand data’s structure ! Non-‐technical users might not have knowledge of underlying data ! Splunk admins do not always know what users will be searching on
![Page 8: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/8.jpg)
There Has to be a Beker Way…
8
![Page 9: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/9.jpg)
Data Model Goals
9
Make it easy to share/reuse
domain knowledge
Admins/power users build data models
Non-‐technical users interact with data via pivot UI
![Page 10: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/10.jpg)
Data Models
.
![Page 11: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/11.jpg)
What are Data Models?
11
! Hierarchically structured search-‐>me mapping of seman>c knowledge about one or more datasets – Fields that data models use are
called akributes – To create an effec>ve data model,
you must understand your data sources and your data seman>cs
![Page 12: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/12.jpg)
A Data Model is a Collec>on of Objects
12
![Page 13: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/13.jpg)
Objects Have Constraints and Akributes
13
![Page 14: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/14.jpg)
Child Objects Inherit Constraints and Akributes
14
![Page 15: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/15.jpg)
Child Objects Inherit Constraints and Akributes
15
![Page 16: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/16.jpg)
Building Data Models
![Page 17: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/17.jpg)
Three Root Object Types
17
Event ! Maps to Splunk events ! Requires constraints
and akributes • Search • Maps to arbitrary Splunk search (may
include genera>ng, transforming and repor>ng search commands)
• Requires search string akributes • Transac.on • Maps to groups of Splunk events or
groups of Splunk search results • Requires objects to group, fields /
condi>ons to group by, and akributes
![Page 18: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/18.jpg)
Three Root Object Types
18
Event ! Maps to Splunk events ! Requires constraints
and akributes Search ! Maps to arbitrary Splunk search (may
include genera>ng, transforming and repor>ng search commands)
! Requires search string akributes • Transac.on • Maps to groups of Splunk events or
groups of Splunk search results • Requires objects to group, fields /
condi>ons to group by, and akributes
![Page 19: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/19.jpg)
Three Root Object Types
19
Event ! Maps to Splunk events ! Requires constraints
and akributes Search ! Maps to arbitrary Splunk search (may
include genera>ng, transforming and repor>ng search commands)
! Requires search string akributes Transac.on ! Maps to groups of Splunk events or
groups of Splunk search results ! Requires objects to group, fields/
condi>ons to group by, and akributes
![Page 20: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/20.jpg)
Anatomy of a Data Model
20
Data models are comprised of one or more objects
Hierarchical Parent/ Child Rela>onship
The child objects inherit the constraints (searches, filter on the data) and akributes.nt object
![Page 21: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/21.jpg)
Where are they Stored?
21
! Each data model is a separate JSON file
! Lives in [app_name]/[local|default]/data/models
! Edi>ng this file is not supported!
![Page 22: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/22.jpg)
Data Model Accelera>on
22
! Data model accelera>on is a tool that you can use to speed up data models that represent extremely large datasets
! Data model accelera>on summaries take the form of >me-‐series index files (.tsidx) stored on Indexers
! Cannot edit accelerated models
![Page 23: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/23.jpg)
Data Model Accelera>on
23
ACCELERATION
Admin/Power User
splunk> Server
Non-‐technical User NO ACCELERATION
Turn on accelera>on
via UI
Run a pivot report
Poll: are there new accelerated models?
Sebng wriken to conf file
Kick off collec>on
Run search using on-‐disk accelera>on
Kick off ad-‐hoc accelera>on and run search
![Page 24: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/24.jpg)
Best Prac>ces
24
! Use event objects as much as possible: benefit from data model accelera>on
! Minimize object hierarchy depth where possible: Constraint based filtering is less efficient as you move down the tree
! Data model accelera>on is most efficient if the root event object being accelerated includes in its ini>al constraint search the index(es) that should be searched over
![Page 25: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/25.jpg)
Things to Watch For…
25
! Data model accelera>on only affects the first event object hierarchy in a data model
! Object constraints and akributes cannot contain pipes or subsearches
! Lookups used in akributes must be globally visible (or at least visible to app using the data model)
! No versioning on data models or objects
![Page 26: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/26.jpg)
Pivot
![Page 27: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/27.jpg)
How Does Pivot Work?
27
! Data models used to define the broad category of event data
! Hierarchically arranged collec>ons of data model objects to subdivide the original datasets
! Define the akributes that you want Pivot to return results on
![Page 28: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/28.jpg)
Pivot Editor
28
![Page 29: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/29.jpg)
Search Commands
29
! Search commands that allow you to u>lize data models – datamodel – tstats – pivot
![Page 30: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/30.jpg)
Pivot
30
! Required arguments – datamodel-‐name – objectname – Pivot search
ê Has its own syntax that is different than the Splunk Search Processing Language
ê There are various elements that can be used here such as cell values, rows, columns, filters, limits, row and column formabng, and row sort op>ons
! Command that fuels the Pivot Editor
![Page 31: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/31.jpg)
Using the Pivot Command on a Custom Dashboard
31
! Create a search sing the Pivot Editor ! Click the Open in Search magnifying glass ! Create a dashboard with mul>ple user inputs ! Use the |pivot search you created with the Pivot Editor as the search for your dashboard panel
! Update the |pivot search with the filters you wish to provide your users
![Page 32: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/32.jpg)
Demo This demo will show you how to use the power of accelerated data models on a custom dashboard
32
![Page 33: Speeding*up*Dashboards* with*Pivot...MeetFunc>on1* 4! Founded*in*2007,*Func>on1*is*an*enterprise*technology*solu>on* firm*and*has*been*aPreferred* Splunk*Partner*since*2011*! We*have*11*consultants*in](https://reader034.fdocuments.us/reader034/viewer/2022051902/5ff20bdba9365b542d714433/html5/thumbnails/33.jpg)
THANK YOU