#evolverocks

REV UP YOUR MARKETING ENGINE

GORDON PIKE3SHARE ARCHITECT

09/01/2016

2#evolverocks

FINE TUNE YOUR AEM MARKETING ENGINETHE KEY IS TO GET MORE FROM LESS

WHAT WOULD A 10% CONVERSION RATE MEAN TO YOUR BUSINESS?WHAT ABOUT 15%, 20%, 25%?

3#evolverocks

You already have themYou have an endless supply• Medium to Large sites

can generate 20-30 GB a day

Usually collected and forgotten• Retained for a short

time• Used to diagnose

issuesCan be used for much more• Proactive improvement• Business Insight• Business user analysis• User analysis

LET’S TALK ABOUT LOG DATAI THOUGHT WE WERE TALKING ABOUT REVVING ENGINES?

4#evolverocks

Source of real-time eventsHigh performance collection• Optimized for fast writes• Minimal impact to request• Other methods insert

themselves into the request

Analyze post collection• Once stored can be

queried and visualizedSource of critical insights• Your systems• Your applications• Your users

THERE’S GOLD IN THEM THERE LOGSUNDER OUR NOSE ALL THE TIME

5#evolverocks

LOG COLLECTION VS. INLINE COLLECTIONMULTIPLE WAYS TO GET YOUR METRICS

Log Collection• Measured in nanoseconds or microseconds

• Optimized for write• Usually local write to server file system

• Minimum impact on site performance

• You already write to logs

Inline Collection• Measured in milliseconds or seconds

• Include JavaScript• External Link• External write

• Http requests• Increases Page Latency• Impacts site functionality

6#evolverocks

TOP TEN USE CASES FOR LOG DATA25,000 LOG MANAGEMENT USERS SURVEYED

Production Monitoring; 71%

Production Troubleshooting; 66%

Debugging During Devel-opment; 41%

Web Application Analytics; 34%

Support; 32%

Real User Monitoring; 28%

Security; 22%

Quality Assurance and Test-ing; 21%

Business Analytics; 10%Mobile Application Analytics;

6% All Other Responses; 5%

7#evolverocks

LOG MANAGEMENT MATURITY LEVELS

Implement• Debug during

development• Verification of bugs

during testing

Troubleshoot• Issue Forensics• Root Cause Analysis

Monitor• Real User

Monitoring• Performance• Alerts• Stabilize

environment

Analyze• KPI Measurement• Trend Analysis• Track how

enhancements add/subtract from business goals

Reactive Proactive

8#evolverocks

REVVING THE ENGINEA COMPLETE LOG DATA STRATEGY IN 4 STEPS

Measure

Monitor

Discover

Audit

9#evolverocks

MEASURE

IF YOU CAN'T MEASURE IT, YOU CAN'T IMPROVE IT. - Peter Drucker

10#evolverocks

MEASURE - NOT ALL METRICS ARE EQUALCHOOSE BALANCED FOOD NOT JUNK FOOD

Vanity Metrics•# of Visits•# Tweets•Google Ranking

Common Metrics• Requests per

second• Processing time• Request timeouts• Status codes• Broken pipes• Exceptions

Key Performance Indicators

•Conversion Rate•Cart Abandonment Rate•Products Per Order

11#evolverocks

Is your Key Performance Indicator (KPI) SMART?• Is the objective specific?• Can you measure progress?• Is the goal attainable?• Is the goal relevant to your organization?• What is the timeframe to achieve the goal?

MEASURE - CHOOSING A KPIA KEY PERFORMANCE INDICATOR IS ONLY VALUABLE IF IT INSPIRES ACTION

12#evolverocks

MEASURE -YOUR KPI SHOULD FIT THE SITEEXAMPLE KPI’S FOR 3 SITE TYPES

Blog Site

• Subscriber Rate• Leads Per Day• Income Per Page• Income Per

Visitor

eCommerce Site

• Conversion Rate• Cart

Abandonment Rate

• Products Per Order

• Average Order Value

• Upsell Rate

Lead Generation Site

• Leads Per Day• Form

Abandonment• Content Requests

13#evolverocks

MONITORTURNING MEASUREMENTS INTO ACTIONS

• Measure metrics over time

• Establish norms• Highlight

deviations

Detect

• Dashboards• Email• Text

Alert • Identify routine actions

• Automate actions• Trigger action on

event

Automate

14#evolverocks

DISCOVERQUERY METRICS TO UNCOVER INSIGHTS

Normalize

Common Date

formats

Enrich

StoreFlexible Schema

Optimized for Read

Search

Query

Learn/Query Again

Analyze

Ask 5 Why's

Develop Insights

15#evolverocks

AUDITRETRIEVABLE HISTORY

Monitor• Alert

Store• Restrict access• External• Preferably write once

• Input / Output validations• Authentication• Authorization• Sessions• Errors• Legal Events

16#evolverocks

BUILDING YOUR AEM ENGINE DIAGNOSTIC SERVICE

ENTER ELK STACK

17#evolverocks

ARCHITECTURE

ApplicationCollect, Parse, Ship

Collect, Enrich,

Transport

Store, Search, Analyze

Explore, Visualize,

Share

Kibana Elasticsearch Logstash

Filebeat Author

Filebeat Publisher

Filebeat Dispatcher

18#evolverocks

BEATS DATA COLLECTORSTopbeat•Infrastructure Metrics•Resource Utilization•CPU•Memory

Metricbeat•Fetches metrics on predefined intervals

•From operating system•Operating System•Services (Apache, etc.)

Packetbeat• Network Data• Web• Database• Other network protocols

Filebeat• lightweight log forwarder• Collect• Pre-processes• Forwards

Beats

19#evolverocks

Lightweight Log Data ShipperHarvester• One per log file• Keeps track of

statements sentProspector• Set of processing rules• Multiple log file types

Spooler• Ships events to

consumer• Logstash• Elasticsearch

FILEBEATAGENT FOR LOG FILE SHIPPING

20#evolverocks

Real time PipelineUnify Data From Disparate Sources• Log files• Geo data• Twitter feed

Data Normalization• Convert date formats• Common field names

Pluggable Pipeline Architecture• Inputs• Filters• Output

LOGSTASHDATA COLLECTION ENGINE

21#evolverocks

LOGSTASH DATA PIPELINEInputs• Beats• Files• Syslogs• Twitter feeds

Filters• Grok• Date/Time

Normalization• Geo Data Lookup

Outputs• Elasticsearch• Pagerduty

22#evolverocks

Real time PipelineDocument Oriented Storage• All fields indexed by default• Schema-Free

Searchable• All data searchable• Full-Text search• Automatically detects data

structure and type Analyze • Powerful query language• Built on Lucene

ELASTICSEARCHSTORE, SEARCH AND ANALYZE

23#evolverocks

Flexible Analytics and Visualization PlatformReal-time Summary and Charting of Data• Flexible queries• Flexible Time windows

Visualizations• Charts• Tables• Maps• Widgets

Embeddable Dashboards • Combine Visualizations into

Dashboards• Share dashboards and needed

KIBANAEXPLORE, VISUALIZE, SHARE

24#evolverocks

ARCHITECTURE WITH AEM

LogsAuthor

Publisher

Dispatcher

CDN

Logs

Logs

Logs

Logs

tash

Elas

ticse

arch

KibanaAuthor

Publisher

Dispatcher

CDN

Filebeat

Filebeat

Filebeat

Filebeat

25#evolverocks

DEMO TIME

26#evolverocks

Q & A

#evolverocks 27

ABOUT US

Facebookfacebook.com/3share

MailGordon.pike@3sharecorp.com

Twittertwitter.com/gpike

Linkledinlinkledin.com/gordon-pike

Gordon PikeArchitect3Share Corporation

Contact Uswww.3sharecorporation.com

Phone: 720.608.6159

#evolverocks

THANK YOU!