Insights from designing a Sitecore-BI

integration

Why are we talking about this?

• Data as competitive advantage;

the “new currency”

• Sitecore as a contributor of

behavioural data

• Accessibility of tools: PowerBI,

Azure, Google Data Studio

The data collected by Sitecore can

inform important business

decisions, especially combined

with CRM and ecommerce

3/28/2018 4

Single View of …. Everything?

ERP

Finance

AI / ML Data Consumers

Etc…

A more present-day example

6

1

Lessons learned

from recent

Sitecore-BI

integration

[Disclaimer: Things get easier in Sitecore 9!]

Our ongoing client story: business context

• Professional membership association

• Seeking to continually increase brand value and offer

deeper value to the professional community

• Online revenue from publications, subscriptions, courses,

educational materials

• Recently revised and re-issued corporate objectives

• Running Sitecore 8.1

8

The asks

• Build a reporting framework for user behaviour to aid in the

measurement and optimization of marketing activities

• Collaborate with Client’s BI team to extract relevant data

into Client’s data warehouse

• = a great opportunity to demonstrate the value of xDB!

9

Insight #1: Consensus on how to

measure the digital business is a

fundamental prerequisite to BI

Initial ingredients

KPI Measurement Framework Maximizing use of Sitecore metrics 20 KPIs and tagging detail

12

Business Objective

Increase membership signups

by 10% in 2018

Marketing Strategy

- Align channel experiences

to customer intents

- Streamline the signup

process

KPI

- Members signups

- Signup conversion rate

- Engagement value

Segment

- Channel

- Persona

- Goal

Measurement Framework

** maximize usage of Sitecore-specific metrics and filters so they could be easily

implemented and measured

Insight #2: Marrying your KPIs to

marketing questions ensures the

dashboard can be designed to

answer them

3/28/2018 14

KPI: Session Value

This key measurement provides an understanding of the business value [Client]’s audience produces

Segments

+ All Data (Sitecore, GA)

+ Pattern Card (Sitecore, GA)*

+ Content Category/Tag (Sitecore, GA)

+ Member vs Non-Member (Sitecore, GA-Long Term)

+ Goal/Event (Sitecore, GA)

+ Country, Region, City (Sitecore, GA)

+ Channel, Campaign, Source (Sitecore*, GA)

+ New vs Returning (Sitecore, GA)

Questions That Can Be Answered

+ How much business value is being generated per visitor?

+ Which profile produces the most/least business value per visitor?

+ How much business value is generated on a particular category per visitor?

+ How much business value are Members producing per visit?

+ How much business value is being produced that included a particular asset/goal per session?

+ What geography is producing the most/least business value per visitor?

+ What marketing activity is driving the most/least business value?

+ How much business value are return/new sessions producing?

Insight #3: Choosing the right data

source requires careful

consideration and future-proofing

3/28/2018 16

Google Analytics vs. Sitecore Analytics?

+ Client had established maturity in using Google Analytics

+ Google Data Studio possible for short-term drill-down reports

+ However, GA came with some limitations:

+ Data queries would begin to apply sampling after a certain threshold to

keep performance in check

3/28/2018 17

3/28/2018 18

Pros of Sitecore that won out

+ Full, unrestricted access to all behavioral data and ability to do a nightly pull

into the data warehouse

+ Consistency between use of engagement value both as a KPI and later for

onsite personalization (as opposed to using only GA goals)

+ Extensibility of contact data model, useful later with future ecommerce

+ Client’s commitment to Sitecore as digital platform and licensing investment

Insight #4: Choosing the right

connection method gives

maximum data flexibility for BI

3/28/2018 20

Option 1: Connecting to the SQL Reporting Database using native connectors in Excel or Tableau

+ The way that Sitecore aggregates the data into those tables (facts and

dimensions) did not support the pivoting and filtering of data that the

dashboards needed

+ For example, the tables aggregated around specific metrics, or they removed

too many other fields that we wanted in the database

3/28/2018 21

Mapping dashboard data points to source options

3/28/2018 22

Option 2: Using Sitecore’s Reporting API to fetch data

+ This API will deprecate once xConnect is released in Sitecore 9

+ I would not recommend investing in a code/ETL layer using this older API

3/28/2018 23

Option 3: Querying Mongo directly

+ We had a need for more freedom in manipulating and pivoting the entire dataset in

multiple ways

+ We could not derive all data from the aggregated tables in the way we needed

+ We settled on making as much of the raw data available in the data warehouse so

the BI team could have complete freedom in building the dashboards and joining

to other business data

+ Caveats: we’ll invest in the 8.1 Mongo spec which will evolve with upgrades, and

we’ll need to do some customizations/API calls to get the full data we need

Insight #5: Pulling extra data at

collection time future-proofs the

architecture, with some exceptions

3/28/2018 25

Example 1: Content Category Tag lookup

+ How to discern which visitors had looked at content tagged with a particular

category?

+ We couldn’t do it reliably by template type, URL or any other piece of data

natively stored in the xDB

3/28/2018 26

Considering options

+ Option 1: Do an API call at transformation time to look up whether a content

tag existed on a viewed page

+ This runs the risk of tagging having changed when doing the lookup retro-

actively

+ Option 2: Capture the tag in a Page Event when the interaction was recorded

at collection time

+ We have a snapshot of the tag string at the moment of capture

+ Caveat: additional storage space used in Mongo

3/28/2018 27

Example 2: Storing Traffic Type lookup locally in data warehouse

Insight #6: Iterations work really

well for dashboard assembly and

testing consumption and value

3/28/2018 29

Educating the BI team and iterating on questions

+ Our project ran across a number of weeks alongside BI sprints

+ The BI team was working with a SQL-based data warehouse, but were

unfamiliar with Mongo

+ This structure allowed for a weekly rhythm of questions to naturally arise as

they mapped the dashboard field requirements to their transformation pipeline

+ I kept a living document of questions and answers

+ Very important: the BI team was provided a full dev data environment with

expected enrichment tagging

3/28/2018 30

Example: Translating Mongo GUIDs to Sitecore-searchable GUIDs

+ Run this Mongo Shell command (in this example, Page ID of first page visited)

+ db.getCollection('Interactions').find({_id:LUUID("f5a7ece7-5073-ad47-a4bd-

7963ad484513")})[0].Pages[0].Item._id

+ Browse to this utility:

+ http://guid-convert.appspot.com/

+ Use the HEX value to search the GUID in the Sitecore interface

+ Alternatively – the Studio 3T tool lets you paste and convert the GUID from Java encoding to .NET

encoding, with same result of being search in Sitecore

+ Must remember to change Preference to .NET notation

3/28/2018 31

In particular:

+ The implementation will challenge the KPI definitions and clarifications will

likely be needed

+ “what decisions will be made with this KPI?”

+ “why did we prioritize it as important?”

3/28/2018 32

Example: Translating dashboard requirements to formulas with Mongo fields

+ The formula for KPI “Loyalty Rate” is defined as:

+ 1. Total Visits (Interactions.Id.Count with at least one page contains “thought leadership” page event)

+ 2. Total Contacts (Interactions.ContactId.Count(Distinct) within the total visits defined as point 1 )

+ 3. Total Repeat Contacts (Total Contacts that have more than 1 Visits within the total visits defined in

point 1 )

+ Rate(%) = Total Repeat Contacts / Total Contacts(%)

Transformation pipeline (2 minutes)

Pull every 24h

Join on primary key (CustomerID)

“primary-eligible”

CRM

3/28/2018 34

v

35

2

The vision of

Sitecore 9

xConnect: making it easy to work with xDB data

+ Build applications to read, write or

search xDB data making it easy to

retrieve and consume analytics

data in an easy and flexible way

+ Sample applications

+ CRM integration

+ Data warehouse

+ Predictive modeling

3/28/2018 37

How organizations can leverage xConnect

Pulling CRM data

into Sitecore for

segmentation and

personalization

Capturing

physical/IoT data

such as in-store

beacon tracking

Simplified export

to data

warehouse

Applications of

data science for

greater insight

Sitecore Cortex: “Your own data scientist inside Sitecore’

+ Embedded machine learning

+ Coming with version 9.1

3/28/2018 39

The promise of machine learning for marketing

… or why is everyone putting machine learning in their software?

+ Machine learning lets us replace human judgement with prediction

+ What do computers do better than humans? Learning from and finding patterns within large datasets to make “good enough” predictions

+ (What do humans do better than computers? Creative applications of ML!)

+ Tedious, data-crunching processes that exist – in marketing, or in any discipline – are candidates for automation with machine learning

+ A classic marketing example:

+ The automation of image tagging via image recognition techniques

+ Instead of a human looking at each image and tagging “flower” or “car”, a computer could do that, and get smarter about it

Native connectors … happy dashboarding!

thank you