Charles Schwab: Optimized Customer Experiences with Big Data and Oracle Real-Time Decisions...

Charles Schwab: Optimized Customer Experiences with Big Data and Oracle Real-Time Decisions [CON5352] Customer Experience Optimization

Andy Welch, Principal Architect Charles SchwabRich Masi, Partner NewVantage PartnersJoe Khazen, Director, Real Time DecisionsOctober 1, 2014

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 4

Agenda

1

2

3

Overview of Real Time Decisions

Charles Schwab-RTD and BigData Use Case

Future Plans

Q & A4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |5

CREATING GREAT EXPERIENCES IS THE IMPERATIVE

ATM

PORTAL

BRANCH

Call Center

WEB

MOBILE

SOCIAL

KIOSKS

“know me, and wow me”“understand me, and reward me”

“meet me, and engage me”“delight me, and guide me”


LEVERAGEDATA

OPTIMIZEEXPERIENCE

ADAPT QUICKLY

What We Hear from Customers


SocialIn Store Contact Center

Field Service

Channel SalesCX for

SalesCX for

MarketingCX for

Commerce

Real Time Decisions (RTD)

CX forServiceCloud

PlatformServices

SocialPlatformServices

Common Hardware Systems Infrastructure

Direct Sales

Web

Mobile

Oracle’s Customer Experience Portfolio


Optimize the Customer Experience with RTD• Flexible Way to Make Decisions

– Single decision engine supporting a consistent customer experience across all channels

– Easily Integrated into Existing Applications– Goals, Rules, Models, Optimization, Arbitration

• Automated Self Learning– Incrementally builds Analytical models for Learning

and Decisions– Analytical Adaptive Decisions

• Quantifiable Results– Quantifiable and Measureable Lift on Each Project

– Various Test and Control Scenarios

Optimal & Personalized Customer AND Business Centric

Recommendations

KPIArbitration

Eligibility Rules& Models

Offers/NBAs

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Real time contextual data

Historical data

Relevant external sources eg Social Media

Target Audience

Predictive Modeling

+

+

+ =

σ

Exceptional Customer Experience

Self-LearningLoop

Personalized recommendations,

offers & actions

Real Time Decisions

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Restricted

10

Oracle Customer ExperienceOptimize Every Decision

Marketing• Customer experience

optimization– Themes, Colors, Navigation

• Next best offer

• A/B and multi-variant testing

• Content personalization

Service• Service treatments

optimization

• Customer retention programs

• Call center optimization

• Risk and fraud analysis enhancement

• Next best action

Sales• Customer Acquisition

• Cross Sell/Upsell

• eMail Personalization

• Offer Optimization

• Next best action


12/20/2012 11

11

Optimized Customer Experiences with Big Data and Oracle Real-Time Decisions

Andy WelchPrincipal Architect, Charles Schwab

Rich Masi,Partner, NewVantage Partners


12/20/2012 12

Why Oracle Real-time Decisions

• Speed• Context sensitivity• Scalability

Technology Foundation• Marketing velocity• Goal management• Analytical ideation

• Across a channel experience• Across channels• Across a customer lifecycle

Scope of Decisions Marketing Operations

World-Class Decision Management

Key Functional Architecture Component Areas.

1. Open architecture to support real-time communication with channels2. Decision management business interface to manage a decision set and decision strategy3. Decision service that returns an optimized result to channel requests in real-time4. Learning service that is updated in real-time based upon channel actions5. Open architecture to support integration with data systems including a big data platform6. Business insight platform for reporting and analytics

Goal: Optimize customer experience by delivering relevant contentOracle RTD was selected because it best met functional needs for a world-class real-time decision

management environment

12


12/20/2012 13

13

RTD at Charles Schwab

• Initial use of RTD to support content in real-estate across hundreds of Schwab.com pages

• Supports millions of optimized content requests per day

• Meets very tight response time SLA

• More than double the response rate versus legacy approach


12/20/2012 14

Learning Service

Decision Service

14

RTD Database

Big Data Cluster

1. login sends start session informant to RTD with a customer identifier when the user has been successfully authenticated

2. RTD sources the customer profile from the big data cluster using the customer identifier though a custom integration and establishes RTD session

3. RTD sources choice history from RTD database4. Page visit sends Advisor request for optimized content to RTD.5. RTD decision service processes business rules, predictive analytics

and decisions6. RTD returns an ordered list of content ids to page7. Web page calls content system to render content8. A content response sends a response informant to RTD9. RTD updates session profile10. RTD logs choice history and learnings11. Learning service processes learning records

1

5

RTD

RTD High Level Flow

4

3

RTD Session2

6

CMS7

8

9

10

11

Login Pages


12/20/2012 15

15

Enterprise Decision Management

• Created a decision management taxonomy that maps to business stakeholders management practices.

• Designed this structure to support:

Enterprise level goal management practices Multivariate testing at a content level Multiple channels Many placements (pages) Many slots (page positions) per placement with varying number of items returned per slot Many slot types (content types) for placements Decision strategy testing within and across placements and slots A learning graph based upon content, user and placement metadata dimensions


12/20/2012 16

16

Initial Test Design

BAU Random Group

Random 5% of

population

Random selection of

BAU banners

Business Rules Group

Random 15% of

population

Random selection

within business

rules

Rules + Analytics

GroupRandom 80%

of population

Response likelihood selection

within business

rules

• Visitors are placed in a group in real-time based upon random assignment which is persistent for future visits.

• Response rate for Business Rules Group is significantly greater than the Business as Usual (BAU) Random Group

• Response rate for Rules + Analytics Group is more than double the Business as Usual (BAU) Random Group response rate

• Group membership and real-time likelihood score are written to table with each event

• Business can add other groups or change definition of current groups through a business interface


12/20/2012 17

17

Decision Processing

Calculate likelihood of

response

Business Rules Group

Filter for content eligibility

Assign random score

Determine Group Membership

Eligibility Rules

Score Content

Process Decision Strategy

Arbitrate based upon likelihood

score

Filter for content eligibility

Rules and Analytics

Group

Arbitrate based upon random

score

BAU Random Group

No eligibility

rules

Assign random score

Arbitrate based upon random

score

Group membership,

rules, and decision

strategy are all configurable

through a business interface


12/20/2012 18

Multivariate Testing

130.000%

0.001%

0.010%

10,000 100,000 1,000,000 10,000,000

Presented Count by Response Rate

Response Rate

Presented Count

Quadrant #1 Choices

presented often and

performing well

Quadrant #2 Choices

presented less often and

performing well

Quadrant #3 Choices

presented less often and

performing not as well

Quadrant #4 Choices

presented often and

performing not as well

Best Practices Benefits Bucket

Define a cycle time period (week) for test and learn activities

Build Best Practices

Beginning of period, monitor the

program changes and identify opportunities

Grow Best

Practices

Middle of the period, review opportunities

with business and make change

decisions

End of the period, start

change process and make

changes that can be made.

• Rolling out multivariate testing practices to promote continuous campaign improvement

• Based upon a proven practice developed over 15+ years of real-time marketing program optimization

• Practice identifies program areas with the most opportunity for improvement and aligns marketing levers with improvement areas

• As a continuous champion-challenger optimization strategy, the value received from these practices compound rapidly and are typically are the largest value driver for programs

18


12/20/2012 19

19

Learning Graph

Placement

Slot

Slot Type

Channel

User OS

User Application

User Device Type

General Subject

Detailed Subject

Category

Content

• Created a learning graph by configuring RTD to learn on metadata of content, placement, and user info

• All presentation and response events are written to predictive analytics that build a response profile of higher order elements up and across the learning graph

• All analytics partitioned by channel to allow for multi-channel rollout


12/20/2012 20

20

RTD / Big Data Integration

RTD Database

RTD

RTD Session

Data Service

Decision Service

Learning Service

Java GetProfile

Data Cluster

• Developed custom big data integration with RTD.

• Big Data Cluster has a robust customer profile information for millions of customers.

• Creates customer profiles from multiple data sources and application-specific definitions.

• Significantly outperforms benchmark database retrieval


12/20/2012 21

21

Highly Available and Scalable Environment

• Highly Available: Multiple Active/Active Data Centers

• Scalable: Multiple servers running Decision Services in each location with learning service on separate instance

DR DB Replica

Web Page


12/20/2012 22

Next Steps

Decision strategy testing Multivariate testing Rollout to additional placements/slots Enhance data model Expand channels

22

Charles Schwab: Optimized Customer Experiences with Big Data and Oracle Real-Time Decisions...

Documents

Transcript of Charles Schwab: Optimized Customer Experiences with Big Data and Oracle Real-Time Decisions...