1

Optimization System Architecture2015/07/28 BPStudy#95Kaizen Platform, Inc. @dtaniwaki

2

About dtaniwaki

2008 - 2011 : Trend Micro (in Taiwan)2011 - 2014 : Tabelog, Inc. (in New York)2014 - : Kaizen Platform, Inc. (in Tokyo)

Computer Language : Ruby on Rails, Node JS, C, C++ and etc.Human Language : English, Chinese, Spanish (un poco)Interest : Scuba Diving, Rugby, Yoga

Github : https://github.com/dtaniwaki

3

?

What is Kaizen Platform?

ROI Optimization PlatformA/B Testing Tool

4

Basic Optimization

5

VariationsOriginal

Web Optimization

Distribution Ratio

Original

Variation A

Variation B

Variation C

Variation D

A B

C DThe best variation!

6

Web Optimization Steps

✓ Generate JavaScript with the test condition✓ Attach it on the customer’s page✓ Collect visit logs by the pixel✓ Collect conversion logs by the pixel✓ Calculate the distribution ratio✓ Update the JavaScript with the ratio

7

Customer B

Customer A

LP / CV

Web Optimization Architecture

JavaScript Template

LP / CV

Creatives

Creatives

Log App

Log Storage AppTest

Condition

Distribution

Ratio

Test Condition

Distribution

RatioVisit / Conversion Log

Generate

Generate

1st Party

1st Party

8

Creatives

AD Optimization

The best creatives!

9

AD Optimization Steps

✓ Generate JavaScript✓ Submit it as 3PAS✓ Collect impression logs by the pixel✓ Collect click logs through the redirector✓ Collect conversion logs by the pixel✓ Calculate the distribution ratio✓ Stop low performance creatives

10

AD Optimization Architecture

JavaScript Template

Media (3PAS)Creatives

Log App

Log Storage AppDistributio

nRatio

Impression / Conversion Log

LP

Redirector

CV

Click Log

3rd Party

3rd Party

Generate

Generate

11

AD x Web Optimization

AD Optimization Web Optimization

Maximize the inbound Optimize with clicked banners

12

AD x Web Optimization Steps

✓ Memorize the clicked banner ID through the redirector✓ Get the clicked banner ID by XHR✓ Collect logs with clicked banner ID✓ Get the result of clicked banner by the ID

13

Web x AD Optimization ArchitectureMedia (3PAS)

LP

Redirector

CV

Cookie Sync APIGet clicked Banner ID

AD Apps

Web Apps

Visit / Conversion Log w/ Banner ID

Get results by Banner ID

1st Party

1st Party

3rd Party

Impression / Click Log

3rd Party

Generate

Generate

Generate

14

Basic Optimization Issues

15

Legacy Optimization

A

Creatives Audiences

B

C D

CVR 30% CVR 15%

CVR 20% CVR 5%Distribute based on

CVR

16

Legacy Optimization Issue

✓ The distribution ratio is calculated by overall CVR✓ Each audience has different feeling on creatives

17

Per-Segment Optimization

Male 30sFemale 30s

Male 40sFemale 40s

A

Creatives Audiences

Gender

Age

Gender

Age

B

C D

Female 30s Male 30s

Male 40sFemale 40s

Distribute Creatives based on segments

18

Per-Segment Optimization Steps

✓ Collect logs with audience segments✓ Calculate the distribution ratio per segment✓ Choose creative based on the distribution ratio of their segments

19

Per-Segment Optimization Architecture

Log App Log Storage App CreativesDistributio

nRatio for Segment

ADistribution

Ratio for Segment

B

Distribution

Ratio for Segment

CDistribution

Ratio for Segment

D

Log with Segment LPGenerate

Calculate per segment

20

Per-Segment Optimization Issue

✓ Hard to choose from segment combinations○ Ideal case

e.g. “Male 30s”, “Female 30s”, “Male 40s” and “Female 40s”

○ Difficult case

e.g. “Male”, “Female”, “30s”, “40s” “Male” x “30s”, “Male” x “40s”, “Female” x “30s”, “Female” x “40s”

21

Kaizen Optimization Platform(a.k.a. CVR Predictor)

22

Kaizen Optimization Platform

Female x 30s

Male x 50s

Female x 40s

A

BA: 70%

B: 30%

?Male x 40s B

Female x 50s

A…

Big Data

Machine Learning

23

Kaizen Optimization Platform Steps

✓ Set up a scheduled batch task✓ Upload the content into the storage✓ Collect logs with audience segments✓ Calculate the coefficients of estimated CVR based on the audience

segments✓ Choose the best creative based on the audience segments on the

fly

24

Kaizen Optimization Platform Architecture

AudienceKaizen Optimization Platform

Log Server

App ServerKaizen KVS Kaizen Predictor

Log Storage

Kaizen Test API

Web Optimization

Platform

Prediction Batch

AD Optimization

Platform

Bandit Algorithm

Machine Learning

AD Log App

Round API

Variation API

CVR Prediction

UUID Segments Round Variation CVABCX Male, 30s R1 V1 1ABCY Female, 20s R1 V2 0

...

R1 CoefficientsV1: { a: 0.8, b1: 0.3, b2:

0.6 }V2: { a: 0.3, b1: 0.1, b2:

0.4 }

R1

Batch Options for R1

Send log w/ segments

Dispatch

Web Log App

V1

V2

V1

AD JS Web JS

25

Per-Audience Optimization

Device

Access Time

Access DoW

DMPClicked Banner

Language

CRM

Female

40s

Clicked Banner A

Friday

Japanese Tokyo

Gender

Age

Place

26

Thank You!