Unexpected facts of life by Orly Amrany Nov 2015

Orly Amrany

General Manager, Wix

When the question ‘why?’ is not relevant

The Unexpected Facts of Life

Hi, I’m Orly Amrany General Manager, Head of Premium Services @ Wix

Question #01

• Assume you have 3 Million Subscription (Monthly/Yearly)

• Each recurring charge is done on it’s purchase date

• Re-try upon failure is done every 7 days

• Success rate of recurring charge is ~90%

• Which day do we have the best changes to charge ?

24th or 25th?

85.25%

81.94%

84.14%

79.80%

82.79%

85.67%

85.13%

83.24%

81.59%

86.80%

83.55%

85.48%

86.49%86.96%

79.00%

80.00%

81.00%

82.00%

83.00%

84.00%

85.00%

86.00%

87.00%

AU BR CA ES FR GB US

24th Vs. 25th, initial recurring

85.25%

81.94%

84.14%

79.80%

82.79%

85.67%

85.13%

83.24%

81.59%

86.80%

83.55%

85.48%

86.49%86.96%

79.00%

80.00%

81.00%

82.00%

83.00%

84.00%

85.00%

86.00%

87.00%



+0.5% increase in approval rate

85.25%

81.94%

84.14%

79.80%

82.79%

85.67%

85.13%

83.24%

81.59%

86.80%

83.55%

85.48%

86.49%86.96%

79.00%

80.00%

81.00%

82.00%

83.00%

84.00%

85.00%

86.00%

87.00%




+15k saved subscriptions

85.25%

81.94%

84.14%

79.80%

82.79%

85.67%

85.13%

83.24%

81.59%

86.80%

83.55%

85.48%

86.49%86.96%

79.00%

80.00%

81.00%

82.00%

83.00%

84.00%

85.00%

86.00%

87.00%




+15k saved subscriptions

+1.5M

And the winner is…

The 24th!

You’ve won 1,500,000million dollars

Question: Why checking the 24th vs. 25th?

Question: Why checking the 24th vs. 25th?

Answer: Because we could

• We had data of million of transactions

• We actually checked all days of the month

• A lot of other assumptions failed..(15th Vs 16th)

• Should we AB test? (NO)

On the 24th there is more changes for charges to succeed

We gained +1.5M USD with a simple change!

When you have tones of data, you can find out a lot of interesting and unexpected facts

You should do it as soon as possible: Time is money

Make all recurring charges on the 24th ? (NO) Improve re-try logic (Which Tool)

So what did we learn?

So what did we learn?

Don’t ask yourself

‘why?’

Ask yourself

‘why didn’t I check this before?’

Some more fun examples

Question #02

• Same data

• Which day do we have the best changes to charge?

Weekdays or weekends?

And the winner is…

Weekday!

4.41%

-1.85%

6.36%

2.42%

5.38%

8.00%

-0.59%

1.18%

5.67%

-4.84%

6.39%

0.45%

2.97%

5.94%

-6.67%

0.10%

-0.62%

5.21%

7.62%

1.50%

2.59%3.22%

-5.88%

3.41%

0.66%

11.80%

6.81%

-0.35%

8.05%

7.64%

2.39%

-10.00%

-5.00%

0.00%

5.00%

10.00%

15.00%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Approval rate difference - weekdays vs. weekends

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

• Same data

• Which hour of the day would be the best? (in Brazil for example)

Time or Day?

And the answer is…

8am – 10pm

20.00%

25.00%

30.00%

35.00%

40.00%

45.00%

50.00%

55.00%

60.00%

65.00%

70.00%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Approval Time, Recurring/UTC time

all

Thanks

Unexpected facts of life by Orly Amrany Nov 2015

Data & Analytics

Transcript of Unexpected facts of life by Orly Amrany Nov 2015