DonorVoice presents:

A Proven Way to Get Better Returns from your Direct Mail Testing Program

Innovation Webinar Series

The Problem

Realities of The New World• Cost per donor to acquire goes up• Massive competition

Current Testing Process• Biased (too much reliance on conventional wisdom)• Slow & Costly (A/B testing to infinity)• Inefficient (Throwing out baby with bath water)

A New Way to Test• Feedback from target audience• Deceptively simple – let’s ask ‘em (how ask matters, a lot).

The Solution

Battle Tested

Product development• Member packages• Consumer package goods• Direct mail

Should bottle be red, white, blue, green, yellow?What should brand name be?What about size/shape of bottle?How much does it cost?What marketing messages do we putOn bottle?

Beat the Control. Period. End of Story.

Why Test?

How Much Cost to Beat Control?

Outer Envelope Type 1 Type 2 Type 3

OE Color goldenrod brownkraft gray

OE Teaser Free Gift!Annual Appeal

Look Inside!

OE Type 6x9 #10 Plain#10 with window

This is 27 tests with all

combinations (3x3x3).

27 TestsOld Method $88,500 (production costs, opportunity cost)New Method $24,000 (includes tool, 3 live tests)

269% difference

But We Don’t Do 27 Tests

If you do 7 or more tests a year, this is financially better deal on first project

Odds of finding control beating product sooner are much higher

For NO TEST organizations – Think your control can’t be beat? How much money is left on the table with a sub-optimum product?

+

Do survey resultsmatch real world?

Test A, #1 in DV Tool

Control, #2 in DV Tool

Test B, #3 in DV Tool

Case Study #1

Control family

Possible Test Appeals• Client needed to identify new family as “face” of organization.

• Wet test results performed exactly as predicted producing a test/family that did beat the control.

• If conventional wisdom had been relied upon the best options would not have been mailed.

Case Study #1Financial Benefit:• Production & postage savings from NOT mailing

all test packages $46,000

• Anticipated annual revenue increase with rollout of better performing control $72,000

$118,000

Color of Outer Envelope

Labels

Calendar Grid

Backend Premium

Personalization

Case Study #2

Case Study #2Financial Benefit:

• Savings from not producing & mailing predictedlosing test packages $45,000

• Increase in net revenue by doubling panel volume on predicted winners $20,400

$65,400

Okay, But…“Our best donors are not online…”

“People don’t do what they say...”

Other Weaknesses of Current Testing

Process

“Process” for Deciding What to Test

What do we test?

Purple it is!

A/B Testing to Infinity (and Beyond!)

6 variations, 216 possible combinations.

10 tests a year, 22 years

+ Can’t analyze across tests

• Test a did poorly• Test b did okay• Test c did well

Unanswered Question:If we combine b and c is that better, great, the same, or worse?

Baby in Bath Water

Test package that changes everything.

It loses.

Now what?

Our Process.

Step OneChoose What Package and What Part(s)

1. Envelope2. Letter3. Premium (front or back end)4. Combination of above

Gift Amount: $50, $75, $100, $125, $150, $200, $250

Teaser Copy: Headline 1…….6

Premium Example

Step TwoIdentify “Ingredients” to be Tested

Johnson Box Ingredient

Teaser Ingredient

Size Ingredient

Reply Form Ingredient

Message Ingredient

Step ThreeFinalize Grid/Artwork

ATTRIBUTE Level 1 Level 2 Level 3 Level 4

Labels L1 (LYH labels)L2 (Annual Appeal

- Nov version) none n/a

Outside envelope OE1 (control)

OE2 (double dopcut)

OE3 (love heart teaser) OE4 (4 color image)

Buckslip insert BS1 (control version)

BS2( double drop black and white

image)BS3( Double drop cut with 4 color image) n/a

Certificate

C1 (certificate of appreciation - Annual

appeal)

C2 (certificate of appreciation - LYH

version) none n/a

Reply Slip & Letter

RS1 (Control reply and letter)

RS2 (double drop cut no image reply

and letter)

RS3 (double drop cut with 4 color image

reply and letter)RS4 (love heart reply

and letter)

Lift Note noneLN2 (lift note black

and white) LN3 (yellow and blue) n/a

Step FourDesign Experiment Our tool determines what

combinations to show each respondent.

Step FiveConduct the Survey

Renewal• Client provides email sample

Acquisition• Develop filter questions• Match any profile

Informational Insert in Control

“Baby” significantly outperforms control Buckslip

Step SixProvide Results

Step SixProvide Results

What to Mail (real examples)• Run live test with lift note in addition to and in place of certificate• Live test (2012) alternative copy against “Year End”• Don’t bother with test colors on OE• Don’t bother with new design on calendar• Flower labels better than anchor for men and women• Tote better than duffle as back-end premium with renewal

Who To Mail • Formal education is big discriminator for package preference• Append to file for renewal selection• Look at applying in acquisition (skews lower HH income, in South)

Contact Information

Pretest Tool:Get to rollout faster.

Kevin SchulmanDonorVoice, CEO(202) 246-9649

kschulman@thedonorvoice.com

Twitter: @kschulmanDV