BEST Practices - Testing & Optimization | Bredan Rendan
-
Upload
caleb-whitmore -
Category
Documents
-
view
690 -
download
0
description
Transcript of BEST Practices - Testing & Optimization | Bredan Rendan
![Page 1: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/1.jpg)
Building a Successful Optimization Program
December 6, 2012 Brendan Regan
Marketing Optimization Specialist Analytics Pros
![Page 2: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/2.jpg)
The Next 50 Minutes…
Outputs
![Page 3: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/3.jpg)
The Next 50 Minutes…
• Preconceptions
• Agenda
• Curiosity
• Tour Guide
Outputs
• Applicable Smarts
• Actionable Steps
• Motivation!
![Page 4: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/4.jpg)
What We’ll Cover
1. A Common Definition
2. Philosophy of Optimization
3. What It Takes to Be Successful
4. Do These 4 Things Tomorrow
5. Q & A
![Page 5: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/5.jpg)
A Common Definition
of Optimization Because being on the same page is cool.
![Page 6: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/6.jpg)
The Art and Science of increasing the marketing
efficiency of a web property.
![Page 7: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/7.jpg)
“Why is it an Art and a Science?!?”
![Page 8: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/8.jpg)
![Page 9: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/9.jpg)
Unpacking the Definition
• Efficiency = more outputs w/ less inputs
• Outputs = sales, leads, subscriptions, engagement, etc.
• Higher efficiency = better Marketing ROI
• Often accomplished via A/B/n or MVT testing
– Less ideal, but “change and observe” counts, too
![Page 10: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/10.jpg)
If Marketing is an Engine…
Image credit: Popular Hot Rodding
![Page 11: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/11.jpg)
Optimization Philosophy
Our beliefs on Optimization
![Page 12: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/12.jpg)
Optimization Philosophy
1. Be data-driven
– Data (quantitative & qualitative) drives testing, not opinion
2. Be agile
– A small lift next week is better than a larger lift next year
3. Lift + Learn
– It’s obviously about lift, but testing also provides valuable customer/business insights
![Page 13: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/13.jpg)
Optimization Philosophy (cont’d)
4. Gamble
– Some tests will lose
– Greater risk = greater potential lift
5. Baby steps
– Do what’s feasible ASAP, and build to more complex tests
6. A process, not a project
– The long-term goal is testing as part of marketing operations
![Page 14: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/14.jpg)
What It Takes To Be Successful in Optimization
A framework
![Page 15: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/15.jpg)
What It Takes to Be Successful in Optimization
People
Process Tools
![Page 16: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/16.jpg)
What It Takes to Be Successful in Optimization
People
![Page 17: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/17.jpg)
People (Skill Sets)
• Business/Marketing Acumen
• Data Analysis
• Heuristic/Usability/UX
• Visual Design
• Optimization SME
• Copywriting
• Technical SME
• Project Management/Coordination
![Page 18: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/18.jpg)
People (Skill Sets)
• Business/Marketing Acumen
• Data Analysis
• Heuristic/Usability/UX
• Visual Design
• Optimization SME
• Copywriting
• Technical SME
• Project Management/Coordination
![Page 19: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/19.jpg)
What It Takes to Be Successful in Optimization
Process
![Page 20: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/20.jpg)
Process (See? It’s Simple!)
Business Goals
Target Audience
Usability Data
Web Data
Heuristic Data
Challenges Hypotheses
Design Variations
Build Variations
Design Test
Build Test
Analyze Results
Report Findings
Next Steps
![Page 21: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/21.jpg)
Process (a simple one)
1. Define the problem
2. Research/observe the problem
3. Form a hypothesis
4. Conduct experiment
5. Analyze experiment results
6. Form a conclusion
7. Socialize results
Look familiar?
![Page 22: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/22.jpg)
Process (7 Simple Steps)
For more details:
http://brendan-regan.com/the-steps-of-the-scientific-method-for-marketers/
Disclaimer: My personal blog’s snark does not represent the views of Analytics Pros ;-)
![Page 23: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/23.jpg)
Process (Different Models)
1. In House
– Build a team
2. Outsourced
– Leverage partners/vendors/agencies
3. Hybrid (most common)
– Outsource specialized expertise
• Data analysis
• Testing SME
• Creative
![Page 24: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/24.jpg)
What It Takes to Be Successful in Optimization
Tools
![Page 25: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/25.jpg)
Tools
• What technologies are needed to:
–Coordinate resources
–Design variations
–Document requirements
–Conduct experiment
–Analyze results data
– Socialize experiment results
![Page 26: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/26.jpg)
Tools
• What technologies are needed to:
–Coordinate resources
–Design variations
–Document requirements
–Conduct experiment
–Analyze results data
– Socialize experiment results
![Page 27: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/27.jpg)
Tools
Etc.
![Page 28: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/28.jpg)
Do These 4 Things Tomorrow
1. Conduct “gap analysis” on people skills and tools
– Do we have enough resource allocation?
– Do we need new tools?
– Do we have all the necessary skills in-house?
![Page 29: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/29.jpg)
Do These 4 Things Tomorrow
2. Get a testing tool in place
– Do we have IT support?
– Build vs. buy?
– Price range?
![Page 30: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/30.jpg)
Do These 4 Things Tomorrow
3. Build a business case (if necessary)
– Case studies, calculators, presentations?
– Opportunity cost?
– Tying to corporate goals?
– Who is scared of testing?
![Page 31: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/31.jpg)
Do These 4 Things Tomorrow
4. Get resources and/or outside help
– Agency partners?
– Consultants/specialists?
– Testing vendors?
– Training?
![Page 32: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/32.jpg)
Questions? Comments?
Stretch those vocal chords.
![Page 34: BEST Practices - Testing & Optimization | Bredan Rendan](https://reader034.fdocuments.us/reader034/viewer/2022051817/547994c3b4af9fd3158b4884/html5/thumbnails/34.jpg)