Cultivating a Culture of Experimentation
-
Upload
optimizely -
Category
Technology
-
view
470 -
download
0
Transcript of Cultivating a Culture of Experimentation
![Page 1: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/1.jpg)
Speakers
Cultivating a Culture of Experimentation
Bryan Berger, Product Design Lead, General Assembly
Taylor Gilbert, Partnerships & Alliances, Optimizely
![Page 2: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/2.jpg)
Today’s Speakers
Bryan Berger
Taylor Gilbert
![Page 3: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/3.jpg)
Housekeeping
• We’re recording!• Slides and recording will be
emailed to you after the webinar is complete
• There will be time for questions at the end
![Page 4: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/4.jpg)
Agenda
• ‘Culture’ of Experimentation
• General Assembly’s Approach
• How to Energize Your Team
• Some Things We Tested
![Page 5: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/5.jpg)
‘Culture’What does that
mean?
![Page 6: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/6.jpg)
![Page 7: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/7.jpg)
![Page 8: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/8.jpg)
Turns out… culture is a
big deal.
![Page 9: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/9.jpg)
of Fortune 500 companies from 1955 are still in business today12%
¼ companies fell off the list in the last year alone
![Page 10: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/10.jpg)
The Organization of the Future…. Arriving Now
![Page 11: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/11.jpg)
1. Culture has never been more important, experimentation is the vehicle.
2. Your executives are struggling and they need your
help. They are looking for someone to step up and lead/orchestrate.
![Page 12: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/12.jpg)
General Assembly (GA) is a global educational company. Focusing on the most relevant and in demand skills across data, design, business and technology.
General Assembly is empowering a global community to pursue work they love through best-in-class instruction and access to opportunities.
![Page 13: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/13.jpg)
Experimentation at
General Assembly
![Page 14: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/14.jpg)
A Little About Me
● I manage our Product Design Team, whose focus is on designing an
ecosystem to support online and in-person learning experiences.
● I also work quite extensively with our Marketing and Website CRO team to
improve the positioning of our Products.
● One of my many goals is to foster a fun and impactful culture of
experimentation deeply rooted in the design process.
![Page 15: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/15.jpg)
Why Experiment?● Progress over perfection. Experimentation allows our team to ideate multiple
solutions to a problem and not paralyze the process in favor of “the one”.
● Expand potential. Anyone with a hypothesis can partake. Inclusion has become
an ally in bringing down silos among our product teams.
● Gain insights. To develop a better understanding of our data, and how we can
affect the levers of improvement.
● Build consensus. Save time and money by testing your assumptions.
![Page 16: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/16.jpg)
![Page 17: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/17.jpg)
Experimentation at General Assembly
● Started to hit the limits of our current solutions.
● Our business was scaling quickly and along with it, our growth targets.
● We needed to start questioning things that have been “established”.
● Our testing program has been running for about a year.
● Prior to that we had no repeatable testing process in place.
![Page 18: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/18.jpg)
How to Energize Your Team
![Page 20: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/20.jpg)
Get Creative
![Page 21: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/21.jpg)
Really Creative...
![Page 22: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/22.jpg)
Make It Known
![Page 23: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/23.jpg)
Highlight and Acknowledge Success
![Page 24: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/24.jpg)
Our Approach
![Page 25: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/25.jpg)
Getting Started: Finding a ProcessWhere we all start
Where some give up
The goal, a smoother repeatable process
![Page 26: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/26.jpg)
Our Initial Process
![Page 27: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/27.jpg)
Our Current Process
![Page 28: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/28.jpg)
Finding Great Ideas● All ideas must contain:
○ Problem definition○ Hypothesis○ Audience Type (Persona)○ Page url(s) affected
● All ideas are added to a backlog list○ Tickets are created or updated once ideas are vetted for impact, time to significance, and
resource allocation.
● Hold weekly A/B testing meeting to: ○ Discuss how a specific test would be implemented ○ Review cost / benefits for different possible solutions
○ Amount of traffic needed○ Length of the experiment○ Next steps post test
![Page 29: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/29.jpg)
Prioritizing Test Ideas
Website Tests are prioritized by our Marketing Team & Director of SEO/CRO
● We define quarterly Objectives & Key Results (OKRs) and roll up each test result toward one of those objectives
● Test key routes (purchase flow, wayfinding, and value prop improvement)● As of today, all tests aim to improve conversion and engagement at key steps of our funnel
![Page 30: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/30.jpg)
Stay Organized
![Page 31: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/31.jpg)
Share Results
Clear communication, acknowledgement, and collaboration.
● #CRO Slack channel for asking questions, and posting results.
● Announces test results and shoutouts.● Big wins are also sent to an email group.● Slack is effective. Everyone is engaged in it,
from engineering to execs.
![Page 32: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/32.jpg)
Program GoalsOur current benchmark is to have at least one live test on each of the most critical pages in our conversion funnel (to keep momentum):
○ New user signup modal○ Homepage○ Navigation○ Product pages○ Wayfinding systems○ Discovery & catalog pages
![Page 33: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/33.jpg)
Some Things We Tested
![Page 34: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/34.jpg)
25% Off Class & Workshop Offer
Modal Incentive Copy Test
Original Variation
![Page 35: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/35.jpg)
Drive more urgency to the next course start dateCourse Start Date
Original Variation
![Page 36: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/36.jpg)
Copy and styling tests in Navigation menu Copy and Style in Navigation
Original Variation
![Page 37: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/37.jpg)
Show the customer potential ROI of our courses
No original variation
Adding Sections
Original Variation
![Page 38: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/38.jpg)
Looking Forward
● As we scale our experimentation program we’ll likely bring in an agency to help build, design, & deploy tests in Optimizely.
● With their support, we’ll be able to move more quickly on our backlog of 80+ test ideas.
● Continue testing to inform design direction for our new site.● Penetration into our Product Apps and onboarding flows.
![Page 39: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/39.jpg)
Key Takeaways● Become the champion of your experimentation program● Make your testing program known, give it a physical presence● Start backlogging and organizing ideas● Establish a process to vet, prioritize and execute those ideas● Set up a staging environment to act as your laboratory● Hold yourselves accountable for stagnation● Beware of vanity metrics. ● Demonstrate how testing can be inclusive, impactful and FUN!
![Page 40: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/40.jpg)
Questions?
![Page 41: Cultivating a Culture of Experimentation](https://reader033.fdocuments.us/reader033/viewer/2022052706/58f02cfb1a28ab4c348b464b/html5/thumbnails/41.jpg)
Thank You!Feel free to reach out to continue the conversation:
@bryanberger @ga