Webcast: Measure or Die: Data Informed Product Management: A Yammer Case Study

Post on 14-Sep-2014

361 views 0 download

Tags:

description

In order to increase the number of engaged users, Yammer incorporates data analysis steps into all stages of the product management process from ideation to prioritization to running the project to success metrics.“Data is not perfect, but we try to learn as much as we can from it.” Join Yammer Product Manager Neil McCarthy and Data Analyst Ken Pascual as they show how they use analytics to inform product opinions, make project decisions, and determine the impact of features using A/B testing. Background reading: Testing Yammer’s Signup Flow http://bit.ly/1jo5yh6 Shortening the Signup Flow Quiz http://bit.ly/1emqtwJ About the Speakers: Neil McCarthy, Yammer Product Manager Neil joined Yammer as a Solution Engineer and soon after made the switch to Product Management. He immersed himself in Yammer's data informed approach and has since taken an interest in spreading the word. He enjoys jogging and beer, usually not at the same time. https://www.linkedin.com/in/neilmccarthy Ken Pascual, Data Analyst, Yammer Ken worked in yield management at CBS Interactive, was the only data person at Tripit. http://www.linkedin.com/pub/kenneth-pascual/11/322/259 Moderated by Cindy F. Solomon, Global Product Management Talk & Founder, Startup Product.

Transcript of Webcast: Measure or Die: Data Informed Product Management: A Yammer Case Study

© AIPMM 2014 www.aipmm.com

www.aipmm.com

© AIPMM 2014 www.aipmm.com

© AIPMM 2014 www.aipmm.com

Follow AIPMM: @AIPMM

Use Hashtags

#AIPMM #ProdMgmt #datadriven

Tweet!

© AIPMM 2014 www.aipmm.com

Participate and Win!

A lucky winner will get a free copy of the Guide to the Product Management and Marketing Body of Knowledge (ProdBOK®)

Must be present to win!

© AIPMM 2014 www.aipmm.com

Today’s Speaker

Presenters:

Neil McCarthy Product Manager @hardkornelius

Ken Pascual

Data Analyst @vorped

Moderator:

Cindy F. Solomon, CPM, CPMM @cindyfsolomon

Measure or Die Neil McCarthy

Ken Pascual

Who we are

Neil McCarthy Product Manager

Kenneth Pascual Data Scientist

What Yammer is

Justify features

Make decisions

Determine success

HOW DATA HELPS US PRIORITIZE Step 1: Justifying Ideas

Prioritize

• Measurable impact

• How much we learn

• Progress toward vision

Last attempt

A/B test this

Clues from the past

Brainstorming

Idea!

Does this align with our goals?

• Vision: Yammer is a primary communication tool that users use to get work done.

• Goal: Grow the number of users that use the product.

• Problem: We lose too many potential users in our signup flow.

How important is this?

big drop!

And HALF of our invites come from the signup flow!

Why justify?

HOW DATA HELPS US DETERMINE THE SUCCESS OF THE PROJECT

Step 2: Determining Success

What’s your hypothesis?

What we came up with…

If we move the email confirmation step to the end of the signup flow,

then more users will make it through the signup flow and more users will be retained.

A/B Test Results: Moving the email activation step

A/B Test Results: Moving the email activation step

Interpretations

• Users saw the steps in a different order, and dropped off in similar percentages, BUT

– Less users saw the prompt to “check email” because it was at the end.

– Thus, less users activated via email.

– Thus, less users had the opportunity to engage.

Why we chose this example

How do you do this?

• Gotta log data!

• Choose your own metrics

• Data driven vs. data informed

• Build your processes around data

• How much PMs should know about statistics.

Moderator: Cindy F. Solomon @cindyfsolomon Global Product Management Talk

Presenters Neil McCarthy @hardkornelius

Ken Pascual @vorped Yammer

Q & A

Thanks for coming!

• Neil McCarthy, @hardkornelius

• Ken Pascual, @vorped

• Adam Zadikoff (original PM for this project), @zadikafka

• More info and a quiz on data informed PM: https://medium.com/yammer-product/13eeb6435a73

© AIPMM 2014 www.aipmm.com

Upcoming Courses

Course & Location Dates Days Time

CPM® Certification Prep Course & Exam

Seattle, WA

McLean, VA

May 5-6, 2014

June 9-10, 2014

M, T

8:30 am – 5 pm

CPMM® Certification Prep Course & Exam

Seattle, WA

McLean, VA

May 7-8, 2014

June 11-12, 2014

W, Th

8:30 am – 5 pm

ACPM® Certification Prep Course & Exam

Seattle, WA

McLean, VA

May 9, 2014

June 13, 2014

F

8:30 am – 5 pm

For more information, visit:

Transform Your Career.

© AIPMM 2014 www.aipmm.com

Athens, Greece

Belgrade, Serbia

Athens, Greece

Follow the links above for more information, or contact:

certification@aipmm.com.

Upcoming Courses

© AIPMM 2014 www.aipmm.com

Athens, Greece

Belgrade, Serbia

Upcoming Courses

Follow the links above for more information, or contact:

certification@aipmm.com.

© AIPMM 2014 www.aipmm.com

Please Join Us Again! Why Your Product Needs A Data & Analytics Strategy April 18, 12 pm ET How Software Analytics Can Improve Your Products April 25, 12 pm ET

AIPMM Webinar Series: http://aipmm.com/aipmm_webinars

Global Product Management Talk http://www.blogtalkradio.com/prodmgmttalk

Call For Speakers: http://www.boldpm.com/call_for_speakers

Announcements: http://www.aipmm.com/subscribe

LinkedIn: http://www.linkedin.com/company/aipmm

Membership: http://www.aipmm.com/join.php

Certification: http://aipmm.com/html/certification/