Dan Crow - Becoming a Data Driven Company LEANCONF 2013

24
Songkick: becoming a data-driven company Dan Crow

description

Talk given by Songkick's Dan Crow at Leanconf 2013

Transcript of Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Page 1: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Songkick: becoming a data-driven company

Dan Crow

Page 2: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

whoisDan CrowCTO, Songkick@crowquine

Ten years in Silicon Valley● Apple● Verb● Unicru● Co-founder: Blurb

Five years at Google● Google Squared● Ads and Android

Page 3: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Songkick

We do live music● Started in 2007● One of first 10 startups in London● Tell us who your favorite bands are, we’ll

alert you when they play in your town● Second largest live music service in the

world

Page 4: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Core product

Page 5: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

BoomtownIt’s a rat trap, and you’ve been caught

Page 6: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Where we wereMid 2010:● 2M monthly uniques● Big ball of Ruby mud● One big dev team● One release every 2 weeks● Velocity trending to zero● Unhappy dev team

But: talent + agile = hope

Page 7: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Simplify the productAfter four years our product had accrued cruft● Complexity in user interface● Complexity in features

Time to radically trim what we delivered● What do users actually use?● What do we need to support core use

cases?

Page 8: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Simplify the technologySimpler product means:● Delete a lot of code● Service-oriented architecture● Page-component web app model● Reduced build times● Continuous integration● Web apps separated from backends● Create new apps fast● Support many more apps

Page 9: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Technology impacts teamComponentized tech means:● Small, independent teams● Clear goals● Clearer roles, e.g. Tech Lead● Distributed, devolved decision making● Need for process across teams

Page 10: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Results● Build time from 4 hours -> 25 minutes● Developer velocity improved 4x● Site performance improved 2x● User satisfaction increased

○ faster site○ less cluttered, easier UI

Page 11: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

The impactre

leas

es p

er w

eek

Page 12: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Continuous deploymentI made my way to the top of the hill...

Page 13: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

How it works for us● Jenkins build server● Ten minute build time● Automatic push to production● Features can accumulate behind flippers● Acceptance testing in production● Manual testing in production with real data● Business metric monitoring● Roll forward

Page 14: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

CD workflow

Page 15: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Mostly a cultural change● Quality becomes a dev issue● Whole company must buy in● Business must be willing to break things● Development is a flow, not a cycle● Some people won’t adapt to it● Took us 9 months to do right

Page 16: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Experiments

Page 17: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

How to experimentEverything in a startup is an experiment: your company, each product, each feature● Experiments answer a question

○ Don’t present a fully formed feature● What’s the lowest cost way to the answer?

○ Most experiments fail, so move fast○ e.g. Do our users want to share? Sharing buttons

that do nothing

Page 18: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

How we do itMultivariate testing environment:● Feature flippers allow us to direct traffic● Split traffic across versions A/B/C…● Use a variety of stats packages to analyze

○ Google Analytics○ Songkick “skab” package

● Code and design are always thrown away● Successful experiments = feature rebuild

Page 19: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Again, a cultural change● Always find the MVE● Be comfortable with lower quality● Get used to experiments failing● Experiment often● Trust your users, not your intuition

Page 20: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Downsides● Frequently changing user experience● Can get stuck in local minima● Hard to maintain consistency● Stats can be challenging to interpret● You need traffic to make it work

Page 21: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Benefits● Only users can tell you what works● Avoids question-asking biases● Ties feature improvements to business goals● Results constantly surprise us

Page 22: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Wrap upData is good

Page 23: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

Where we are now● 9M monthly unique users● iPhone and Android app (2M users)● Tourbox - portal for artists● Detour - crowdfunding for concerts● Integrations with Spotify, Foursquare● Comprehensive ticketing product● Smaller dev team, moving faster

Page 24: Dan Crow - Becoming a Data Driven Company LEANCONF 2013

In summary● Getting the technology right is critical● Experiments are the heart of lean● It’s always about the culture

○ Hardest thing to change○ Takes time and effort○ You need everyone to work that way○ Has the biggest impact