Post on 12-May-2015
description
Optimizing for change: Taking risks safely & e-commerce
Kellan Elliott-McCrea@kellanCTO, Etsy
Monday, October 8, 12
Monday, October 8, 12
Launched June 18, 2005 in Brooklyn875,000 monthly active sellers33.5MM items for sale$525MM in sales in 20111.43B page views, in Aug102 engineers74 releases, yesterday
Monday, October 8, 12
Take more risks.Build a better software.
Have more fun.
Monday, October 8, 12
“Sure that works when you’re building social software but what about a real business with $$$ involved?” - everybody always
Monday, October 8, 12
Continuous Deployment:
small changes, pushed frequently
Monday, October 8, 12
you can’t avoid making mistakes
you can avoid making BIG mistakes
Monday, October 8, 12
MTBFMTTR
What are you optimizing for?
Monday, October 8, 12
MTBFMTTR
Monday, October 8, 12
4 core techniques: 1. Put a Button On It 2. Branch in Code 3. Trunk is Always Deployable4. Dark/Incremental Launches
Monday, October 8, 12
Put a Button On It.
Monday, October 8, 12
4 core techniques:if ($cfg[‘awesome_new_search’]) {
# new hotness$rsp = do_solr();
} else {# boring old stuff$rsp = do_grep();
}
Branch in code:use features flags
Monday, October 8, 12
4 core techniques:Branch in code:use features flags
for free you get:1% launchesadmin only launchesdark launchessplit tests
Monday, October 8, 12
any engineer can launch an experiment to 1% of users
57 experiments live right nowMonday, October 8, 12
Metrics driven development
measure everything!feedback loops!
Monday, October 8, 12
Engineers love to measure
make it ridiculously easy
Monday, October 8, 12
Metrics driven development
StatsD::timing("page.render", $msec);
Monday, October 8, 12
Metrics driven development
Monday, October 8, 12
Metrics aren’t optional
a feature isn’t done without metrics
Monday, October 8, 12
Make metrics visible
remove the passwords
Monday, October 8, 12
Some tools:
Graphite, Ganglia, Logster*, StatsD*, event beacons, log files, EMR, Vertica, Splunk
Monday, October 8, 12
Getting started? Use StatsD
StatsD @Instagram, Pinterest, Github, Mozilla, LAN.com, Zynga, Kickstarter, LivingSocial and 70+ other companies
Monday, October 8, 12
Step 1: your 5 core metrics:
@ Etsy:sign ups, logins, checkout, new listings, posts in the bugs forums
Monday, October 8, 12
Who watches the graphs?
Monday, October 8, 12
USE COMPUTERS!
Automate your analysis
Monday, October 8, 12
Automate your analysis
holtWintersConfidence(Upper|Lower)
Monday, October 8, 12
Automate your analysis
continuous integration: unit tests, coding standards, static analysis, risky code paths
Monday, October 8, 12
Make effective security easy by default
Make insecure patterns “grep-able”
Monday, October 8, 12
Actively monitor for attacks.
Spikes in 500s and failed logins are your first clue.
Monday, October 8, 12
“I discovered the vuln late Friday afternoon and wasn't quite ready to email it to them. Saturday morning, I confirmed the hole was still there and fixed a few bugs with my demo.I had my girlfriend test it from her house. It didn't work for her. I tested again and it had stopped working for me. Sure enough, it was now properly sanitized and had the correct JSON MIME type.The following Monday I received a response thanking me for reporting it, and telling me I was right. “
Monday, October 8, 12
Treat independent security researches with respect.
Monday, October 8, 12
“Culture eats strategy for breakfast”*
(*possibly apocryphal) Monday, October 8, 12
Thank you!
Monday, October 8, 12