UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

65
Humanizing Data Bridging UX Research and Analytics Alex Wright Director of Research, Etsy [email protected] | @alexgrantwright

Transcript of UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Page 1: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Humanizing Data Bridging UX Research and Analytics"

Alex Wright Director of Research, Etsy [email protected] | @alexgrantwright

Page 2: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

2

Page 3: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 4: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 5: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

“Production by the masses, not mass production.” - Gandhi

Page 6: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

BY THE NUMBERS

6

1.5M active sellers

AS OF JUNE 30, 2015

21.7M active buyers

AS OF JUNE 30, 2015

$1.93B annual GMS

IN 2014

32M items for sale

AS OF JUNE 30, 2015

Photo by Kirsty-Lyn Jameson

Page 7: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Etsy is proud to be a certified B Corporation — a new kind of company that uses the power of business to solve social and environmental problems.

Page 8: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 9: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 10: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 11: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

wfs2015wfs2015

Page 12: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 13: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 14: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 15: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 16: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

16

Credit: Mike Brittain

Page 17: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

17

WE DEPLOY THE SITE ~30 TIMES PER DAY

Page 18: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 19: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 20: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 21: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Credit: Dan McKinley

Page 22: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

CI Status (1 hour)

End-User (1 hour)

Error Logs (All) (1 hour)

Other Errors (1 hour)

Business Graphs (1 hour)

Page Performance by Site Section (1 hour)

Statistics displayed represent signed in page requests. See Page Performance Dashboard for more detail.

Web Cluster (1 hour)

API Cluster (1 hour)

Gearman (1 hour)

Memcache (1 hour)

Master01 Database Activity (1 hour)

DBShard Database Activity (1 hour)

For more detailed shard information see the MySQL Shards Dashboard

DBIndex Database Activity (1 hour)

Outgoing Bandwidth (1 hour)

..................................................................

Deploy Dashboard Git Commits Princess Supergrep Supergrep Forum Watcher

Time: 1 hour Until: Now Hide Deploys Deploys hidden: Config Web / Api / Atlas Search Photos Blog Chef (prod) Schemanator

Push

rbateman

on qa6m since 7, 2012 9:54

Three-Armed Sweaterserror pages · historical

clip Akamai Whoopsiesorigin failures · historical

clip Edgecast Whoopsiesorigin failures · historical

clip Screwed Usersunique users · historical

clip

All

File or Script Not Foundcurrent · historical

clip Exceptionscurrent · historical

clip Fatalscurrent · historical

clip Errorscurrent · historical

clip Warningscurrent · historical

clip

Segmentation Faultscurrent · historical

clip PHP Allowed MemorySize Exhaustedcurrent · historical

clip PHP Maximum ExecutionTimecurrent · historical

clip Memcache ConnectionFailurescurrent · historical

clip Thrift Exceptionscurrent · historical

clip

StatsDpacket receive errors · historical

clip CSRF Mismatchcurrent · historical

clip CSRF Missingcurrent · historical

clip Client Side (JS) Errorscurrent · historical

clip Empty Image URLsgrey.gif · Default Avatars

clip

Registrationsfacebook · facebook (historical) · other · other (historical)

clip Logins (82.00%)logins · logins (historical) · errors · errors (historical)

clip Checkouts (97.81%)success · failure

clip

Post Checkoutscurrent · historical

clip Mobile CheckoutsiOS · mobile web

clip Direct Checkout Checkouts (98.00%)success · failure

clip

Listings Creation/RenewalNew & Copied · Renewed

clip Requestslogged-out · logged-in

clip CDN Requests Balanceakamai · edgecast · direct-origin

clip

Search AdsRequests · Responses · Logged

clip Forum Postsbugs forum · help forum

clip Seller Registration (100.00%)success · failure

clip

PHP time (median)listing · shop · profile · search · homepage

clip PHP time (95th percentile)listing · shop · profile · search · homepage

clip webpagetest front-end timelisting · shop · profile · search · homepage

clip lognormal load timelisting · shop · profile · search · homepage

clip

Page 23: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

CI Status (1 hour)

End-User (1 hour)

Error Logs (All) (1 hour)

Other Errors (1 hour)

Business Graphs (1 hour)

Page Performance by Site Section (1 hour)

Statistics displayed represent signed in page requests. See Page Performance Dashboard for more detail.

Web Cluster (1 hour)

API Cluster (1 hour)

Gearman (1 hour)

Memcache (1 hour)

Master01 Database Activity (1 hour)

DBShard Database Activity (1 hour)

For more detailed shard information see the MySQL Shards Dashboard

DBIndex Database Activity (1 hour)

Outgoing Bandwidth (1 hour)

..................................................................

Deploy Dashboard Git Commits Princess Supergrep Supergrep Forum Watcher

Time: 1 hour Until: Now Hide Deploys Deploys hidden: Config Web / Api / Atlas Search Photos Blog Chef (prod) Schemanator

Push

rbateman

on qa6m since 7, 2012 9:54

Three-Armed Sweaterserror pages · historical

clip Akamai Whoopsiesorigin failures · historical

clip Edgecast Whoopsiesorigin failures · historical

clip Screwed Usersunique users · historical

clip

All

File or Script Not Foundcurrent · historical

clip Exceptionscurrent · historical

clip Fatalscurrent · historical

clip Errorscurrent · historical

clip Warningscurrent · historical

clip

Segmentation Faultscurrent · historical

clip PHP Allowed MemorySize Exhaustedcurrent · historical

clip PHP Maximum ExecutionTimecurrent · historical

clip Memcache ConnectionFailurescurrent · historical

clip Thrift Exceptionscurrent · historical

clip

StatsDpacket receive errors · historical

clip CSRF Mismatchcurrent · historical

clip CSRF Missingcurrent · historical

clip Client Side (JS) Errorscurrent · historical

clip Empty Image URLsgrey.gif · Default Avatars

clip

Registrationsfacebook · facebook (historical) · other · other (historical)

clip Logins (82.00%)logins · logins (historical) · errors · errors (historical)

clip Checkouts (97.81%)success · failure

clip

Post Checkoutscurrent · historical

clip Mobile CheckoutsiOS · mobile web

clip Direct Checkout Checkouts (98.00%)success · failure

clip

Listings Creation/RenewalNew & Copied · Renewed

clip Requestslogged-out · logged-in

clip CDN Requests Balanceakamai · edgecast · direct-origin

clip

Search AdsRequests · Responses · Logged

clip Forum Postsbugs forum · help forum

clip Seller Registration (100.00%)success · failure

clip

PHP time (median)listing · shop · profile · search · homepage

clip PHP time (95th percentile)listing · shop · profile · search · homepage

clip webpagetest front-end timelisting · shop · profile · search · homepage

clip lognormal load timelisting · shop · profile · search · homepage

clip

Page 24: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

CI Status (1 hour)

End-User (1 hour)

Error Logs (All) (1 hour)

Other Errors (1 hour)

Business Graphs (1 hour)

Page Performance by Site Section (1 hour)

Statistics displayed represent signed in page requests. See Page Performance Dashboard for more detail.

Web Cluster (1 hour)

API Cluster (1 hour)

Gearman (1 hour)

Memcache (1 hour)

Master01 Database Activity (1 hour)

DBShard Database Activity (1 hour)

For more detailed shard information see the MySQL Shards Dashboard

DBIndex Database Activity (1 hour)

Outgoing Bandwidth (1 hour)

..................................................................

Deploy Dashboard Git Commits Princess Supergrep Supergrep Forum Watcher

Time: 1 hour Until: Now Hide Deploys Deploys hidden: Config Web / Api / Atlas Search Photos Blog Chef (prod) Schemanator

Push

rbateman

on qa6m since 7, 2012 9:54

Three-Armed Sweaterserror pages · historical

clip Akamai Whoopsiesorigin failures · historical

clip Edgecast Whoopsiesorigin failures · historical

clip Screwed Usersunique users · historical

clip

All

File or Script Not Foundcurrent · historical

clip Exceptionscurrent · historical

clip Fatalscurrent · historical

clip Errorscurrent · historical

clip Warningscurrent · historical

clip

Segmentation Faultscurrent · historical

clip PHP Allowed MemorySize Exhaustedcurrent · historical

clip PHP Maximum ExecutionTimecurrent · historical

clip Memcache ConnectionFailurescurrent · historical

clip Thrift Exceptionscurrent · historical

clip

StatsDpacket receive errors · historical

clip CSRF Mismatchcurrent · historical

clip CSRF Missingcurrent · historical

clip Client Side (JS) Errorscurrent · historical

clip Empty Image URLsgrey.gif · Default Avatars

clip

Registrationsfacebook · facebook (historical) · other · other (historical)

clip Logins (82.00%)logins · logins (historical) · errors · errors (historical)

clip Checkouts (97.81%)success · failure

clip

Post Checkoutscurrent · historical

clip Mobile CheckoutsiOS · mobile web

clip Direct Checkout Checkouts (98.00%)success · failure

clip

Listings Creation/RenewalNew & Copied · Renewed

clip Requestslogged-out · logged-in

clip CDN Requests Balanceakamai · edgecast · direct-origin

clip

Search AdsRequests · Responses · Logged

clip Forum Postsbugs forum · help forum

clip Seller Registration (100.00%)success · failure

clip

PHP time (median)listing · shop · profile · search · homepage

clip PHP time (95th percentile)listing · shop · profile · search · homepage

clip webpagetest front-end timelisting · shop · profile · search · homepage

clip lognormal load timelisting · shop · profile · search · homepage

clip

Page 25: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

“When the point of contact between the product and the people becomes a point of friction, then the designer has failed. “ - Henry Dreyfuss

Page 26: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Seller onboarding"

Page 27: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 28: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 29: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 30: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 31: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 32: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Data analysis

95.09% completed step 1 (shop name)

72.77% went on to list an item

Hypothesis: Seller onboarding is overwhelming

Page 33: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Seller onboarding"

Page 34: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 35: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Data analysis

24% increase in new shops opened within 7 days

Page 36: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Data analysis

24% increase in new shops opened within 7 days

BUT fewer listings, and fewer sales

Page 37: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 38: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 39: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Sold orders"

Page 40: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 41: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

41

Page 42: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Sold orders"

Page 43: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 44: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 45: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Exploratory search"

Page 46: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 47: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Data analysis"32 million listings

21.7 million buyers

High incidence of low-intent search queries

Minimal use of category trees

Page 48: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Qualitative insights"Searching on Etsy feels overwhelming

Buyers want to narrow results, but there’s FOMO

Search is used as a browse experience - Buyers expect/enjoy discovery and surprise

Buyers seemed to gravitate to browse paths on other retail sites. What were we missing?

Page 49: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Taxonomy

Page 50: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

PrototypeExperiment

Page 51: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Final release

Page 52: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 53: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 54: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 55: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Regenerative entrepreneurship"

Page 56: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

w

Page 57: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

sure we can bushwicksure we can bushwick

Page 58: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 59: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Rethinking “Capital”"

Page 60: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Financial capital

Page 61: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Social capital

Page 62: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Natural capital

Page 63: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Spiritual capital

Page 64: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"
Page 65: UX STRAT USA, Alex Wright, "Etsy: Bridging UX Research and Data Analytics to Drive Design"

Thank you""

Alex Wright"[email protected] | @alexgrantwright