Analytics @ Wish

Powered by fluentd & MongoDB

Hi

I’m Adam.

Wish ♥︎ MongoDB

• Primary database since 2011

• 67x mongod

• AWS → bare metal (SSDs ftw!)

What’s Wish?

• Mobile eCommerce

• 30M+ users worldwide

• Top 10 iOS & Android

Experiment

‘cause otherwise you’re just guessing…

Hypothesis

“Billing Zip” is confusing outside America

Data

Compare checkout conversions for international,

Android users

Conclusion

~7% boost in mobile sales

Goal

Frictionless analytics to everyone

{“solution”: [“logging”, “aggregation”, “analysis”, “serving”]}

{“solution”: [“logging”, “aggregation”, “analysis”, “serving”]}

{“solution”: [“logging”, “aggregation”, “analysis”, “serving”]}

{“solution”: [“logging”, “aggregation”, “analysis”, “serving”]}

{“solution”: [“logging”, “aggregation”, “analysis”, “serving”]}

{“solution”: [“logging”, “aggregation”, “analysis”, “serving”]}

Request Logs = Source of Truth

{'contest_impressions’:'53060fbd34067e4d6cee70f4,535ad13a7360465e2ca799f8,528b714df689996fdb574800,525976a71c23882ab3b73ecb,5285df6db5baba737f459037,5208ae7d3deaf74a6cc65da4,5209e5c31c238861a1ab91cc,5285df6db5baba735f459061,51f7778f3ba3770a514a5431,527be1fc227d210d2bcdeac5,532fcfe3796f6832713b5c3a,527be203227d210dd5cdeaac,52d3ef2806ea960dde85cb97,527bc781227d210d8acdea47,527bc793227d210d4fcdea48, 5208ad653deaf74a4bc65d41,5208acdd1c238846f9ab9028,5182fc1273c67621e507591b,5311ae6c796f68283f8f86c3, 52de2bf4ab980a2d00da786a,5208a9c53deaf74a75c65c6b,52eca45a717951350382e4be,52d3ef73bb5aa51ccf866c01, 533d6fae5aefb0427771f346,5285df6db5baba734d45901b,51c27d8d5ffe8f0b0b9b0359,52d0e002a30fb227725b6e06, 52f71bd89f5ef741d8f34698,52d3ef71bb5aa53135866d76, 5308bc467360464265101ed9,52d3ef27bb5aa5024d866c09, 52c399d60599170e49fd866e,5209be541c23886177ab91db,5208b15e1c2388615fab91b7', '_country_code': u'CA', '_lang': u'en', '_fb_uid': 500406911, '_device_id': None, '_uid': '4eb346049b120f09f60007c0', '_tid': 2, '_host': 'adam.corp.contextlogic.com', '_last_id': u'cc3aa96b2b3c45bca11009edc049f2f6', '_experiment_tags': ['mobile_commerce_home_v4_female_ignore', 'mobile_large_cart_cell_ignore', 'hannibal_cohort_firsttime_buyer_ignore', 'localize_product_names__fr_ignore', 'mobile_cart_guarantee_view_ignore', 'mobile_related_tags_v2_ignore', 'shipping_price_us_ignore', 'stripe_settle_on_ship_control', 'related_super_feed_iphone_show-v4', 'mobile_commerce_home_v3_male_i18n_show', 'braintree_settle_on_ship_control', 'mobile_show_tabbed_billing_page_i18n_ignore', 'mobile_new_guarantee_text_ios_ignore', 'mobile_use_category_signup_flow_i18n_ignore', 'male_curated_first_ipad_ignore', 'mobile_commerce_home_v4_female_i18n_ignore', 'commerce_product_page_show', 'mobile_use_category_signup_flow_v3_ignore', 'mobile_save_for_price_us_female_relaunch_2_ignore', 'web_stripe_checkout_ignore', 'mobile_show_tabbed_billing_page_us_ignore', 'stripe_checkout_show', 'shipping_price_i18n_fixed-price-promo', 'chukou1_pilot_experiment_ignore', 'mobile_implicit_ratings_v1_show', 'feed_commerce_2_control', 'mobile_commerce_home_v3_male_ignore', 'swap_out_male_feed_show-weight-deep', 'related_super_feed_ipad_ignore', 'female_curated_first_iphone_ignore', 'mobile_psuedo_localized_currency_show', 'hannibal_cohort_repeat_buyer_ignore', 'web_boleto_checkout_ignore', 'exploration_v2_control', 'female_curated_first_android_ignore', 'male_curated_first_android_ignore', 'related_super_feed_android_show-v4', 'curated_feed_female_shopping_ignore', 'mobile_localized_currency_control', 'male_curated_first_iphone_ignore', 'mobile_show_required_shipping_fields_ignore', 'mobile_ct2_variable_shipping_price_showcountry', 'mobile_c2c_ignore', 'localize_product_names__es_ignore', 'related_products_v2_control', 'female_curated_first_ipad_ignore', 'mobile_categories_v1_ignore', 'related_super_feed_show', 'mobile_baby_category_signup_flow_ignore', 'mobile_checkout_offer_v2_control', 'mobile_minimum_notification_interval_ignore', 'mobile_show_tabbed_feed_existing_user_ignore', 'mobile_cart_fake_only_x_left_show', 'late_shipment_apology_v2_ignore', 'mobile_show_tabbed_feed_new_user_ignore'], '_app_type': 0, 'impression_feed_category': None, '_client': 'web', '_refer_url': None, 'sort': 'recommended', '_user_agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36', '_arguments': {}, '_currency': 'CAD', '_protocol': 'http', 'offset': 0, '_method': 'GET', 'count': 40, '_locale': 'en', '_timestamp': 1401996333, '_bsid': '979b5fbcad4f4fdbb1477ae7ba8ed123', '_is_cached': False, '_version': None, '_response_status': 200, 'filter': 'all', '_response_time': 0.2887430191040039, '_uri': '/', '_remote_ip': None, '_is_user_pending': False, '_id': '1e6135e3d2eb4214afdbd99456d71183'}

A feed request…

{'products_shown': '...','feed_category': null,'sort': 'recommended','filter': 'all','offset': 0,'count': 40,

'_uid': '4eb34609ff60007c0', '_client': 'web','_country_code': 'CA',

'_id': '1e6135e3d9456d7183’,'_last_id’: 'cc39edc49f2f6','_experiment_tags': [...],

'_uri': '/','_refer_url': null,'_arguments': {},'_method': 'GET','_locale': 'en','_response_status': 200}

One problem

Searching all requests ever is slow

Transaction Log{'txn_id': '5390c295e9b9bbe68b2', 'user_id': '4eb346049b9f60007c0’,

'total': 18.0, 'shipping': 2.0,

'items': [{ 'product_id': '537b42379b9e3f55f', 'qty': 1, 'price': 16.0 }] }

{“solution”: [“logging”, “aggregation”, “analysis”, “serving”]}

Centralize Logs

• Synchronously?

• Fire & forget?

• fluentd!

Architecture

App server

Wishfluentd

Aggregation serverfluent

d

Aggregation serverfluent

d

Hadoop/Hive

{“solution”: [“logging”, “aggregation”, “analysis”, “serving”]}

Hadoop & Hive

• Great for log analysis

• Arbitrary queries

• No schema design constraints

Hadoop & Hive

• Running a Hadoop cluster sucks– TreasureData’s managed Hive solution

rocks!

{“solution”: [“logging”, “aggregation”, “analysis”, “serving”]}

MongoDB!

• Analysis results → MongoDB

• Store all combinations– Unsexy, but fast– 2 TB total

Schema

{"_id": ObjectId(…), "click_id": 2, "source_page_id": 1000, "count": 20171, "timestamp": 20140601,

Schema

"gender": "Male", "client": "Android", "country": "CA", "experiment_tag": "zip_help_text-show"}

Let’s Review

MongoDB

Logs (app servers)

Fluentd

Hadoop/Hive

Tools

Who doesn’t love nifty graphs?

Dashy

• Graphing dashboard

Perimeter

• A/B test reports– Summary

tables, detailed CSVs

– See trade-offs

Analytics = faster iteration

More growth, more revenue

Analytics = faster iteration

Powered by fluentd & MongoDB

Happy Analyzing!

adam@wish.com