Search Analytics at Enterprise Search Summit Fall 2011

Search Analytics

What? Why? How?

Otis Gospodnetić – Sematext International@otisg ◦ @sematext ◦ sematext.com

sematext.com/search-analytics

About Otis Gospodnetić

• ASF Member: Lucene, Solr, Nutch, Mahout

• Author: Lucene in Action 1 & 2

• Entrepreneur: Sematext, Simpy

Sematext Metrics

100% organic: no GMO, no VC 4 years old < 10 people 7 countries 3 timezones 2 continents > 100 customers

About Sematext

Products & Services

Consulting, Development, Tech Support:

Search (Lucene, Solr, ElasticSearch...) Big Data (Hadoop, HBase, Voldemort...) Web Crawling (Nutch, Droids) Machine Learning (Mahout)

Agenda

What is Search Analytics and why it matters Example reports and their value Optional: Search Analytics in the Cloud

Communication

twitter.com/sematext twitter.com/otisg hash tags: #stsa or #stanalytics http://sematext.com/search-analytics/index.html Raise your hand! otis@sematext.com

searchusers

searchproviders

searchexperience

Why Oh Why

searchproviders

searchexperience

This search sucks!It takes 17 tries to find anything here!

F!?@#$%^&?!?

searchusers

Cool, the latest search tweaks made our site really sticky!

Awesome!

Fill in the Missing Piece

Search Analytics

Performance Monitoring

Quality Assurance

Tuning UI

Blind Leading the Blind

Analytics as Compass

Search logs are your Map

Search Analytics is your Compass

The Bottom Line Why

Measure and monitor everything. Supports (re)design, navigation choices Helps with content acquisition & enhancement Improve search experience Mula

The Moment of Truth

Question for the audience #1

What do you use for Search Analytics?

a) Home grown stuffb) Google Analyticsc) Omnitured) Webtrendse) Otherf ) Nothing

Search Analytics Basics

Collect: queries & clicks & interactions & ... Analyze: actions / xactions / conversions Output: reports – over time Output++: feedback loop

The means, not the goal Ongoing, not one-off

remember this

Search vs. Web Analytics

User intent and information needs vs. inferring Hand in hand Ideally you can relate data from both or even

unify it

Report Types

Failures vs. non-failures

Actionable vs. non-actionable

Trends vs. summaries

Failures vs. Non-Failures

Zero hits Low CTR Low MRR High bounce rate Low conversion rate Deep paging Deep clicking High latency

Query rate Query volume Top seen & clicked

docs Top queries Terms per query Search sessions Search users Distinct queries

Value of Failure Fixes

Re-search

Findability

Relevance Tuning

Performance Tuning

Measure, then Fix

If you can't measure, it you can't fix it!

Relevance A/B Testing

Tracking Zero Hits

Watching Latency

Search Analytics & Measuring

If you can't measure it, you can't fix it!

You can't measure it if you don't have Analytics

Actionable vs. Non-Actionable

Query rate Query volume Top seen & clicked

docs Top queries Terms per query Search sessions Search users Distinct queries

More Fixin' Query rate Query volume Search sessions Search users Top seen & clicked

docs Top queries Terms per query Distinct queries

Navigation & Design

Results Shuffling Diversification

Recommendations

AutoCompleteSearch box size

Output++: Data is Power

AutoComplete - $MM improvement Better DYM Spellchecker Related Searches Recommendations Relevance Feedback ...

Closing the Loop

searchusers

searchproviders

searchexperience

Resources

http://rosenfeldmedia.com/books/searchanalytics/

Search Analytics for Your SiteLouis Rosenfeld

Search Analytics What? Why? How?

Search Analytics with Flume and HBase

Search Analytics Business Value & NoSQL Backend

http://blog.sematext.com/tag/analytics/

Key Take-aways

Without Analytics you are blind

If you can't measure it, you can't fix it

Use Search Analytics to understand, measure and improve search

Using Search Analytics means having a competitive advantage

Time permitting:

Behind the scenes of Sematext Search Analytics

Behind the Scenes

sematext.com blog.sematext.com @sematext @otisg otis@sematext.com

Want SA? Grab me or go to: sematext.com/search-analytics

Hash tags: #stsa or #stanalytics

Contact

What We've Built

Search Analytics SaaS Numerous reports (e.g. query volume,

rate, latency, term frequencies / comparisons, hit buckets, search origins, etc.)

Trending over time Comparisons of time periods Top N reports Filter, slice and dice

Sematext Search Analytics

Big Dreams

SaaS Multitenant Large Scale – Massive Data Cloud

Storage Choices

RDBMS: MySQL, PostgreSQL HDFS Hive HBase Cassandra

SaaS vs. In-House

Question for the audience #2

SaaS vs in-house Search Analytics?

a) SaaSb) in-house

Data Flow See Search Analytics with Flume and HBase

http://blog.sematext.com/2010/10/16/search-analytics-hadoop-world-flume-hbase/

Data Collection See Search Analytics with Flume and HBase

http://blog.sematext.com/2010/10/16/search-analytics-hadoop-world-flume-hbase/

Core Tech

JavaScript Beacons Metric Capture Web App aka Receiver Flume Agents, Collectors, Sinks HBase MapReduce Aggregations Search Analytics Reporting Web App

What is Flume

Distributed data/log collection service Scalable, configurable, extensible Centrally manageable, open source

Agents get data from app, Collectors save it Abstractions: Source → Decorator(s) → Sink

What is HBase

Scalable, reliable, distributed, column-oriented DB On top of HDFS MapReducable

Data Flow, Detailed

Why Flume

Reliable delivery e.g. queue msgs locally if destination unreachable

Easy, centralized management via Web UI or console

Good community, good progress, now @ASF But: more complex, more moving parts On Flume: slideshare.net/cloudera/inside-flume Alternatives: Kafka, Scribe...

Why HBase

Scalable raw & aggregate data storage MapReduce data input Fast scans for time ranges, fast key lookups Easy storage and compute power expansion Good looking roadmap, community, progress

Open Sourcing

2 open-source projects:

github.com/sematext/HBaseWD

github.com/sematext/HBaseHUT See sematext.com/open-source/index.html

Patches for Flume and HBaseblog.sematext.com/tag/flume/

Challenges

Data size. Solutions: Compression (4-5x smaller with lzo) Data pruning (variable levels)

Query string distribution: very long-tail Lots of data to process, update, aggregate

Young tools: Flume, HBase Poor IO on EC2 Hadoop distributions

sematext.com blog.sematext.com @sematext @otisg otis@sematext.com

Want SA? Grab me or go to: sematext.com/search-analytics

Hash tags: #stsa or #stanalytics

Contact

Search Analytics at Enterprise Search Summit Fall 2011

Technology

Transcript of Search Analytics at Enterprise Search Summit Fall 2011

Hr Analytics Introduction Hr Analytics Summit Luk Smeyers 1.1

Applying Semantics to Search Text Analytics Tom Reamy Chief Knowledge Architect KAPS Group Enterprise Search Summit New York.

Search Analytics

The Financial Forecasting, Planning and Analytics Summit€¦ · THE FINANCIAL FORECASTING, PLANNING AND ANALYTICS SUMMIT 2015 This attendee focused summit is a unique blend of analytics,

Open analytics summit nyc

Text analytics summit - West 2014

Open Analytics Summit NYC - Tiger

Welcome - Text Analytics Summit 2010

Search Analytics in Practice

Social Summit: Google Analytics

Scalable Search Analytics

Search Exchange Analytics Insights

Seo Analytics Search Insights

Search Marketing Analytics

Improving site search with search analytics

Big Data & Analytics Innovation Summit

Search analytics #blogbus

Search & Analytics AI-driven Intelligent Influencer · AI-driven Intelligent Influencer Search & Analytics influenceye.com. search & analytics tool that helps marketers and agencies

Adobe Summit - Advanced Advertising Analytics

01 deloitte predictive analytics analytics summit-09-30-14_092514