Online Audience Measurement: From Panel
Measurement to Website Analytics
Saigon Digital Marketing Conference
23rd May 2009
Joe Nguyen Vice President, Southeast Asia | comScore, Inc. ( NASDAQ:SCOR)
hp+65 9478 2100 | [email protected]
2© comScore, Inc. Proprietary and Confidential.
The web is the most measurable
medium in the history of marketing.
Now all that’s left to do is figure out
how to measure it.
The McKinsey Quarterly
“How poor metrics undermine digital marketing”
3© comScore, Inc. Proprietary and Confidential.
Before we get to the HOW, let’s ask WHY?
Typical needs…
� Track visitors to my website: Who, when, where, and what
� Track effectiveness of my campaigns: SEO, SEM, email, Banner ad,
Affiliate, branding campaigns, even TV or print
� Track success events, registrations, DOLLARS made
3
What about…
� Market & competitive knowledge
� Planning your media campaign
� Key demographic knowledge (age, sex, income)
4© comScore, Inc. Proprietary and Confidential.
Website Analytics – Site Centric, Server side
Measurement
5© comScore, Inc. Proprietary and Confidential.
Site Centric Data Collection
5
6© comScore, Inc. Proprietary and Confidential.
Key Common Metrics
� Page Views
� Visits
� Visitors (Daily and Monthly)
� Average Visits / Visitor
� Average Page Views/Visit and Average Page Views/Visitor
� Visit number / New vs. Repeat visitors
� Return Frequency
� Registrations (of various types)
6
7© comScore, Inc. Proprietary and Confidential.
Advance Reports: Countries, Region, Cities
7
8© comScore, Inc. Proprietary and Confidential.
Advance Reports: Search Engine Performance
8
9© comScore, Inc. Proprietary and Confidential.
Advance Reports: Search Engine Performance
9
10© comScore, Inc. Proprietary and Confidential.
Advance Reports: Channel Performance
10
11© comScore, Inc. Proprietary and Confidential.
Advance Reports: Pathing Analysis
11
12© comScore, Inc. Proprietary and Confidential.
Advance Reports: Campaign Performance
12
13© comScore, Inc. Proprietary and Confidential.
Advance Reports: Revenue - “Show me the money!”
13
14© comScore, Inc. Proprietary and Confidential.
Advance Reports: Conversion Funnel Analysis
14
15© comScore, Inc. Proprietary and Confidential.
Dashboards!!!!
15
16© comScore, Inc. Proprietary and Confidential.
Panel Based Audience Measurement
17© comScore, Inc. Proprietary and Confidential.
How Does Panel Measurement Work?
Collect Traffic
Representative set
of individuals
Collect Traffic
Representative set
of individuals
Final reports of market data
based on projections of
individuals
Final reports of market data
based on projections of
individuals
Apply
comScore
Projection
&URL
Dictionary
Apply
comScore
Projection
&URL
Dictionary
Data ServersData ServersProjected
Traffic
Projected
Traffic
18© comScore, Inc. Proprietary and Confidential.
What We Measure
18
AUDIENCEDEMOGRAPHICS
LIFE STAGE
PC ATTRIBUTES
SEGMENTATION
LINKED DATASETS
Acxiom; Claritas/Prizm; internal CRM database, etc.
Browser: IE, AOL, Netscape, …
CONTENT VISITATION
VIDEO / RADIO STREAMING: SEARCH
COMMUNICATION
Email, IM, Mobile
Transactions
Purchase / Subscriptions;
Configurations; Price
Request
Software (Microsoft. Live, Google, Yahoo)
ACTIONS
PC LOCATION:
Home; Work (Corporation, Laptop); University
INTERNATIONAL:
US/Canada
Europe
Asia
Latin America
Language Regions
WORLDWIDE
LOCATIONADVERTISING:
Display, Streaming, search
Reach/Frequecy
Tracking & Exposure
Effectiveness
Sales Results/clickthrough/referrals
ATTITUDINAL:
Awareness, preference, customer sat
IMPACT
19© comScore, Inc. Proprietary and Confidential.
19
37 Countries Processed and Reported Separately…
20© comScore, Inc. Proprietary and Confidential.
20
…but data collected from more than 171 countries
21© comScore, Inc. Proprietary and Confidential.
comScore Media Metrix Methodology: the Building Blocks
How do we establish the universe?
How do we acquire respondents?
How do we collect the raw data?
How do we know who is using the
machine?
How do we correct for bias?
(weighting and projection)
22© comScore, Inc. Proprietary and Confidential.
comScore Media Metrix Methodology: the Building Blocks
How do we establish the universe?
How do we acquire respondents?
How do we collect the raw data?
How do we know who is using the
machine?
How do we minimize bias?
(weighting and projection)
•Enumeration •Online recruitment
•The cproxy •Biometrics
•The Calibration Sample
•The Client Focused Dictionary
23© comScore, Inc. Proprietary and Confidential.
Enumeration
� Accurate estimates of the size and composition of the target populations are
critical to audience measurement. This field of research is called enumeration
research.
� comScore devotes significant effort to researching and developing these
estimates.
� Enumeration research enables comScore to accurately estimate the target
population sizes for the countries we track.
� comScore enumeration sources vary by market.
– comScore surveys: In certain countries we conduct our own surveys to estimate the
size of our target population.
– Other sources: Otherwise we use publicly available, third-party studies for our
enumeration process.
comScore Universe (Outside US and CA): Persons aged 15+ who accessed the
Internet from either a home or a work computer in the past 30 days.
24© comScore, Inc. Proprietary and Confidential.
24
Enumeration Sources for Asia Pacific Markets
� Following are the enumeration sources for countries in Asia Pacific
market, reported by comScore:
Country Enumeration Source
Australia Australian Bureau of Statistics
China China Internet Network Information Center, Intermedia
Hong Kong Annual insertion in Synovate omnibus
India Internet and Mobile Association of India
Japan Bi-annual insertion in Ipsos omnibus, n=1000
MalaysiaComputer Industry Almanac, Malaysian Commumications and Multimedia Commission
survey
New Zealand Stats New Zealand, Computer Industry Almanac
Singapore Singapore Infocomm Develoment Authority
South Korea Insertions in Ipsos omnibus through 2007, National Internet Development Agency of Korea
Taiwan Insertion in TNS omnibus 2007, Taiwan Network Information Center
The Enumeration sources might change as comScore continuously monitors various sources
available in a market to identify the most reliable source for universe estimates.
25© comScore, Inc. Proprietary and Confidential.
25
Recruitment
� The goal of comScore recruitment
activities is to recruit a representative
sample from the target universe
� Broad set of value propositions to
appear to the diverse online
population
� comScore recruits panelists
exclusively online, in two ways:
– The Affiliate Program (“banners-plus-
Permission Research”)
– Third Party Application Providers (TAP)
26© comScore, Inc. Proprietary and Confidential.
26
The Affiliate Program
� Affiliate program
– Affiliate network comprised of web
entities which meet comScore's
quality criteria
– On these sites, panelists are recruited
via banner ads
– Appeals are targeted through a broad
array of smaller web entities
– Respondents are directed to our
online intake entity
www.permissionresearch.com
27© comScore, Inc. Proprietary and Confidential.
Value Propositions
A sampling of the many value propositions offered….
28© comScore, Inc. Proprietary and Confidential.
28
Privacy and Disclosure
Potential members
are informed that
they would be
joining an online
market research
panel that tracks
their online
browsing and
purchasing
behavior.
Potential members
are informed of the
company’s
commitment to
protect the privacy
of members.
29© comScore, Inc. Proprietary and Confidential.
29
Collect User Demographics; Double Opt-in
The Privacy
Statement & User
License
Agreement
informs users that
they would be
participating in
market research
program involving
passive tracking
of their online
browsing &
purchasing
behavior AND
requires their
affirmative
consent.
Registrant
provides own
demographic
information --
including birth
date
30© comScore, Inc. Proprietary and Confidential.
30
Collect Demographics of Other HH Members
After installation
of software, user
asked to provide
household
information
31© comScore, Inc. Proprietary and Confidential.
31
comScore Proprietary Collection Software
Browser
CProxy
Collection
Server
Collection
Server
HTTP
Website
HTTP
Website
HTTPS
Website
HTTPS
Website
Panelist’s PC
Streaming
Site
Streaming
Site
■ comScore captures:
– url
– Engagement (active versus passive)
– Keystroke and mouse activity and intensity
– Information passing between the user and the
Internet entity
– Application installation and usage (Excel, Word
etc.)
■ Data capture tiers
– HTTP: comScore can capture just about
anything over the HTTP protocol (including
HTTP and HTTPS)
– Proprietary Technology for real time
measurement of advanced protocols like
streaming
– Measurement of AOL proprietary and IM
engagement
comScore’s cproxy software agent allows us to “see” user activity at the machine-
or screen-side (user experience as opposed to site-centric)
32© comScore, Inc. Proprietary and Confidential.
32
Who’s Using the Machine?
� In the digital age, tracking the behavior of machines is
easier than ever before
� Moving from machines (who don’t buy products or view
advertisements) to people still poses unique challenges
� In panel measurement, the ideal is passive observation
over an extended time period
� One approach: use “who are you?” pop-ups
– Requiring respondents to self-identify each user session
(similar to traditional media construct used in TV People
Meter)
– Intrusive
– Fatiguing
– Biasing
� Intrusiveness drives turnover to unmanageable levels
33© comScore, Inc. Proprietary and Confidential.
33
Passive Biometric Observation
� comScore develops a unique biometric signature for each panelist
– Respondents passively self-identify through their online behavior
Fill out forms, email/account IDs, etc.
– User Demographic Reporting (UDR): a proprietary, patented technology
that creates biometric “fingerprints” for each person using the machine
Based on keyboard and (with UDR v2) mouse behaviors
� UDR is used to passively identify users
– For a given session, UDR matches the user’s biometric signature to stored
signatures for each machine user
� Machines do not make Media Metrix In-tab until behavior can be attributed
from the beginning of the reporting month
– Media Metrix qualification rules require that at least 90% of a machine’s
usage be attributed to a person for that machine to make monthly in-tab
Activity not attributed to a user will cause the machine to fall out of in-tab for the reporting
month
34© comScore, Inc. Proprietary and Confidential.
Persons Under Measurement (PUM)
� comScore PUM is persons with at least one computer being actively
measured by comScore within the previous 30 day period.
34
Country People Under Measurement
(PUM)
Australia 30,000
China 50,000
Hong Kong 5,000
India 100,000
Japan 10,000
Malaysia 30,000
New Zealand 5,000
Singapore 10,000
South Korea 5,000
Taiwan 5,000
Above are the PUM numbers for Asia Pacific Market
35© comScore, Inc. Proprietary and Confidential.
35
Stratification and Projection
Consistent set of stratification variables:
■ Age
■ Gender
■ Duration Categories
■ Anglophone Cap*
*In certain markets where English is not the primary language, Anglophone Caps are applied to minimize the bias in
weights induced by panelists with a Heavy English bias in their browsing behavior.
36© comScore, Inc. Proprietary and Confidential.
Client Focused
Dictionary (CFD)
Client Focused
Dictionary (CFD)
Proprietary and Confidential Do not distribute without written permission from comScore
comScore’s Client Focused Dictionary
110+ Categories & Sub-
Categories
110+ Categories & Sub-
Categories6-tier reporting structure6-tier reporting structure
PROPERTYPROPERTY
MEDIA TITLEMEDIA TITLE
CHANNELCHANNEL
SUB-CHANNELSUB-CHANNEL
GROUPGROUP
SUB-GROUPSUB-GROUP
TelecomsTelecoms
TravelTravel
FinanceFinance
AutomotiveAutomotive
Ad-NetworksAd-Networks
etcetc
Time WarnerTime Warner
AOLAOL
AOL Ent.AOL Ent.
AOL MusicAOL Music
AOL RadioAOL Radio
AOL Radio
Streaming URL’s
AOL Radio
Streaming URL’s
37© comScore, Inc. Proprietary and Confidential.
Core Reports
• Key Measures
• Demographic Profile
• Digital Calculator
• Source/Loss
• Media Trend
• % Change Media Trend
• Target Trend
• % Change Target Trend
• Cross Visiting
• Instant Messenger Report
comScore Media Metrix Core Reports
AUDIENCE MEASUREMENT
& MEDIA PLANNING
Summary Reports
• Monthly Flash
• Media Dashboard
• Dictionary Definition
Other Tools
• Reach & Frequency
• Brand Metrix
• qSearch
• Segment Metrix
• Mobile Metrix (1H ‘09)
38© comScore, Inc. Proprietary and Confidential.
Regional Overview
38
Audience: All Persons 15+ at Home/Work
Locations
Total Internet
Total
Unique
Visitors
(000)
Total
Minutes
(MM)
Total
Pages
Viewed
(MM)
Average
Usage
Days per
Visitor
Average
Minutes
per
Visitor
Average
Pages per
Visitor
Total Visits
(000)
Average
Minutes
per Visit
Average
Visits
per
Visitor
Asia Pacific 446,496 446,961 751,681 12.0 1,001.0 1,684 16,360,948 27.3 36.6
Europe 312,493 441,968 781,528 15.7 1,414.3 2,501 14,899,425 29.7 47.7
North America 187,679 386,548 551,209 22.7 2,059.6 2,937 13,468,197 28.7 71.8
Latin America 83,912 129,119 163,261 13.4 1,538.7 1,946 3,513,594 36.7 41.9
Middle East - Africa 77,451 127,325 145,687 13.3 1,643.9 1,881 3,485,231 36.5 45.0
39© comScore, Inc. Proprietary and Confidential.
Asia Pacific – Top Properties Report
39
Audience: All Persons 15+ at Asia-Pac Home/Work
Locations
Total
Unique
Visitors
(000)
% Reach
Total
Minutes
(MM)
Total
Pages
Viewed
(MM)
Average
Minutes
per
Visitor
Average
Pages
per
Visitor
Total
Visits
(000)
Average
Minutes
per Visit
Average
Visits
per
Visitor
Total Internet : Total Audience 446,496 100.0 446,961 751,681 1,001.0 1,684 16,360,948 27.3 36.6
Top 100 Properties N/A N/A ... ... ... ... ... ... ...
Google Sites 275,085 61.6 30,383 49,714 110.5 181 4,557,566 6.7 16.6
Yahoo! Sites 227,250 50.9 47,350 49,241 208.4 217 4,081,676 11.6 18.0
Microsoft Sites 205,488 46.0 36,923 14,085 179.7 69 2,189,382 16.9 10.7
TENCENT Inc. 153,458 34.4 26,820 33,966 174.8 221 3,618,394 7.4 23.6
Baidu.com Inc. 146,483 32.8 8,294 19,366 56.6 132 1,711,533 4.8 11.7
SINA Corporation 109,398 24.5 5,061 5,938 46.3 54 855,398 5.9 7.8
Sohu.com Inc. 79,484 17.8 3,594 4,893 45.2 62 463,551 7.8 5.8
Wikimedia Foundation Sites 76,024 17.0 901 905 11.9 12 272,434 3.3 3.6
Alibaba.com Corporation 75,471 16.9 4,506 10,187 59.7 135 469,810 9.6 6.2
NetEase.com Inc. 74,981 16.8 3,375 5,242 45.0 70 587,640 5.7 7.8
Xunlei Networking 68,206 15.3 3,730 3,911 54.7 57 404,005 9.2 5.9
CBS Interactive 67,055 15.0 610 999 9.1 15 146,185 4.2 2.2
FACEBOOK.COM 55,960 12.5 8,717 11,381 155.8 203 856,153 10.2 15.3
Tudou Sites 55,069 12.3 1,501 1,371 27.3 25 244,442 6.1 4.4
Youku 53,085 11.9 2,463 1,965 46.4 37 254,507 9.7 4.8
Amazon Sites 52,431 11.7 475 927 9.1 18 151,141 3.1 2.9
Apple Inc. 44,836 10.0 241 315 5.4 7 77,470 3.1 1.7
eBay 42,742 9.6 1,394 2,586 32.6 60 130,348 10.7 3.1
AOL LLC 40,609 9.1 832 1,393 20.5 34 129,218 6.4 3.2
Rakuten Inc 37,636 8.4 1,488 3,204 39.5 85 269,904 5.5 7.2
40© comScore, Inc. Proprietary and Confidential.
Top Vietnamese Site for Asia Pac Users
40
41© comScore, Inc. Proprietary and Confidential.
Key Takeaways
42© comScore, Inc. Proprietary and Confidential.
Differences
Panel Server Side
� Person demographics
� 360-degree view of customer
� Better duration and usage
measurement
� UVs not affected by bots or cookie
deletion
� Cross-site usage
� Audience overlap
� Difficult to game
� However:
� Universe limited to H&W
� Niche audiences difficult to
measure
� Census: Visibility into total universe
tonnage (outside panel universe)
� More timely reporting
� However:
• Requires industry participation
• No person demographics
• Affected by bot traffic
• UV measurement inaccurate due
to cookie blocking or deletion
• No standardization around page
counting
43© comScore, Inc. Proprietary and Confidential.
Different needs
Panel Audience Measurement
� Methodology: Recruits
representative users to be on panel
and continually tracks & reports
online behavior
� Primary Uses: Understanding the
MARKET, media planning,
demographic profile
� Providers: comScore, Nielsen
Site Centric Analytics
� Methodology: Tools/codes on
publishers’ site that tracks visitors’
behavior as they navigate through
the site.
� Primary Uses: Understanding YOUR
visitors behavior, Site optimization,
campaign tracking and reporting
� Providers: Omniture, Google
Analytics & many, many others
43
44© comScore, Inc. Proprietary and Confidential.
Thank You
Joe Nguyen Vice President, Southeast Asia | comScore, Inc. ( NASDAQ:SCOR)
hp+65 9478 2100 | [email protected]
Understanding Online Measurement: From Panel
Audience Measure to Website Analytics
Saigon Digital Marketing Conference
23rd May 2009
Joe Nguyen Vice President, Southeast Asia | comScore, Inc. ( NASDAQ:SCOR)
hp+65 9478 2100 | [email protected]
Top Related