Internet Measurement Progress Internet Measurement Progress and the Futureand the FutureAn MRC OverviewAn MRC Overview
Presented to the CNA11/26/08
MRC Background – BriefMRC Background – Brief
Tracking the Development of GuidelinesTracking the Development of Guidelines
Newspaper SpecificsNewspaper Specifics
Other MattersOther Matters
11
AgendaAgenda
2
Auditing UpdateAuditing Update
• Understanding Sources of Measurement Differences
MRC Mission StatementMRC Mission Statement
• To secure for the media industry and related users audience measurement services that are valid, reliable and effective;
• Set Standards; and
• Conduct audits to verify compliance with Standards.
4
History and Mission of the MRCHistory and Mission of the MRC
• Congressional hearings held in 1963-64• System of self-regulation established• Founded as the “Broadcast Rating Council”• Not-for-Profit entity• Voluntary process for syndicated
measurement services to:– Supply complete information to the MRC– Comply with MRC Minimum Standards– Conduct the service as represented to clients– Submit to annual audits– Pay for the audit costs (internal & external)
• Confidential process
5
Industry ParticipationIndustry Participation
• 117 Member Organizations• Media Organizations
– Broadcasters – radio and television– Cable MSOs and networks– Internet organizations– Out-of-home media– Print organizations
• Advertising Agencies • Advertisers• Trade Associations
– Voting – CAB, IAB, MMA, MPA, NAA, NAB, RAB, TVB
– Liaison (non-voting) – AAAA, ANA, ARF
6
MRC’s Internet OrientationMRC’s Internet Orientation
• Setting a Strong Foundation– Guidelines, Transparency, Education
• Auditing, validation and accreditation of each method– “Raise all boats” approach– No measurement orientation favored– Goal – enhance trust and understanding of users
• Research required in several areas– Standards application– Measurement accuracy
• MRC Internet Committee – key assessment role– Fact-based, insofar as possible– Based on CPA independent audit– Neutral but aggressive in seeking best
practices– Seeking quality and full audience
representation7
Auditing UpdateAuditing UpdateCensus and Panel…Census and Panel…
Accuracy; Consistency; Transparency
Panel Measurement StandardsPanel Measurement Standards
• Approach• Audit Status• Progress and Learning
Note: Quantcast recently requested an audit for certain census measurement methods, which we hope will be expanded to its full methodology.
9
Overall ApproachOverall Approach
• Research Assessment (Audit Committee and Measurement Service)– Focus on methodological
validity– Non-probability sample
implications– Panel maintenance– Panel turnover– Panel coverage and
accuracy• Work vs. Home, etc.
– Persons identification– Fatigue– Comparison and
Reconciliation to Validated Census-Based Measurements
• Auditing(CPA Procedures)– Explained in following
slides:• Enumeration survey• Calibration panel• Large panel• Meter testing• Back-office processing
and reporting
10
Panel Audit StatusPanel Audit Status
• comScore MM– Pre-audit complete
• Confidential
– Audit committee formed– Audit scope and research
plans defined• Presented to audit
committee
– Research Plan work in process
– Meter Evaluation – Complete
– Enumeration Process Audit – Nearing Completion
– Large Panel Work -- Initiated
• Nielsen//NetRatings– Pre-audit complete
• Shared with Internet Committee
– Audit committee formed– Audit scope and research
plans defined• Presented to audit
committee
– Much of 1st phase of research plan executed
• Presented to audit committee; follow-ups needed and planned
– Meter Evaluation – Complete
– Data Collection and Editing – Complete
– Panel Auditing – Initiated11
Progress and Learning Progress and Learning
• Both Vendors Now Fully Cooperative– Please continue to encourage them!
• We Believe Large Panel Standard Errors Can Be Estimated– Jack Knife Replication (done at NNR)
– Weighting Controls and Sample Sizes need attention
• Turnover Plan A Likely Necessity– Turnover Analysis (some work done at NNR)
• Reported Levels and Coverage– Study approach agreed upon, work in-process
12
Census (Ad-Serving) Audit Status Census (Ad-Serving) Audit Status – MRC Audits– MRC Audits• Accredited
– AOL– Atlas DMT– CNET Networks– DoubleClick (DFP, DFA)– Edmunds.com– Eyeblaster– Forbes– Google (Impressions)– MSN– Turner– Univision.com– WDIG– Weather.com– Yahoo!– 24/7 Real Media
• In-Process– Broadband Enterprises– DoubleClick (DE)– USA Today– WPNI
13
•Geography
•User Types
•Demos
UniverseUniverse
•Naming
•Aggregation
ContentContent
•Usage Types
•Applications
•Devices
BehaviorBehavior
Sources of Counting Differences: Sources of Counting Differences: Definitional IssuesDefinitional Issues
15
• Users vs. Computers• Cookie Deletion
• Latency and Ad Delivery• Auto-Refresh
• Internal Traffic• In-Session Gaps
• New Types of Usage• International Traffic
Site-CentricSite-Centric
• Panel Representation• Meter Coverage
• Capturing All Access Locations• Metering All Computers
• User Data Collection• User In-Session Identification
• Compliance• Sampling Error
User-CentricUser-Centric
• Operational Robustness• Sound Internal Controls
• Handling Ambiguous Data• Disclosures
CommonCommon
Sources of Counting Differences: Sources of Counting Differences: Potential Quality IssuesPotential Quality Issues
16
Tracking the Development of GuidelinesTracking the Development of GuidelinesKey Points…Key Points…
Accuracy; Consistency; Transparency
IAB Measurement GuidelinesIAB Measurement Guidelines
18
Impression GuidelinesImpression Guidelines
Click GuidelinesClick Guidelines
Audience GuidelinesUniques, Time Spent
Audience GuidelinesUniques, Time Spent
2004 -
2007
The IAB’s Measurement Guidelines program began in 2004 with its first impression guidelines release.
Although new technologies will always create a need for additional measurement guidelines, the IAB’s overall plan continues in motion with Click Guidelines and Audience Guidelines being finalized.
Next On the Horizon…
• Internet– Partner: IAB
• Process Addendum (2008 release)– Important for Agencies and ASP-Based Publishers
• Clicks (being finalized)• Audience Reach Measurement (being
finalized)• Internet Enabled Gaming (in-process)
• Mobile– Partners: MMA, IAB, GSMA, ABCe
Basic PrinciplesBasic Principles
• Client-initiated Counting– Rejection of server-side counting– Adoption of beacon (and 302, etc.) at client side (as
close as possible to final delivery)
• Robot and Spider Filtration– Creation of IAB/ABCe international spiders & robots
list– Behavioral filtration guidelines were specified
• Caching– HTTP header controls and random number
assignments
• Internal Traffic– IT and .com employees are focus– Segregation, if material
• Disclosures– Goal is transparency– Description of measurement methodology– Reporting tool
20
Other Impression AdditionsOther Impression Additions
• Broadband– Post cache, revenue event counting– Encourages development of additional
measurement metrics
• Rich Media– Standardize 3rd and 4th party counting –
critical area
• Rich Internet Applications– First count per existing impression
guidance– 2nd count linked with strong user activity
• If absent, default to auto-refresh counting
– Disclose “in focus” where possible
21
Click Measurement GuidelinesClick Measurement Guidelines
Key Components:• Click Measurement Definitions• Impression-Click-Referral Cycle
– Initiated click (HTTP get)– Re-directed (measured) click (HTTP re-direct)– Received click (HTTP get)– Resolved click (HTTP 200 OK)
• Filtration Procedures– Control analyses for invalid clicks– Filtration of general activity
• Auditing Guidelines• General Reporting Parameters• Description of Methodology• Click Process Schematic
22
Audience Reach Measurement Audience Reach Measurement GuidelinesGuidelines
Focus – “Uniques”• Unique Hardware
– Cookies
– Cookied Browsers
– Devices
• Unique People– Proper terms defined and will be required– Linkage to People-Sourced data required
• Time Spent and Sessions– Ancillary needs in determination of Uniques,
so defined in guidelines
23
Suggested ActionsSuggested Actions
Stay Informed:• IAB
– Emerging Guidelines– All Available at IAB U.S. Website
• MRC– Many International Organizations
Participate in MRC
Legitimize and Validate Your Own Measurements:– Evaluation and Adopt Guidelines– Seek to Reduce Counting Differences and
Validate Accuracy25
Suggested ActionsSuggested Actions
Apples to Apples• Maximize “Combined Audience” focus in
your marketing:– Seek to understand unique traffic to each
medium and duplicated users• Internal and External Research• Content Consumed
– Leverage total audience (unique and duplicated traffic – with separate customized approaches) fairly while considering ad content
• Think of content consistency as a strategy in the appropriate content locations – Frequency2 (daily and medium)
• Maximize effectiveness of ad-type linkages• Consider Emerging Trends – e.g., Strong
Video Focus in Interactive, Ajax, etc.
26
Suggested ActionsSuggested Actions
Other Items• Use appropriate preambles in your
research to allow respondents to distinguish hard-copy from Internet readership.– Internal– Ensure external audience measurement
organizations do so also
• Account for “Pushed” Internet content properly– Count upon access
27
Other MRC ActivitiesOther MRC Activities
• Reconciliation Project– Ad Serving to Publisher– Census to Sample
• New Initiatives and Testing– Internet/Print Confusion– Nielsen Internet/TV Integration – Live Test– Arbitron PPM
• Standard Errors and Variability– Promote Further Understanding– Push for Better Tools– Internet?
• Seeking to Establish Industry Dialogue– On-Line Ad-Effectiveness Studies – Ground Rules
• AAAA, IAB, etc.
29
Other ResourcesOther Resources
• MRC Website (www.mediaratingcouncil.org):
– Accreditation Status• What Products are Accredited/Audited• Products In-Process
• Status-Update – NOL, CS, Quantcast– Educational MRC Staff Papers
• POV – Non-Probability Sampling Validation
• POV – Internet Measurement Alternatives
– Standards-Related Materials• MRC Minimum Standards
• Data Integration Guidelines
• Contact MRC if You Have Questions…Or if you’d like to get involved.
30
George W. IvieGeorge W. [email protected]
David GunzerathDavid [email protected]
Thank You!Thank You!
Top Related