Defining A Product Vision

Defining a Product Vision using

Remote Research Methodologies presented to

The 14th AnnualUsability Professionals’ Association

ConferenceJuly 1st, 2005

Mark SafireDirector of User Experience Research

Sachs Insights

Presentation• Who am I• The Products & The Vision

• The Company – NPD Group • The Software Products – Solution Folders & PowerView• The User Profiles• The Research Challenge• Remote Moderated Usability

» Definition; Benefits; Challenges» Audience Participation – Experiences with Remote Research

• The Transformation of PowerView through Research• Research Road Map• PowerView Research

» Phase 1: Remote Contextual Inquiry» Phase 2: Remote Moderated Usability» The Upshot

• Q&A

Who Am I?

• Mark Safire• Director of User Experience Research,

Sachs Insights» Qualitative research provider for NPD Group online/offline

software products since 2002.

• Spoken at UPA 2003 (Presentation) and UPA 2004 (Idea Market) on Remote Moderated Usability.

» Also at NYC and Dallas UPA chapters, and for Nielsen Norman Group conferences in Las Vegas and Amsterdam.

• 2003 Presentation is online at www.sachsinsights.com/services/remotemoderatedusability.html

Qualitative research consultancy founded in 1987

• Serve automotive, fashion, financial services, media, pharmaceutical, technology and travel industries.

Practice Areas Include: Brand & Advertising Research New Product & Service Development User Experience Research

In our User Experience practice, we conduct:• Concept & Process Focus Groups• Taxonomy & Nomenclature Focus Groups• Usability• Contextual Inquiry

Sachs Insights

The Company

• A leading provider of sales and marketing information based on combined point-of-sale and consumer data for over a dozen major industries.

» Apparel, appliances, automotive, beauty, cellular, consumer electronics, food and beverage, foodservice, footwear, housewares, imaging, information technology, movies, music, software, toys, travel, video games, ...*

» Supplemented by consumer panel data to give a more complete picture – demographics, etc.

» Data from “over 230 retail partners, representing over 90,000 doors worldwide”*

• Clients: Manufacturers, Retailers, Analysts• Quoted in media regularly

*Source: www.npd.com

The Software Products

• Goals of Software• Allow customers to analyze data that NPD has collected and

organized.

• Usage Scenarios• Monthly reports: Same reports, different numbers• Ad-hoc queries: Different reports each time

• Sample Queries for Reports • What is my market share vs. my competitors?• How are inkjet printer sales trending over the past 3 mo.’s?• What were the top ten products in $ sold last month?

• Solution Folders • PowerView

Solution Folders

Make Reports from Templates

PowerView

Make Reports from Scratch

FlexibilityLess

Shorter

Little (Online)

Room for Error

Learning Curve

Setup Required

Longer

A Lot (Offline)

More Flexible

EasierTo Learn

Less Flexible

Harder To Learn

PowerView

SolutionFolders

The User Profiles – Ingoing Assumptions

• Disseminators• Market Research experts – the “gatekeepers”• Experienced in performing quantitative analysis• Pull, analyze, develop reports for others

• Users• Support personnel, bus. analysts & owners, sales & mktg.• Less experienced in performing quantitative analysis• Pull, analyze, developer reports for themselves

• Viewers• Bus. analysts & owners, sales & mktg. • Less experienced in performing quantitative analysis• Review reports created by others

Research Challenge – Finding Respondents

• Some had low incidence• “User” profile not yet a significant percentage.

• Geographical dispersion• Key retailers not clustered• Expensive and time consuming to fly them in

• 2nd Round of Research – Timing• Week before Christmas

Solution: Remote Moderated Usability

Usability in a Lab

Respondent Moderator Observer(s)

Remote Moderated Usability

Respondent Moderator Observer(s)

Remote Moderated Usability• Differences from In-Lab Usability

• Different Locations• Software Tool• Teleconference• Observers usually apart from Moderator

Remote Moderated Usability• Benefits

• Recruiting benefits» Reach geographically dispersed populations» Reach hard to recruit populations – don’t leave home/office

• Avoid cost and inconvenience of travel for all parties involved.

• Get “buy-in” from stakeholders in distant locations

NPD Research Conditions:• All had access to high-speed Internet connections• Participants & their companies were open to the technology• NPD Group had a WebEx license and in-house expertise,

used primarily for remote demos and help desk support

Remote Moderated Usability• Challenges

• Body Language

• Rapport• Interruptions (phone, visitors, email, IM, …)• Logistics (time zones, ...)• Security & Privacy • Observer dynamics• Broadband connection recommended• Not for newbies & less coordinated

Audience Participation

• Any Experiences? 1-2 Min. Summary:• Your Name, Title, & Company• Research Description: Industry, (Client),

Audience, & Objective• Why Remote Moderated Usability?• Challenges you faced & Tips for dealing

with them

Remote Moderated Usability Tools & Methods

• Many tools available• NPD done with WebEx Meeting Manager

• Others commercial apps/ASPs to consider:» IBM Lotus Web Conferencing (f.k.a. IBM Lotus Same Time)» Microsoft LiveMeeting (f.k.a. Placeware)» Citrix GoToMeeting (f.k.a. DesktopStreaming)» Macromedia Breeze

• Other free or open source solutions to consider:» Microsoft Messenger/MSN Messenger (f.k.a. Microsoft NetMeeting)» VNC (open source – various implementations, including RealVNC)

• Others kinds of tools» Full-time Remote Control Apps (See pcmag.com)

• pcAnywhere, GoToMyPC, Timbuktu, etc.

Research Roadmap

• Solution Folders• Initial Launch

» In-Lab Interviews

• Optimize Navigation» Remote Moderated

Usability

• PowerView• Needs Assessment

» Internal Focus Groups

» Remote Contextual Inquiry

• Prototype before Relaunch (version 5.0)

» Remote Moderated Usability

PowerView Research

Phase I

Remote Contextual Inquiry

• How do they use it?• They define & demo tasks.

• Does it meet actual needs?• See Role in Work Process

» Ultimate reports delivered

• How easy is it to use?• First Use / Experienced Use

• Things to change /Things NOT to change

Phase 2

Remote Moderated Usability

• Can they use it?• Use their pre-defined tasks

• Do fixes meet needs?• Role-play work process

• Observe ease of use.• First Use / Experienced Use

• Points of pain improved? / Have to relearn a lot?

Phase 1, Remote Contextual Inquiry (RCI)

• First time & “Ad Hoc use”• Even old-timers stumble when trying to create new reports.

• Drag & drop interface – good idea, don’t change!

• Inscrutable 8 character field names

• Hard to envision report layout from interface» Especially for “nested” reports

• Expert & Repeated use• Gargantuan monthly reports: Same thing, different numbers

» Each report is run individually, not as a “batch”» Typically formatted each time in Excel – not saved in tool.

• Often requires specific colors for company and competitors in for tables and charts.

• Extra columns created during export to show “suppression.”

Everyone: Look & Feel “dated” – want standard Windows conventions

Phase 2, Remote Moderated Usability (RMU)

• First time & “Ad Hoc use”• Drag & drop approach still used.• Comprehensible, full text, categorized field names

» But can change back to 8 char. abbrev’s in preferences...

• Preview of report layout

• Expert & Repeated use• Run as a batch

» But “Favorites” menuv – confusing metaphor

» Final: “MyReports”

• Save formatting • No extra columns!

» gray or red italicized numbers with asterisks

PowerView, Phase 1, RCI

2) Inscrutable 8 character field names, without any categorization

1) Drag & drop a good idea: don’t change!

3) Hard to envision report layout from interface

4) Look & Feel “dated” – not standard Windows conventions.

Widget Production Quarterly Trend

What they need

Q1 ’04 Q2 ’04 Q3 ’04 Q4 ‘04

Us 12.3 16.2 27.7 29.5

Them 32.0 28.1 20.1 18.2

Channel 70.1 89.3 76.2 88.4

Q1 ’04 Q2 ’04 Q3 ’04 Q4 ‘04

Channel

What they get

Q1 ’04 Q2 ’04 Q3 ’04 Q4 ‘04

Us 20.4 27.4 90 20.4

Them 30.6 38.6 34.6 31.6

Channel 45.9 46.9 45 43.9

5) Extra columns

6) Formatting not saved

PowerView, Phase 2: RMU1) Drag & drop not changed!

2) Full text, categorized field names

3) Preview of Report Layout

4) Overall Look & Feel is Contemporary

PowerView, Phase 2: RMU

5) No extra columns 6) Formatting can be saved

PowerView Version 5.0

• Validation• Changes in Prototype were well received.

• Tweaks• “My Favorites” was not an intuitive menu name

» Worked for Solution Folders, but not here. Since PowerView is not a Web application, the “My Favorites” web metaphor was not effective.

» Solution: Changed menu name to MyReports for launch.

The Upshot

More Flexible

EasierTo Learn

Less Flexible

Harder To Learn

PowerViewSolutionFolders

Contact Information

Mark Safire

Director of User Experience Research

Sachs Insights

200 Varick Street, Suite 500

New York, NY 10014

www.sachsinsights.com

msafire @sachsinsights.com

212-924-1600 x168

www.remoteusability.com

Defining A Product Vision

Technology

Transcript of Defining A Product Vision

Mn/DOT– ACEC/MN Annual Consultant Conference. Overview Background Background Defining the 50 Year Vision Defining the 50 Year Vision Mn/DOTs Family of.

Defining Tomorrow's Tourism Product: Packaging Experiences

Product vision

BRITISH PETROLEUM (A): DEFINING A STRATEGIC VISION

Larc2014 - Product & Technology Vision

Artopex Vision Product Catalog

From Vision To Product

Novell Identity Vision: Defining an Identity Fabric Technical ......Technical White Paper SECURITY AND IDENTITY Novell ® Identity Vision: Defining an Identity Fabric Part One in the

NEW VISION NEW WORLDS HIKVISION MACHINE VISION PRODUCT CATALOG · en.hikrobotics.com NEW VISION NEW WORLDS HIKVISION MACHINE VISION PRODUCT CATALOG

Vision for Imagination MACHINE VISION PRODUCT CATALOG

Defining your target product profile: Medical device product ...

Product vision board

Defining your product

PRODUCT MANAGEMENT CHAPTER 4: Defining the competitive set.

Defining the Business: Vision, Mission and Long-term Objectives

Vision - Defining the opportunity Market Research Market Testing Stuart W. Hillston

Defining your Product Strategy using UX

GS1 Solutions Collectively defining sustainability for product categories€¦ · Collectively defining sustainability for product categories An overview of global hotspot initiatives

Our Product Vision

Arecont Vision Product Catalogue