Ethnography in Software Design - An Anthropologist's Perspective
-
Upload
kelly-moran -
Category
Software
-
view
694 -
download
0
Transcript of Ethnography in Software Design - An Anthropologist's Perspective
Ethnography in Software Design An Anthropologist’s Point of View ACE! Conference 2015, Krakow Poland
: https://www.flickr.com/photos/101187156@N03/14366224997/
Kelly Moran Lead Design Researcher
@Kel_Moran
Education @ CSUN
Early Influences
Uzbekistan
Masters Program @ UNT
Study Abroad @ Saudi Arabia
Worked for:
Worked with:
“Uncover user needs, Design great solutions,
and build out solutions to launch.”
“Uncover user needs, Design great solutions,
and build out solutions to launch.”
Dist
ill M
eani
ng fr
om O
bser
vatio
n
Build
the
Back
log
CONTACT BUILDING & EVOLVINGFOCUSED INNOVATION
Only a subset of these activities will be appropriate for any given project.
Qualitative Research
Quantitative Research
Affinity DiagrammingConstruct themes from qualitative data.
Analysis & Synthesis Opportunities
Ideation & Iteration
Validation & Evaluation
Approach Planning Design Research Experience Strategy & Strategic Ideation PlanningDefinition
Iden
tify
Chal
leng
es in
Con
text
Set t
he S
tage
REVEALING REALITY
Foundational Analysis
Heuristic EvaluationIdentify inital breakdowns and opportunities
Digital Marketing SWOT AnalysisIdentify strengths, weaknesses, opportunities & theats.
Stakeholder InterviewsUnderstand staheholders’ business goals & strategy.
Technical Organization Capability AanalysisAnalyze existing skills and toolsets.
Scenario-based System WalkthroughsDemo of existing solution.
Metrics EvaluationEstablish quantitative baseline of engagement & conversion data.
Content InventoryCatalog the content of the site or application.
Competitive AnalysisEvaluate competitors and comparables on specific axes.
User Experience
Data
Technology
Enterprise Architecture Capability AssesmentReview existing enterprise technology infrastructure.
Solution(s) Architecture AssessmentReview existing application(s) architecture.
RFP RequestDeliver proposal to defined scope document and existing requirements.
Project ApproachAsses a possible project and plan high level approach.
Align & Assess WorkshopAssess readiness across core capabilities.
Backlog GroomingReprioritize backlog, add new stories.
Zero Feature ReleaseDemonstrate CI, automated testing, core solution setup.
High Level Technical ArchitectureDescribe high level architecture, including packaged components.
Development Infrastructure ConfigurationContinuous integration setup.
Technical Package Identification & EvaluationPerform product evaluations for package solution components.
Architecture Spikes & Proofs of ConceptProve candidate architectures via top bottom spikes.
Existing Research ReviewMarket research, website feedback, corporate strategy, etc.
Brin
g th
e So
lutio
n in
to F
ocus
Research PlanDesign activities to meet research goals.
Contextual InquiriesObserve & document user in context & environment.
User Journals & DiariesUser document their experiences over time.
Participatory DesignCo-creation explorations with users.
Card SortExplore users’ mental models for content and labeling.
SurveysSolicit structured feedback from users.
User Workflow ModelingVisually document workflows & work systems.
Ideation WorkshopsImmerse stakeholders in data and brainstorm opportunities.
Persona DevelopmentCreate customer types to document observed behaviors and values.
Consolidated Workflow DiagramAggregate individual user workflows into one diagram.
Current Journey MapVisualize the user’s perspective of the current experience.
Quantitative Data VisualizationPresent quantitative data visually.
Opportunities Generation & EvaluationOpportunities & prototype choice.
Opportunities MatrixPrioritize in three dimensions, including user experience impact.
Marketing OpportunitiesStrategic planning of owned, earned & paid online tactics.
Engagement PlanDevelop the strategic and tactical plan to achieve the client’s goal.
Experience-Driven RoadmapPlan how great UX can be achieved through the design.
Design PrincipalsArticulate design principals to guide the design and development process.
Requirements & User Stories DefinitionWrite user stories based on detailed user scenarios.
Application & Navigation FrameworkValidated navigation and framework.
Wireframed Key WorkflowsValidated wireframes of key workflows.
Visual Design LanguageVisual design language defined.
Information ArchitectureMap the product from the users’ point of view.
StoryboardsIllustrate graphical representations of scenarios.
Qualitative & Quantitative Data SynthesisAnalyze validation data..
User ScenariosWrite detailed narratives for user experience flows.
Future Journey MapVisualize the user’s future, improved experience.
Concept ValidationValidate design prototypes through user feedback.
Application & Navigation Framework ConceptsCreate models for the navigation & framework of the application or site.
Workflow ConceptsDraw high-level wireframes for key workflows.
Visual ExplorationExplore different visual treatments and styles of the application or site.
KANO Feature PrioritizationPrioritize features with users through KANO analysis.
we make software make sense.TM
Heuristic Evaluation Findings and themes from the research synthesis Personas Consolidated Workflow Diagram Journey Map Stakeholder workshop: Findings presentation and Opportunities brainstorm Experience Principles Opportunities generation and consolidation Documentation/Data Review Identify participants, begin coordinating recruiting Scenario-based Product Walkthroughs Stakeholder Interviews SME Consultations Heuristic Evaluation Revealing Reality Continue contextual inquiry (CI) participant scheduling CI protocol development Workspace Observations
“Design Research directs the strategy & design for
our software projects”
WHAT DOES ANTHROPOLOGY
LOOK LIKE?
http://michaelvhurley.com/2013/01/04/my-worthless-degree/
Anthropologist?
CSI Vegas?
Me!
“Anthropology demands the open-mindedness with which one
must look and listen, record in astonishment, and wonder that which one would not have been
able to guess" – Margaret Mead
WHAT IS ETHNOGRAPHY?
10 things to know
1
Ethnography is the descriptive study of people and cultures
@Kel_Moran
2
Ethnographic Research is…
An approach not a specific method
@Kel_Moran
3
Ethnographic Research…
Favors qualitative over quantitative - the ethnographer
seeks to provide “thick” description
@Kel_Moran
;-}
;-/
;-)
4
Ethnographic Research is…
Conducted in context, typically over an extended period of time, and holistic, seeking the wider
picture@Kel_Moran
To understand the pen, you must understand the paper.
5
Ethnographic Research is…
Systematically (purposefully) conducted, but is responsive to
emerging trends and themes (flexible).
@Kel_Moran
6
Ethnographic Research…
Utilizes key informants who can act as guides and help provide
access to the community
(Also offer “reality checks”)@Kel_Moran
7
Ethnographic Research…
Seeks out the insider (emic) perspective and layers in outsider
(etic) insights and interpretations
@Kel_Moran
8
Ethnographic Research is…
Generative - done to discover new information, not to test
existing hypotheses
@Kel_Moran
“Never theorize before you have data. Invariably you end up
twisting facts to suit theories instead of theories to suit facts.”
- Sherlock Holmes (Arthur Conan Doyle)
9
Ethnographic Research…
Seeks to tease out the implicit, not typically stated, features of a group
“Making the familiar strange and the strange familiar”
@Kel_Moran
The Nacirema
The NaciremaAmerican
Ethnographic Research…
Is inclusive, typically including a participatory component
10
@Kel_Moran
BONUS!
Bronislaw Malinowski was a Polish born Anthropologist who pioneered extended fieldwork.
@Kel_Moran
WHAT IS DESIGN ETHNOGRAPHY?
3 things to know
1Design Ethnography…
Is a way “…to increase the success probability of a new product or service or, more appropriately, to reduce the
probability of failure specifically due to a lack of understanding the basic
behaviors and frameworks of consumers.”
Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,” Design Management Journal (pp. 35-41). p.37 @Kel_Moran
1Design Ethnography…
Is a way “…to increase the success probability of a new product or service or, more appropriately, to reduce the
probability of failure specifically due to a lack of understanding the basic
behaviors and frameworks of consumers.”
Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,” Design Management Journal (pp. 35-41). p.37 @Kel_Moran
2
Avoids an over-reliance on self-reported data
(what they say is not always what they do)
@Kel_Moran
3
Qualitative research, done in the context (environment) of the intended users, seeking to discover and understand their problems from their viewpoint,
with the designer’s viewpoint used to ideate potential solutions.
@Kel_Moran
INTEGRATINGIN-CONTEXT RESEARCH .
Discovery Phase • Designer pairs with a Researcher • Researcher leads with a background in the social
sciences • Designer assists
Researcher Designer
Design Phase • Roles switch, and the Researcher assists the Designer
ResearcherDesigner
After Detailed Design • Designer typically is embedded into the Development
Phase • Researcher comes back on board for user testing
DesignerDeveloper x N
Researcher
Discovery Design Build
Provides continuity and keeps the user’s voice present
User Focused Innovation
CASE STUDY
ENTERPRISE SOFTWARE ACCOUNT RECONCILIATIONS
The Client and the Product
Vendor of accounting software • Customer feedback of “too many clicks” and “hard to use” • Sales were lagging
Used in organizations with large, multi-functional accounting departments • Needed to be customizable • Should fit within a suite of other enterprise products
Research Basics
Who • 19 users at 6 user sites, plus 3 users inside the client company
• = 22 total observed users • Both primary reconcilers (doers) and reviewers (managers)
Where • 7 locations across 4 states • 2-4 users at each location
From 22 users: Around 800 unique insights and observations
were recorded
IN-CONTEXT OBSERVATION
“We have everything set up for you in a conference room.”
What do you see?
Contextual Learnings
• 5 out of 6 of the client-user groups observed used two monitors
• adding machines (calculators) still in use • typically in either a cube or an open workspace
• = lots of noise and movement • users “get into a zone” to focus on their work
• paper everywhere • one user had multiple post-it notes with the same number
for customer support displayed near the phone • User observed walking to a locked room (with a broken ankle)
to look up reference numbers
CONTEXTUAL LESSONS FROM USERS
Contextual Learnings
• With a lot on their minds, Excel is king • High cognitive load • High use of Excel quick keys • Importing data of multiple types into Excel as images • Highlighting and/or circling important data
http://icons8.com
“It’s better than having it rejected back.”
Contextual Learnings (continued)
• Attachments and cover sheets • Manager preference for a single attachment • Printing out documents from multiple sources to scan
them into one attachment • Making a Custom “cover sheet” in Excel to sum up the
work
“I look at [the product] as basically just a holding station.”
Contextual Learnings (continued)
• “Roles” as defined in the software did not match the work-based roles of the users
• In several locations a “reconciler” also held the role of “administrator,” but juggling these in the software was cumbersome
• The administrator role could become overwhelming
Assumed User Role with Linear Workflow
Observed Interrupted User/Admin Workflow
StartAdministrative Work
Start EndUser Work
EndUser WorkUser Work
Interruption
“I have to change gear…to move to that other role.”
“Soon I won’t have any accounting work. It will all be
[administrative] work.”
Contextual Learnings (continued)
• The problem goes beyond the system • Lack of technical support • The “real work” was done outside the system before we
arrived.
EXTENDED RESEARCH FOLLOW-UP RETROSPECTIVES
Understanding What We Didn’t See
• Visited 4 new groups • Plus an internal
check with the client’s accounting department
• Increased our overall reach
• Used a custom-built research activity
• Helped us understand how the users view their work stream, and where the software fits in
Now what do we do with all that data?
Synthesizing the Data - Major Insights1. Learning, training, & support 2. Don’t make it hard on us 3. We know who we are and
what we need 4. We need an agile, smart
workflow and tracking system 5. Don’t add to our cognitive
load 6. Be our partner 7. We have a lot going on
besides what you do for us 8. Collaboration, teamwork,
communication, & working with others
Insights Lead to Concepts
TRANSITIONING INTO DESIGN
Concepts Lead to Design• Organizing the system’s
workstream to match the user’s view.
• Renaming the “account home page” the “cover sheet” and emphasizing its similarities to the Excel-made coversheets.
• Bringing in more Excel patterns
• Users viewed the system as supplementary, and part of a larger process.
• Cover sheets were created and standardized by each accounting group.
• Observed high use of Excel.
Concepts Lead to Design (continued)
• Making both uploading and viewing attachments easier and more consistent with the rest of the UI.
• Overall more consistent UI
• Putting the information the users needed most to keep in mind in a prominent, and persistent, place on every screen.
• Uploading and viewing attachments was a common problem; and it didn’t match the rest of the experience.
• Inconsistent UI was jarring to navigate.
• Users had to scan the page to find the most relevant information on each screen.
VALIDATION TESTING
Three Pages Tested - Two Versions Each (revised and re-tested for a total of three testing rounds)
• Reconciler Dashboard • List Page - results of search
• Refined filtering • Details page - account home; i.e. cover sheet
User-Led Validation and Changes
• Most users don’t need the 6-currency view panel • Changed the default to 3 currencies
• Accountants need to be sure their entries save • Ties back to the generative research • Added a “save” confirmation button instead of auto-saving
• “Build-a-filter” concept performed better than the “expose all filters” option
• Attaching files to the cover page was validated as preferable to using a separate attachments page
BRINGING ETHNOGRAPHY TO YOUR PROJECTS .
Start Small - Start Anywhere!
Does not need to cover 7 regions across a continent (seeing a few users makes a difference)
Get into the environment (try Skype if needed)
Don’t go in with solutions in mind (don’t be too sure you know the problems already either)
@Kel_Moran
How to ObserveThe physical environment - open? cramped? hot? cold?
Lighting, noise - pleasant? distracting? manipulated by the user?
People - who interacts with whom?
Artifacts - equipment, paper notes, binders…
Document it all - notes, yes, but also photographs and audio/video if permitted.
@Kel_Moran
How to Ask QuestionsBe respectful, but don’t be shy - they have a job to do, and so do you
You are not an expert in their work/play - even if you think you are
Rephrase what they say and ask if you got it right - let them correct you even if you know you did
Avoid leading questions - ask them to describe instead
Take note of their ideas and ask “What problem does this solve?” - they’re not the experts at finding solutions, but they tried their best
@Kel_Moran
Honor the Idea of Reciprocity
Your users (research participants) are giving you something, so be sure to give something back.
@Kel_Moran
RECOMMENDED READING
: https://www.flickr.com/photos/101187156@N03/14366224997/
Kelly Moran Lead Design Researcher
Thank You
@Kel_Moran
SUGGESTED RESOURCES
How Tos Brian A. Hoey. "A Simple Introduction to the Practice of Ethnography and Guide to Ethnographic Fieldnotes" Marshall University Digital Scholar (2014): 1-10.Available at: http://works.bepress.com/brian_hoey/12
Methods of Discovery a Guide to Research Writing http://methodsofdiscovery.net/?q=node/19
Ethnography https://www.academia.edu/1022047/Picken_F._2009_What_is_Ethnography_in_M._Walter_Ed_Social_Research_Methods_Melbourne_OUP
What is Ethnography http://www.cusag.umd.edu/documents/workingpapers/epiontattrib.pdf
Articles Horace Miner’s (1956) article “Body Ritual among the Nacirema” Available at: https://www.msu.edu/~jdowell/miner.html or at: http://www.sfu.ca/~palys/Miner-1956-BodyRitualAmongTheNacirema.pdf