Putting it all together: Reﬂecting on Data and How to ...€¦ · Human-Centered Computing at...

Human-Centered Computing at University of Maryland, Baltimore County

1

HCC 729��Putting it all together: Reflecting on Data and How to Write a Useful

Usability Report��

Human Centered Design Dr. Amy Hurst Dr. Aqueasha Martin


2

Homework Critiques


3

Homework Critique

•  Pair with your UAR partner •  Discuss what you found and how you

ranked the issues identified •  Ask your partner for feedback on your

severity ratings and rankings


4

Reading Quiz

When you are done, take a break!


5

Break


6

Reading Quiz Discussion

What are the things we need to go in more depth with?


7

Lecture


8

– Competitive analysis – Direct Observation & Interviewing – Task analyses – Personas – Heuristic evaluations (internal/external)

To date you have conducted:

How do you synthesize the data you have collected so far and provide guidance for the

future?


9

Summary Report

1.  Table of Contents 2.  Introduction 3.  Executive Summary 4.  Methods 5.  Findings and recommendations 6.  Conclusion 7.  Appendices


10

2. Introduction

Brief introduction to the website or software and it’s purpose. •  This should include a brief description of the

product and its context of use. Summarize your competitive analysis

•  List related software •  Compare strengths and weaknesses to your

client’s software


11

3. Executive Summary

Should include a sentence or two for each of the following:

– Purpose of the study : The primary reasons for conducting the study. The usability goals identified for the product.

– Date of tests – Number of participants – Length of tests – Location – Major findings (Brief Overview of Results)


12

4. Methods

Sufficient information for another person to replicate the study

•  What was recorded, e.g. Errors, comments •  Entire process from intro to conclusion •  Task description (may include the actual list

in the appendix) •  Facilitation techniques •  Roles of people involved in the session •  Technical details: network connection speed,

browser version


13

4. Methods

Include information about •  Individual Sessions •  What you asked participants to do •  Explain materials used: questionnaires,

surveys


14

4. Methods

•  Participants •  How they were contacted •  How they were compensated •  Total number of participants •  Appropriate demographic data (age, gender)

–  Computer/web experience –  Occupation/major


15

5. Findings and recommendations

Information about what is and is not working The severity of the problems Specific recommendations for improvement Screen shots of application Descriptive statistics of data collected for effectiveness and efficiency Satisfaction survey results, or other summative results


16

6. Conclusion

A summary of the study, including what went well and anything that went wrong, e.g. High user cancellations Possible next steps for the design or development team


17

7. Appendices

Introductory protocol List of tasks Blank data collection instruments Summary of qualitative comments Other important information


18

Aggregating Data

When you are done, take a break!


19

Things to Consider

•  Examine data related to … •  Paths participants took •  Problems participants experienced •  Comments/Recommendations •  Answers to Questions

•  Be exact and concise


20

See What You Have

•  Make the Data Physical and Visual •  Put important quotes on sticky notes •  Draw connections across data


21

Play with the Data

Build connections and Look for Patterns •  Group similar data •  Aggregate redundant data •  Compare related data •  Name groups of data •  Compare unrelated data


22

Sort Data

•  Alphabetical •  Numerical •  Chronological •  Frequency •  Subject


23

Affinity Diagramming

General technique used in usability to deal with qualitative data – Both practice and research

Big idea: how to take lots of disorganized

data and add structure Put ideas on cards or post-its; group them


24


25


Designers and researchers –  Identify content structure –  Identify common themes in research findings

Example

–  Interview 20 people about a web site – Each person mentions 3 problems – How to figure out what the problems actually

are?


26


Participants in user studies (“card sorting”) – Help identify information architecture –  Identify content structure: main pages for a web

site, app functions – Either open (users come up with cards) or

closed (set of cards provided) – May have a fixed number of categories

•  “Place these cards in 4 piles”


27

In-Class Activity

Evaluating the Library Website


28

Generate Data

•  Split into 2 groups (10 minutes x 2) •  Go to http://library.umbc.edu/

•  1st group get into pairs and conduct a contextual inquiry of the website ( 1 person as observer, 1 person as user) and generate notes about observations

•  2nd group each person generate UARs


29


•  As a class, Use Affinity Diagramming to Aggregate Data (10 minutes)

•  Reflect on the Process (10 minutes) •  How did you group the data? •  What relationships did you find? •  Did you find anything interesting?


30

Example

HE1 AM

Text Too Small

+/- 5


31

Analyzing Data

Adapted From Reporting Usability Test Results – Usability.gov


32

Analyzing Data

•  What does this mean? What did you find? Why is this important? •  Analysis •  Summation •  Extrapolation •  Abstraction


33

Analyzing Results

•  List all critical incidents, both positive & negative

•  Rate each critical incident for severity and fixability

•  Include references back to original data •  Analyze why each difficulty occurred •  Suggest remedy or strategy for further

inquiry


34

Reflecting on the Data

What do the data tell you? – Does the UI work the way you thought it would? – Users take approaches you expected? – What, if anything, is missing? – What should be removed or changed?

Inform the next iteration & next round


35

For Next Time…


36

Project Assignment

Midterm Project Report •  Use Usability.gov report template

•  Include the following sections –  Table of Contents –  Introduction –  Executive Summary –  Methods –  Findings and recommendations –  Conclusion –  Appendices


37

Project Assignment

Midterm Project Report •  Aggregate Data from :

•  Competitive analysis •  Direct Observation & Interviewing •  Task analyses •  Personas •  Heuristic evaluations (internal/external)


38

Project Assignment

Midterm Project Report •  Analyze Data

•  Tie ratings back to data

•  Make Recommendations •  What does your data tell you? •  Clearly Identify Recommendations and Next Steps

•  Attach Materials and Raw Data


39

Reading Assignment

No Reading Assignment for Spring Break!

Putting it all together: Reﬂecting on Data and How to ...€¦ · Human-Centered Computing at...

Documents

Transcript of Putting it all together: Reﬂecting on Data and How to ...€¦ · Human-Centered Computing at...