The Power ofThe UX Evaluationa LOCO | UX Meetup

TONIGHT

A FEW SHORT WORDS

LOCO | UX Community MissionBuild a network of UX practitioners local to the Longmont/Boulder area for the purpose of:• Access to opportunities (seeking, hiring, colabos)• Sharing Ideas, Tools, Methods, Techniques• Building community for mutual enhancement/benefit

• each other• local businesses• Longmont municipality

Who Tweets During Meetups?

@jkooda of @liminaUX at @loco_ux

Expert Heuristic (huh?)

Evaluation

heu·ris·tic hyo͞oˈristik/

adjectiveenabling a person to discover or learn something for themselves."a “hands-on” or interactive heuristic approach to learning"

nouna heuristic process or method.

Is a usability engineering method for finding usability problems in a user interface design, thereby making them addressable and solvable.

It involves a small set of expert evaluators who examine the interface and assess its compliance with “heuristics,” or recognized usability principles.

Heuristic Evaluation

10 Usability Heuristic PrinciplesVisibility of system status

Match between system and the real world

User control and freedom

Consistency and standards

Error prevention

Recognition rather than recall

Flexibility and efficiency of use

Aesthetic and minimalist design

Helps users recognize, diagnose, and recover from errors

Help and documentation

ExamplesintheWild:http://designingwebinterfaces.com/6-tips-for-a-great-flex-ux-part-5

Source:https://www.nngroup.com/articles/ten-usability-heuristics/

Donald A. Norman

Jacob Nielson

“DISCOUNT USABILITY

ENGINEERING SERVICE”

CAVEATA heuristic evaluation should not replace usability testing.

Although the heuristics relate to criteria that affect your site’s usability, the issues identified in a heuristic evaluation are different than those found in a usability test.

PROS• Providesquickandrelativelyinexpensive(cheap)• SystematicandIntuitive,applyingasetofpredefinedrules/heuristics• Requiresverylittleplanning• Canbeusedearlyinthedesignprocess•Doesnotrequireafinishedinterface• Providescorrectivemeasures• Canbeusedtogetherwithotherusabilitytestingmethods• Canhelpidentifyandprioritizeissuesinneedoffurtherusabilitytesting

CONS• Requiresknowledgeandexperiencetoapplytheheuristicseffectively• Trainedusabilityexpertsaresometimeshardtofindandcanbeexpensive• Requiresmultipleexpertsandaggregatetheirresults• Theevaluationmayidentifymoreminorissuesandfewermajorissues• Tendstofocusonproblemsrather than solutions

1. Set Up The Team2. Agree on Evaluation Criteria

3. Conduct Cognitive Walk Through

4. Analyze Interface & Record Issues

5. Internal Review

6. Present to Client

7. … Next steps & business development

6 MAJOR PARTS

1.WHO DO YOU NEED?

PICK 3

UX LEAD USER RESEARCHER INTERACTION DESIGNER

VISUAL DESIGNER DEVELOPER CONTENT SPECIALIST

WHY 3?

Rows = 19 evaluators Column = 16 usability problems

= Problem found

Economics of the Evaluation Team

Source:https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/

Weak Evaluator

Strong Evaluator

Hard Easy

0%

25%

50%

75%

100%

1 5 10 15Number of Evaluators

Perc

ent I

ssue

s Fou

nd

Strength in Numbers

0%

10%

20%

30%

40%

50%

60%

70%

1 5 10 15

Number of Evaluators

Cos

t / B

enef

it Ra

tio

ROIEval Quants

Freak-o-nomics1.Emergence over authority – Talks about the shift in authority from the top-down to the bottom-up. Think Wikipedia.

2.Pull over push – Another example of bottom-up demand. User-driven instead of mass-marketing. Think on-demand instead of broadcast.

3.Compasses over maps – Methodology not mission. Teach a man to fish. A play for Seymour Papert’sconstructionism.

4.Risk over safety – When the cost of innovation is low, like it is now, it’s easier take more risks and try more things. Negroponte famous MediaLab motto was, “Demo or Die”, Ito says “Deploy.”

5.Disobedience over compliance – “You don’t win a Nobel Prize by doing what you’re told” ( Joi Ito). Ask forgiveness, not permission.

6.Practice over theory – “In theory there is no difference between theory and practice. In practice there is” (Yogi Berra). Both should inform one other.

7.Diversity over ability – Crowdsourcing (a word Howe coined) can solve some problems that dedicated professions can’t solve internally.

8.Resilience over strength – They quote the old adage that an oak tree is strong, but can be blown over in a storm; while the weaker but flexible willow bends with it and can survive. Akin to what Angela Duckworth popularized as “grit” in her great book with the same title.

9.Systems over objects – The value of interdisciplinary networking.

2. SO, WHO DOES WHAT?

• Workflow• Information

Architecture• Ease of Use• Taxonomy /

Terminology

• Interface Elements• Window/Pane

Management• Error Handling• Access to Help

• Page Layout• Color Use• Visual Language• Data Visualization• Iconography

BREAKDOWN OF RESPONSIBILITY

USER RESEARCHER INTERACTION DESIGNER VISUAL DESIGNER

3. COGNITIVE WALKTHROUGH

Getting Domain ExpertiseBUSINESS REPRESENTATIVES — Wrangle business objectives and understand opportunities for the business. Understand how to make or save money for the business. Understand how the design meets the needs of the product domain and its users.

PROJECT MANAGER — Ensure the client can implement this design in time and when/what the project milestones are.

TECHNICAL EXPERT — Understand technical framework (opportunities and limitations). Validate the client can implement the product recommendations (feasibility / time intensity).

YOUR TEAM - provides the domain expertise in “USER EXPERIENCE & USABILITY”, they provide industry domain expertise.

SETTING TARGETSKEY PRODUCT FEATURES

Plan to evaluate all key features that differentiate this product from it’s competitors. Ask who else is doing this, is any one doing this better?

KEY TASKS / WORKFLOWS

Plan to evaluate all key workflows in the system. Unless problematic, avoid evaluating common tasks (ex: login, logout)

KEY PERFORMANCE INDICATORS

Understand what metrics are important to the business: (ex. time to completion, task completion ratios, user registration, engagement volume, etc.)

BE A SPONGE:• Take this opportunity to understand the product from the business and user perspective. • Remain objective, do not adopt or project your cognitive bias.• Take screenshots and where possible, record the session (voice and screen)

4. GETCHO ANALYSIS ON!

IngredientsSHARED REPOSITORY• Directory: (Recording, Screenshots,

Documents, Tracking Sheets)• Collaboration Workspace: Google Slides,

InvisionApp, 3rd Party Tools

EVALUATION STRUCTURE• Outline by Issue Category or Functional

Areas• Issue category overview• Individual issue slides• Category findings

SCREEN MARKUPS• Point out problem areas• Make note of successes• Assign heuristic (1-13)• Assign Issue Category• Assign Severity / Prevalence (H, M, L)• Cite example solutions and

recommendations

KEY TAKEAWAYS (Executive Summary)• Findings• Key Issues• Severity/Priority

THANK YOU XEROX

• 22 Pages• 13 Principles• 288 Check Points

Source: http://users.polytech.unice.fr/~pinna/MODULEIHM/ANNEE2010/CEIHM/XEROX%20HE_CKLST.pdf

THANK YOU CHRIS GALLELLO

• Chrome Plugin• 10 Principles• - Heuristic• - Severity• - Notes• - Recommendations• In-Line Annotations• Word.doc Export

Source: : http://www.uxcheck.co/Reference: https://medium.com/@cgallello/low-cost-usability-testing-61b5f8a2a1be

5. PRE-GAME HUDDLE

Draft Run ThroughGAP ANALYSIS – Any areas need more attention? Use your KPI’s, Key Workflows, Key Product Features

INTERIM CLIENT CHECKPOINT – Depending on confidence and success, it’s always a good idea to check in with the client around the 60% complete mark

ORGANIZE FINDINGS — Look across what everyone has analyzed, group logically (by issue category or functional areas – avoid redundancy (cite instances and severity/prevalence)

DRIVE VALUE – Without giving up free work, provide broad recommendations an cite examples of successful implementations.

ASSIGN PRESENTERS / NOTE TAKERS —Depending on skills mix you may want to mix it up, unless you’re confident with one presenter

ESTIMATE PRESENTATION TIME — These reports can be dense and packed with rich discussion, you will need to account for extra time per issue.

6. SHOW TIME

EmpathyBE SENSITIVE TO POLITICAL / EMOTIONAL IMPLICATIONS <VERY IMPORTANT>• Always start on a positive note it’s important to

acknowledge where they hit the mark and showed innovative, unique and successful design

• Often times the interfaces you’re evaluating represent a significant amount of commitment, time, and/or cost investment on your client’s part

• Realize that every observation can be perceived as a personal critique by any number of client team members

BREAK THE NEWS IN THE BEST LIGHT POSSIBLE• Your perspective represents an objective 3rd

party observations that should not be charged with subjectivity and aesthetic/technical preference• This is particularly important for the visual

analysis• Your observations should be supported by

empirical evidence and sound solutions and recommendations

• Make room for conversations, check the pulse of how each observation and recommendation is landing

EngageYOU MAY HAVE CLARIFYING QUESTIONS • Make sure you’re taking time to validate your observations and recommendations

THIS IS NOT A PASSIVE SESSION• Create opportunity for the client to articulate priority and effort levels against your

observations and recommendations (H, M, L) this will be useful in backlog management

• You are proxy for the user, be sure to advocate for the usability impact of resolving or deprioritizing a fix, they are relying on you to be the expert

• Continue to articulate your recommendations and solutions in terms of KPI’s and what the would be most valuable to the business without negatively impacting ease of use and user value.

Drive MomentumIF YOU’VE DONE YOUR JOB…• The client would have agreed with 60%-80% of your findings• They will already be thinking about low handing fruit and how to parse the

enhancement into near term sprints and long term release cycles – this is your opportunity to step up.

THE EVALUATION REPORT IS THE BEGINNING, NOT THE END• You now own the issue log and recommendations with client feedback• Ranked by: Client Priority, User Benefit, Technical Complexity…• This is the making of what we call a Feature Matrix: a useful tool for UX

prioritization, sprint planning and backlog management

SAMPLEUX EVAL

<ATTENDEES ONLY>

FEATURE MATIX(sneak peek)

• Catalogue UX enhancements or requirements into swim-lanes

• Assign each cell with User Benefit, Business Benefit and Technical Complexity

• Use this information to build the priority ranking for sprint planning and backlog management.

YOU SAID POWER.

POWER FOR YOUYOUR BUSINESS DEVELOPMENT TOOL FOR UX WORKBrings you top of mind for follow on detailed design contract workClearly articulates the value of UX in terms that matter to the clientWets the appetite of the client without giving up detailed design workStrong business planning tool – showing UX as a strategic practice

POWER FOR YOUR CLIENTTACTICAL HITLISTIn most cases your observations and recommendations can dovetail into their existing development plan – the client can yield immediate value from the reportFor a relatively low cost, the client has gained critical external feedback for near and long term product enhancements – outputs from the report are typically folded into long term development plans

THANK YOU!Questions?

LOCO | UX Community News?Announcements?Next Topic Ideas?

THANKS TO OUR SPONSORS!

https://twitter.com/loco_ux

+CoSolve