Adoption Metrics Dashboard

James Gray

CONFIDENTIAL INFORMATION –Distribution on a Business Need to Know Basis Only

The initial challenge

How to complete a heuristic review so that there was no question to its objectivity?

Standard heuristics procedure:

• baseline client’s environment

• perform competitive research

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 2

A SUS / binary scale for objectivity

My goal was to measure the major UX design categories and value points known at the time.

Then, apply a finite, objective SUS score to each value point.

• Navigation

• Organization

• Taxonomy

• Content

• Features & Functions

• Calls to Action

• Design and Brand Promise

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 3

The resulting scale was expandableand sustainableData and revised criteria could be updated as features/functions and value criteria changed over time.

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 4

Successful heuristics deliverable

• Atlanta Children’s Museum

• Emory University

• Equifax

• Florida Citrus Sports

• Food Lion

• Georgia Government Web Site

• The Home Depot

• ING Financial

• Kmart

• SunTrust Bank

• State Farm Insurance

• Yahoo!

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 5

New challenge: Spring 2019Can we measure compliance to a Design Management System?• Are patterns being adopted?

• Where and how often?

• By which UX teams?

• Can we maintain a constant, up-to-date measure of pattern adoption when the sites and apps are constantly evolving?

• Can we make this data useful without it being a “policing” issue?

• Can we turn subjective design uses into an objectivemeasure of adoption?

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 6

Initial solution: The manual MVP

Measure adoption based on eightcharacteristics:

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 7

1. Type Specs(adherence to standard punctuation, capitalization, and approved terms)

2. Grammar(content construction and subject/verb agreement)

3. Content Voice & Tone (adherence to approved SF active voice and conversational tone)

4. Visual Design Compliance(Line weights, color use, grid position, padding, and other design placement standards)

5. Visual Design Appropriateness(correct use of pattern – example: using a primary button vs a secondary button as per standards)

6. Architectural and interaction appropriateness(correct use of object within the flow of an interaction, its animation processes, and appropriate states)

7. Architectural/Hierarchical Placement(Information architecture and appropriate screen prioritization/placement of pattern objects)

8. 1X Developer Code and CSS compliance

Initial solution:Reports access

Invision-accessible documentation:• Data updates based on reporting from

• Design Evolution Updates

• Lift & Shift Team

• Conversion Teams

• Monthly score cards

• Alerting UX teams to their current status and standing

• Trending data over time

• Gauge their success vs other teams

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 8

Initial solution:Excel format

Sharepoint-accessible spreadsheets:

• Pivot tables for personalized adoption data by Team

• By-pattern and by-page conversion status

• Breadth of page workloads

• Redirect data relevant to content strategists and SEO

• Inter-page continuity of pattern uses and designs

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 9

North-star solution:Automated access

Unless we could find a way to automate the lion share of the data collection, report push and pull processes, and streamline support, the project was not sustainable.

I sought a solution that could:

• On-page metadata coding allow automated collection of pattern data

• Web-accessible dashboard allow teams to more quickly access and manipulate pattern data on the fly

• DMS pages can provide on-page links to live-site examples of patterns in use.

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 10

North-star reality:Automated access

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 11

Plan of Action

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 12

In order to prove value, I set about creating a strategy for approaching the issue holistically.

North-star reality:Automated access

• Manual process requiring weeks of QA is now done nightly within a few hours

• Over 31,000 data points tested on 560+ live site pages

• Over 200 Design System elements andcode snippets

CONFIDENTIAL INFORMATION – Distribution on a Business Need to Know Basis Only 13

1x AMDFull Lifecycle Strategic Plan

• AMD ecosystem on four experience tracks:1. Interface2. Support3. Frontify4. Communications

• Timeline statusü Development and MVP use cases - Completed MY 2021ü Pilot POC strategy – Began June 2021• Official rollout – Begins July 2021• Post-rollout – Ongoing leadership reporting models

• Axure Prototype

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

1x AMD ecosystem tracks: Application interface

Jenkins Developer Interface (detail screens)

Qlik Sense Enterprise Interface (reporting screens)

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

How it works

16

AMD allows for real time, automated analysis of .com pages, ensuring adherence to 1x standards.

Data summarized on the dashboard indicate where corrections need to be made. Not only will it keep our channels unified, on brand, and in code compliance - it’ll ensure they receive automatic refreshes when updates are made to those 1x components.

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

1x AMD ecosystem

17

1x AMD

Support

FrontifyData interface

Communications

4 key components to ourcomplete user experience

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

1x AMD ecosystem

18

1x AMD

Support

FrontifyData

Interfaces

Communications

Qlik Sense

Qlik SenseDASH

BOARD

Jenkins Details

Excel & CSV

1x Site Reference

s

YammerAnnouncements

LeadershipReports

Email Communi

cations

Periodic Special Request

Application News/

Updates

1x Online

1x Email

1x Authentic

ated 1x Content

…future

…future

Gitlab

Use Documen

tation

About AMD

Get Started

Documentation

End User Direct

Support

Product Team

memberDeveloper

Leadership

Support Form

Troubleshooting

Creating a complete user experience

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

1x AMD ecosystem tracks: Frontify support site

19

• Introduction to AMD (about)• What is it?• Who is the AMD Team?

• Getting started quick guide• How to get to AMD• Interface basics• Producing your first report• Saving/sharing a report

• Report repository• Monthly backlog of reports• Requesting unique reports

• Help• How to interpret reports• FAQs• Resource links for using QS• Support Request Form• AMD Contact Info

• Application news/updates• Version updates

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

1x AMD ecosystem tracks: Frontify continuity

20

• Each Design System channel is built to use the same interface and organization

• Elements are created to enhance brand continuity

• 1x AMD follows this format forease-of-use and identification of elements and patterns

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

1x AMD ecosystem tracks: Communications

21

• Pre-pilot• Communicate to POC

Participants• Documentation and instructional

support

• Pilot• Kickoff event• Periodic communications• Feedback Mural• Interviews?• Exit survey?• Agile iterative approach

• Post-pilot• Rollout communications• Yammer• Email updates• Monthly (?) leadership reporting• Frontify “care and feeding”

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

1x AMD ecosystem: TimelinesMay June July

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

Pilot Participants Feedback via Mural translated into punch list in Version 1

23

For Internal State Farm Use Only – Contains confidential information which may not be disclosed without express written authorization.

Personal growth

24

This project allowed me the real-world opportunity to:

• Expand my own skills in analytics tools and processes• Think “outside the box” of subjective vs objective data• Provide valuable heuristics and compliance data in record time• Create a useful application area that my colleagues can use to measure and

improve their own deliverables• Augment that application with a complete user experience including support,

documentation, reporting, communications, and UI• Assemble and lead a complete team of UX professionals with a clear vision for

success