Driving Fit & Finish in UX Quality: Desirability is in the Details
description
Transcript of Driving Fit & Finish in UX Quality: Desirability is in the Details
Driving Fit & Finish in UX Driving Fit & Finish in UX Quality: Desirability is in the Quality: Desirability is in the Details Details
Yingzhao Liu, Program Manager, WEX User Interface PlatformYingzhao Liu, Program Manager, WEX User Interface PlatformDonna Wallace, Senior UX Manager, DDE User ExperienceDonna Wallace, Senior UX Manager, DDE User ExperienceKynan Antos, UX Designer, Windows Home ServerKynan Antos, UX Designer, Windows Home Server
Dan Harman, Program Manager, Dynamic Systems FoundationDan Harman, Program Manager, Dynamic Systems FoundationAugust 2007August 2007
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
About this presentationAbout this presentation
Introduction Introduction What is Fit & FinishWhat is Fit & FinishDriving Fit & Finish for Windows VistaDriving Fit & Finish for Windows Vista
Fit & Finish everywhereFit & Finish everywhereVisual StudioVisual StudioWindows Home ServerWindows Home ServerServer ManagerServer Manager
Best practicesBest practicesTimelineTimelineCulture ChangeCulture ChangeIngredients for a successful processIngredients for a successful processWarning signsWarning signs
Take-awaysTake-aways
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
What is Fit & FinishWhat is Fit & Finish
A term in the manufacturing industry in general and A term in the manufacturing industry in general and typically refers to typically refers to the process of doing the final the process of doing the final assemblyassembly: putting the pieces together perfectly (Fit) : putting the pieces together perfectly (Fit) and adding the final layer of paint, polish, removing and adding the final layer of paint, polish, removing fingerprints, etc. (Finish).fingerprints, etc. (Finish).
In software as in manufacturing, it can not wait to In software as in manufacturing, it can not wait to the end. the end.
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Vista: requires high Fit & FinishVista: requires high Fit & Finish
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Some examples of Fit & Finish Some examples of Fit & Finish bugsbugs
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Some feedback on Release Some feedback on Release CandidateCandidate
““There are still too many pre-Vista icons. There are still too many pre-Vista icons. Take a Take a look inside the Windows folder, for example. The look inside the Windows folder, for example. The “Offline web pages” icon is 95-style, and the “Offline web pages” icon is 95-style, and the “Downloaded program files” icon is 2000-style. “Downloaded program files” icon is 2000-style. These have got to be updated! There is ZERO These have got to be updated! There is ZERO excuse for not doing it…excuse for not doing it…
““There are too many slightly-different shades of There are too many slightly-different shades of blue/green throughout the product. blue/green throughout the product.
““Each of the following apps have different Each of the following apps have different back/forward buttons: Windows Explorer, IE, Help & back/forward buttons: Windows Explorer, IE, Help & Support (*quite* different), Windows Media Support (*quite* different), Windows Media Player/Photo Gallery (these two are the same—yay!), Player/Photo Gallery (these two are the same—yay!), Windows Defender…”Windows Defender…”
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Fit & Finish bugsFit & Finish bugs
10,000 FnF bugs were entered; 8,000 unique; 4400 10,000 FnF bugs were entered; 8,000 unique; 4400 fixed. fixed.
Bug Categories:Bug Categories:
Visual glitchesVisual glitches
Incorrect text or graphicsIncorrect text or graphics
Layout errorsLayout errors
Animation errorsAnimation errors
Visual clarity problemsVisual clarity problems
Windows Vista UX Guidelines violations Windows Vista UX Guidelines violations (http://uxguide)(http://uxguide)
UI implementation errorsUI implementation errors
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Driving Fit & Finish for VistaDriving Fit & Finish for Vista
Got VP approvalGot VP approval
Focused on areas of the UI with the highest impactFocused on areas of the UI with the highest impact
Conducted review meetings, enforced UX Guide and Conducted review meetings, enforced UX Guide and exit criteriaexit criteria
Made it easy for anyone to enter visual bugsMade it easy for anyone to enter visual bugs
Staffed with our own testers and developersStaffed with our own testers and developers
Partnered with Basics: usability and accessibilityPartnered with Basics: usability and accessibility
Attended central shiproomAttended central shiproom
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
About this presentationAbout this presentation
IntroductionIntroduction What is Fit & FinishWhat is Fit & FinishDriving Fit & Finish for Windows VistaDriving Fit & Finish for Windows Vista
Fit & Finish everywhereFit & Finish everywhereVisual StudioVisual StudioWindows Home ServerWindows Home ServerServer ManagerServer Manager
Best practicesBest practicesTimelineTimelineCulture ChangeCulture ChangeIngredients for a successful processIngredients for a successful processWarning signsWarning signs
Take-awaysTake-aways
Visual Studio WhidbeyVisual Studio Whidbey
Donna Wallace, Senior UX Manager, DDE User ExperienceDonna Wallace, Senior UX Manager, DDE User Experience
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Visual Studio Whidbey (v7)Visual Studio Whidbey (v7)““One big piece of feedback from [our recent VP One big piece of feedback from [our recent VP reviews] that I agree with is that we need to focus reviews] that I agree with is that we need to focus more on end-to-end UI consistency across these more on end-to-end UI consistency across these products…. products….
……When you look at a product that is completely When you look at a product that is completely consistent in its UI model versus one that isn’t, I consistent in its UI model versus one that isn’t, I personally jump to all sorts of conclusions about the personally jump to all sorts of conclusions about the consistent one beingconsistent one being faster, more productive, faster, more productive, more stable, more compellingmore stable, more compelling, etc, etc. all because of . all because of that one attribute… in the end, the that one attribute… in the end, the consistency has consistency has a multiplier effect that helps sell the overall a multiplier effect that helps sell the overall valuevalue.”.”
— — Craig Symonds, GM Craig Symonds, GM
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Whidbey consistency initiatives Whidbey consistency initiatives
Promote designing for consistencyPromote designing for consistencyVia design guidelinesVia design guidelines
Target high profile inconsistenciesTarget high profile inconsistenciesVia core scenario walkthroughsVia core scenario walkthroughs
Address “death by 1000 paper cuts”Address “death by 1000 paper cuts”Finding and fixing the right issuesFinding and fixing the right issues
Invest in engineering for consistencyInvest in engineering for consistencyVia greater implementation sharingVia greater implementation sharing
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Whidbey consistency initiatives Whidbey consistency initiatives
Promote designing for consistencyPromote designing for consistencyVia design guidelinesVia design guidelines
Target high profile inconsistenciesTarget high profile inconsistenciesVia core scenario walkthroughsVia core scenario walkthroughs
Address “death by 1000 paper cuts”Address “death by 1000 paper cuts”Finding and fixing the right issuesFinding and fixing the right issues
Invest in engineering for consistencyInvest in engineering for consistencyVia greater implementation sharingVia greater implementation sharing
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Components of UX issue Components of UX issue managementmanagement1.1. Identify and prioritize all UI elementsIdentify and prioritize all UI elements
2.2. Determine prioritization schemeDetermine prioritization schemeSystematic method, communicated broadlySystematic method, communicated broadly
3.3. Find the right issues / bugsFind the right issues / bugs
4.4. Fix the right issues / bugsFix the right issues / bugsDefine success: set quality targetsDefine success: set quality targetsTriage team keeps teams honest and ensures Triage team keeps teams honest and ensures consistent severity ratings across the divisionconsistent severity ratings across the division
5.5. Track and reportTrack and report
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Identify UIIdentify UIUX : track UX reviews, Fit & Finish progressUX : track UX reviews, Fit & Finish progress
UE, Loc: track UI work and changesUE, Loc: track UI work and changes
Today some of the list is generated, most is manualToday some of the list is generated, most is manual
Dialogs Tool windows Editors
Total 630 76 50
# New ? 25
24
Reviewed ? 43 9
No review planned
236 14 11
Whidbey UIComponents.xls Whidbey UIComponents.xls
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
UI Element Priority: EP ratingUI Element Priority: EP rating
Upgrade priority based on Upgrade priority based on Part of main scenario: EP1Part of main scenario: EP1Market focus or affecting user SATMarket focus or affecting user SAT
EP1EP1
EP2EP2
EP2EP2
EP3EP3
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Severity definitionsSeverity definitionsSev 1 • Can’t use the feature
• Product appears unstable• Inadvertent user action causes data loss
Sev 2 •Feature is inconsistent with other elements • Customers are frustrated•Hard to use •Product appears poorly engineered
Sev 3 • Customers annoyed by UI inconsistencies• UI appears sloppy
Sev 4 • Little or no user impact
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Example: Severity 1 UI BugsExample: Severity 1 UI Bugs
Breadth:Breadth: 90% of customers will regularly see 90% of customers will regularly see issueissue
Depth:Depth: Prevents usage Prevents usage
Impact:Impact: Same impact as any Sev 1 technical Same impact as any Sev 1 technical bugbug
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Example: Severity 2 UI BugsExample: Severity 2 UI Bugs
Breadth:Breadth: > 50% of customers will regularly see > 50% of customers will regularly see issueissue
Depth:Depth: Significantly impacts usage Significantly impacts usage
Impact:Impact: Customer questions overall stability Customer questions overall stability
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Example: Severity 3 UI BugsExample: Severity 3 UI Bugs
Breadth:Breadth: ~ 25% of customers will see paper ~ 25% of customers will see paper cutcut
Depth:Depth: Annoying but doesn’t significantly Annoying but doesn’t significantly impact usageimpact usage
Impact:Impact: Multiple bugs portray sloppiness & lack Multiple bugs portray sloppiness & lack of professionalismof professionalism
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Example: Severity 4 UI BugsExample: Severity 4 UI Bugs
Breadth:Breadth: Only the trained eye will notice paper Only the trained eye will notice paper cutcut
Depth:Depth: Doesn’t impact daily usage Doesn’t impact daily usage
Impact:Impact: Customers won’t question specific bug, Customers won’t question specific bug, but multiple bugs make UI appear unfinishedbut multiple bugs make UI appear unfinished
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Prioritization matrix defines Prioritization matrix defines goalsgoals
EP1 EP2 EP3Sev 1
Pri 1 Pri 1 Pri 2
Sev 2
Pri 1 Pri 2 Pri 3
Sev 3
Pri 2 Pri 3 Pri 4
Sev 4
Pri 3 Pri 4 Pri 4
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Conducted by UI experts and feature teams Conducted by UI experts and feature teams together at each milestone start or end together at each milestone start or end
Evaluate both task flow and UI detailsEvaluate both task flow and UI details
Avoid surprises: Disclose upcoming feature workAvoid surprises: Disclose upcoming feature work
Get a “B” in coverage: Get first look at low Get a “B” in coverage: Get first look at low priority areaspriority areas
Gain consensus, set expectations accordinglyGain consensus, set expectations accordingly
Provide the management team a milestone Provide the management team a milestone checkpoint on the “Overall Product UI Health” checkpoint on the “Overall Product UI Health” scorecardscorecard
Many issues can be fixed without logging bugsMany issues can be fixed without logging bugs
Bug finding: UI reviewsBug finding: UI reviews
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
High profile inconsistenciesHigh profile inconsistencies
Smart TagsSmart Tags
Early designs: similar needs, Early designs: similar needs, but multiple but multiple implementations, very implementations, very different appearance and different appearance and behaviorbehavior
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
High profile inconsistenciesHigh profile inconsistencies
Final designs: fewer implementations Final designs: fewer implementations with a common look and feel. with a common look and feel.
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Bug finding: UI testingBug finding: UI testing
Ad hoc UI bugs (from UI Guidelines posted for teams)Ad hoc UI bugs (from UI Guidelines posted for teams)
Systematic approach: UI test casesSystematic approach: UI test cases
Derive objective test cases from guidelinesDerive objective test cases from guidelines
Format test cases to fit QA toolsFormat test cases to fit QA tools
Teach QA teams how to review UITeach QA teams how to review UI
Categories of bugs:Categories of bugs:CommunicationCommunicationEase and efficiencyEase and efficiencyError preventionError preventionVisual designVisual designConsistencyConsistencyEngineeringEngineering
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Detail design Fit & Finish: Setup Detail design Fit & Finish: Setup
Before:Before:Pseudo-web Pseudo-web appearance appearance and navigationand navigation
Brand Brand imagery imagery not not consisteconsistentnt
Nonstandard Nonstandard themed themed controlscontrols
Nonstandard Nonstandard font face & font face & colorcolor
nonstandard nonstandard colorscolors
Use of links to Use of links to perform perform actionsactions
nonstandard nonstandard grid grid
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Detail design Fit & Finish: Setup Detail design Fit & Finish: Setup
Updated use of Updated use of brand imagerybrand imagery
Standard Standard themed themed controlscontrols
Standard Standard foreground foreground color & color & fontsfonts
Standard colorsStandard colors
Appropriate use Appropriate use of linksof links
Use of buttons for action Use of buttons for action commandscommands
Standard grid Standard grid
After:After:
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
ReportingReporting
All Active Fixed Won’t Fix/Postponed
All UX Bugs 4303 342 1952 706
Sev 1 / Sev 2 1993 136 922 239
XP Theme Bugs
105 2 55 9 (2 violate exit criteria)
Key Guidelines Violations
120 3 56 0
• Due to the security push, seeing lower fix rates over those weeksDue to the security push, seeing lower fix rates over those weeks• 102 UX bugs added since 11/12102 UX bugs added since 11/12• 73 UX bugs resolved as Fixed since 11/1273 UX bugs resolved as Fixed since 11/12• 13 UX bugs closed as Won’t fix or Postponed since 11/1213 UX bugs closed as Won’t fix or Postponed since 11/12
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
In summaryIn summary
Things that worked wellThings that worked wellKey: had clear directive from executivesKey: had clear directive from executives
Built reviews into divisional schedule and set expectationsBuilt reviews into divisional schedule and set expectations
Systematic approach resonated well with engineering practicesSystematic approach resonated well with engineering practices
Ensured flexible process to tailor fit individual teamsEnsured flexible process to tailor fit individual teams
Had adequate reporting and communication both up and downHad adequate reporting and communication both up and down
Things that we’re still working onThings that we’re still working onUnderstanding how to better integrate into Feature Crew model Understanding how to better integrate into Feature Crew model
Tuning quality gates and exit criteriaTuning quality gates and exit criteria
Measuring success beyond using only bug counts (find or fix Measuring success beyond using only bug counts (find or fix
rates)rates)
Defining a method to better identify “dirtiness factor”Defining a method to better identify “dirtiness factor”
Windows Home ServerWindows Home Server
Kynan Antos, UX Designer, Windows Home ServerKynan Antos, UX Designer, Windows Home Server
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Fit & Finish is about qualityFit & Finish is about quality
“From what we’ve seen so far, we’d rate Windows Home Server as one of Microsoft’s most polished and most impressive 1.0 releases to date.”
— APC Magazine
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Examples of FnF issues we Examples of FnF issues we encountered…encountered…
Improperly rendered transparency…Improperly rendered transparency…
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
colors didn’t match…colors didn’t match…
Background mismatchBackground mismatchAliased edges: lack of Aliased edges: lack of transparencytransparency
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
placeholders were still placeholders were still placeholding…placeholding…Placeholder icons…Placeholder icons…
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
deep investigation led to big deep investigation led to big issues…issues…
Collision…Collision…
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
poor implementation created poor implementation created issues…issues…
Artifacts from Artifacts from stretching…stretching…
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
creative solutions are often creative solutions are often necessarynecessary
btn_end_lft.pngbtn_end_lft.png btn_bg.pngbtn_bg.png btn_end_rt.pngbtn_end_rt.png
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
sometimes things were all sometimes things were all wrong…wrong…Coordinates and spacing…Coordinates and spacing…
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
all of these issues add up fast…all of these issues add up fast…
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
it isn’t risky (or difficult) to fix it isn’t risky (or difficult) to fix them…them…
Fit & Finish Bugs Filed
Bugs Resolved Regressions
114 74 1
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
...fixing these issues has a big ...fixing these issues has a big impact.impact.“We don’t want to award anyone before the
final product is out, but we’ll say that we came out of the closed beta very impressed.”
— CNET
“I found Home Server to be intuitive to use... Its simplicity and automation will make it a winner with many home users.”
— Computer World Magazine
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
What workedWhat worked
Adoption of the Vista Fit & Finish Adoption of the Vista Fit & Finish
Involve team in prioritizing feature areasInvolve team in prioritizing feature areas
Use soft skills, build relationshipsUse soft skills, build relationships
Get out of your officeGet out of your officeWork directly with developers to fix issuesWork directly with developers to fix issues
Write good bugs, always use screenshotsWrite good bugs, always use screenshots
Focus on quality in the product, not on bug countFocus on quality in the product, not on bug count
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
What didn’t workWhat didn’t work
Lack of early executive buy-inLack of early executive buy-in
Started too late in the scheduleStarted too late in the schedule
Feature slip was prioritized over Fit & Finish scheduleFeature slip was prioritized over Fit & Finish schedule
No time to file, triage, and manage bug overheadNo time to file, triage, and manage bug overhead
No ship criteria with teethNo ship criteria with teeth
No incentives for building qualityNo incentives for building quality
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
What we are doing next timeWhat we are doing next time
Getting early executive buy-inGetting early executive buy-inGetting Fit & Finish on the schedule earlyGetting Fit & Finish on the schedule earlyScheduling Fit & Finish within each milestone after ZBBScheduling Fit & Finish within each milestone after ZBB
Using Product Studio for tracking Fit & Finish issuesUsing Product Studio for tracking Fit & Finish issues
Developing and enforcing Fit & Finish ship criteria Developing and enforcing Fit & Finish ship criteria
Giving UX the authority to get resultsGiving UX the authority to get results
Sharing the Fit & Finish processSharing the Fit & Finish process
Server ManagerServer Manager
Dan Harman, Program Manager, Dynamic Systems FoundationDan Harman, Program Manager, Dynamic Systems Foundation
Presented by Yingzhao LiuPresented by Yingzhao Liu
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Server ManagerServer Manager
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Review Process for Server Review Process for Server ManagerManager““We were able to fix numerous issues that We were able to fix numerous issues that
would not have been fixed otherwise, and would not have been fixed otherwise, and collectively, the impact on the UX has been collectively, the impact on the UX has been significant. Combined with the changes to significant. Combined with the changes to our team culture after having witnessed the our team culture after having witnessed the cost versus benefit, we’re impressed with the cost versus benefit, we’re impressed with the results.”results.”
—Dan Harman, PM
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Review Process for Server Review Process for Server ManagerManager• Used Vista Fit & Finish review process as a baselineUsed Vista Fit & Finish review process as a baseline• Conducted two 2-hour review sessions in March 2007Conducted two 2-hour review sessions in March 2007• Focused on high priority areas:Focused on high priority areas:• Core console elementsCore console elements• Core wizard framework elementsCore wizard framework elements• Custom controlsCustom controls• Dynamics in the UIDynamics in the UI• Key experiencesKey experiences
• Involved participants from 4 disciplines: PM, design, Involved participants from 4 disciplines: PM, design, dev, and testdev, and test• PM drove process, took notes, and filed bugsPM drove process, took notes, and filed bugs• Design contributed to reviews and assigned severity to issuesDesign contributed to reviews and assigned severity to issues• Test contributed to reviews and provided test environmentsTest contributed to reviews and provided test environments• Dev triaged, costed, and fixed bugsDev triaged, costed, and fixed bugs
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Filing BugsFiling Bugs
• Used same bug categories as Vista Fit & Finish Used same bug categories as Vista Fit & Finish reviewsreviews• Visual glitchesVisual glitches• Incorrect text or graphicsIncorrect text or graphics• Layout errorsLayout errors• Animation errorsAnimation errors• Visual clarity problemsVisual clarity problems• UX Guide violationsUX Guide violations• UI implementation errors UI implementation errors
• Followed Vista Fit & Finish process for filing bugsFollowed Vista Fit & Finish process for filing bugs• Filed bugs for all issues foundFiled bugs for all issues found• Assigned Priority 2, Severity 2 for most bugsAssigned Priority 2, Severity 2 for most bugs• Added “RmtVisual” keyword for tracking purposesAdded “RmtVisual” keyword for tracking purposes
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
ResultsResults
• 41 issues identified:41 issues identified:• Fixed: 27Fixed: 27• Pushed: 7Pushed: 7• External: 4External: 4• By Design: 3By Design: 3
• Fixed top 80% of bugs with highest impact/cost ratio Fixed top 80% of bugs with highest impact/cost ratio (27/34)(27/34)
• Examples of bugs fixed:Examples of bugs fixed:• Default OOBE layout should not scrollDefault OOBE layout should not scroll• Painting issues on initial console loadPainting issues on initial console load• Role captions not aligned properlyRole captions not aligned properly• Padding around all tiles should be consistentPadding around all tiles should be consistent
• Examples of bugs pushed to Windows 7:Examples of bugs pushed to Windows 7:• Visual inconsistency in NotificationsVisual inconsistency in Notifications• OOBE status column font size slightly too large in 120 DPIOOBE status column font size slightly too large in 120 DPI
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Culture ShiftCulture Shift
• Dev fixed all Fit & Finish bugs from reviews in 2 Dev fixed all Fit & Finish bugs from reviews in 2 daysdays
• Team realized visual impact of fixing these bugsTeam realized visual impact of fixing these bugs• Given good results and low cost, team embraced Given good results and low cost, team embraced
Fit & Finish reviews as a worthwhile investmentFit & Finish reviews as a worthwhile investment• Success has lowered the barrier Success has lowered the barrier to fixing other to fixing other
Fit & Finish bugs since the reviewsFit & Finish bugs since the reviews• Moving forward, plan to make Fit & Finish a Moving forward, plan to make Fit & Finish a
more formal part of development process for more formal part of development process for future releasesfuture releases
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
About this presentationAbout this presentation
IntroductionIntroduction What is Fit & FinishWhat is Fit & FinishDriving Fit & Finish for Windows VistaDriving Fit & Finish for Windows Vista
Fit & Finish everywhereFit & Finish everywhereVisual StudioVisual StudioWindows Home ServerWindows Home ServerServer ManagerServer Manager
Best practicesBest practicesTimelineTimelineCulture ChangeCulture ChangeIngredients for a successful processIngredients for a successful processWarning signsWarning signs
Take-awaysTake-aways
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
TimelineTimeline
Get Fit & Finish on divisional scheduleGet Fit & Finish on divisional schedule
During planningDuring planning
Conduct spec review, verify UX Guide complianceConduct spec review, verify UX Guide compliance
At key points during product cycle At key points during product cycle (e.g. milestones, beta, (e.g. milestones, beta, RC)RC)
Conduct Fit & Finish review and triage bugsConduct Fit & Finish review and triage bugs
Encourage customer bugs and feedback on UXEncourage customer bugs and feedback on UX
Conduct Fit & Finish bug bashConduct Fit & Finish bug bash
Before visual freezeBefore visual freeze
Coordinate focused Fit & Finish dev pushCoordinate focused Fit & Finish dev push
Work with localization on late graphical resource Work with localization on late graphical resource changeschanges
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Culture changeCulture change
In our competitive markets, quality experience is In our competitive markets, quality experience is criticalcritical
Get executive buy-in—in the schedule and funded Get executive buy-in—in the schedule and funded
Clarify what we mean by shipping qualityClarify what we mean by shipping quality
Set bug Severity and Priority correctly and Set bug Severity and Priority correctly and consistentlyconsistently
Be clear that low quality work isn’t acceptableBe clear that low quality work isn’t acceptable
Be helpful, open, and respectfulBe helpful, open, and respectful
Reward high quality UX Reward high quality UX
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Ingredients for a successful Ingredients for a successful processprocess
Use http://uxguide as baseline guidelinesUse http://uxguide as baseline guidelines
Define what successful FnF looks likeDefine what successful FnF looks like
Equip testers with specific, testable set of guidelinesEquip testers with specific, testable set of guidelines
Circulate best practices for UI codingCirculate best practices for UI coding
““Conference room testing” to call out UX issuesConference room testing” to call out UX issues
Triage and drive Fit & Finish bugsTriage and drive Fit & Finish bugs
Coordinate focused UX bug fix pushesCoordinate focused UX bug fix pushes
Communicate and report, let people know of your Communicate and report, let people know of your impactimpact
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Warning signsWarning signs
““We are really serious about FnF, but there’s no We are really serious about FnF, but there’s no budget, nor time in the schedule.”budget, nor time in the schedule.”
You and customers never know when to file FnF You and customers never know when to file FnF bugs: “Now is too early.” “Now is too late.”bugs: “Now is too early.” “Now is too late.”
““We will have to change documentation if we fix this We will have to change documentation if we fix this bug.” bug.”
““These FnF bugs are just in ______ mode, most These FnF bugs are just in ______ mode, most customers won’t see it.”customers won’t see it.”
““FnF is nice to have, but we have to resolve our FnF is nice to have, but we have to resolve our realreal bugs first.” bugs first.”
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
About this presentationAbout this presentation
IntroductionIntroduction What is Fit & FinishWhat is Fit & FinishDriving Fit & Finish for Windows VistaDriving Fit & Finish for Windows Vista
Fit & Finish everywhereFit & Finish everywhereVisual StudioVisual StudioWindows Home ServerWindows Home ServerServer ManagerServer Manager
Best practicesBest practicesTimelineTimelineCulture ChangeCulture ChangeIngredients for a successful processIngredients for a successful processWarning signsWarning signs
Take-awaysTake-aways
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
The value of Fit & FinishThe value of Fit & Finish
Conducting Fit & FinishConducting Fit & FinishPut it in the schedulePut it in the schedule(Culture) change is inevitable(Culture) change is inevitablePlant ingredients for successPlant ingredients for successKnow the warning signsKnow the warning signs
Everyone can make an impact with Fit & FinishEveryone can make an impact with Fit & Finish
Take-awaysTake-aways
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
ResourcesResources
Windows Vista User Experience GuidelinesWindows Vista User Experience Guidelineshttp://uxguidehttp://uxguide (internal) (internal)
http://msdn.microsoft.com/windowsvista/uxguidehttp://msdn.microsoft.com/windowsvista/uxguide (external) (external)
Guidelines-based User Experience DesignGuidelines-based User Experience Design online course online course
Vista Fit & Finish ProcessVista Fit & Finish ProcessProcess document by Dave VronayProcess document by Dave Vronay
Talk by Everett McKayTalk by Everett McKay
AppendixAppendix
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
FnF ReviewsFnF Reviews
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
FnF Bugs Root Cause Analysis FnF Bugs Root Cause Analysis (RCA)(RCA)
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Analysis of Apple’s development Analysis of Apple’s development process:process:
From Russ Albright’s Analysis of Apple’s Development From Russ Albright’s Analysis of Apple’s Development Process (FTE’s only)Process (FTE’s only)User focused culture (p. 16, 49)User focused culture (p. 16, 49)Smaller code base (15 vs. 50 mloc), leverage open Smaller code base (15 vs. 50 mloc), leverage open source code, focus on code reuse (p. 18)source code, focus on code reuse (p. 18)Better UI development frameworks and toolsBetter UI development frameworks and toolsFnF bugs are P2/P3 instead of P3/P4 (p. 52)FnF bugs are P2/P3 instead of P3/P4 (p. 52)Ready to release when 85% of bugs are resolved (p. Ready to release when 85% of bugs are resolved (p. 54)54)Have a Senior HCI Director responsible for cross Have a Senior HCI Director responsible for cross product consistency (p. 56)product consistency (p. 56)All designs must conform to their guidelines, managers All designs must conform to their guidelines, managers are responsible for consistency of their products (p. are responsible for consistency of their products (p. 58)58). .
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Connection Dialog: refreshConnection Dialog: refreshGenerally, Generally,
provider tab rarely provider tab rarely used after the first used after the first
timetime
Generally, Generally, provider tab rarely provider tab rarely used after the first used after the first
timetime
98% of the time 98% of the time the same the same
connection page is connection page is used for all used for all connectionsconnections
98% of the time 98% of the time the same the same
connection page is connection page is used for all used for all connectionsconnections
““All” is there only All” is there only to expose more to expose more
advanced settings advanced settings in a gridin a grid
““All” is there only All” is there only to expose more to expose more
advanced settings advanced settings in a gridin a grid
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
settings for the current provider settings for the current provider are “wrapped” by the frameare “wrapped” by the frame
settings for the current provider settings for the current provider are “wrapped” by the frameare “wrapped” by the frame
Provider is now Provider is now changed by launching changed by launching
secondary dialogsecondary dialog
Provider is now Provider is now changed by launching changed by launching
secondary dialogsecondary dialog
Advanced settings now available under an Advanced settings now available under an advanced… buttonadvanced… button
Advanced settings now available under an Advanced settings now available under an advanced… buttonadvanced… button
Engineering Excellence • Trustworthy ComputingMicrosoft Confidential
Method Communi-cation
Consistency
Ease & Efficienc
y
Engineering
ErrorPreven
t.
VisualDesign
AdHoc 1.33% 48% 9.33% 2.66% 0 38.66%
QA Guideline Test Case
30.66% 53.33% 4% 1.33% 0 10.66%
Observational Field Research
33.33% 18.66% 33.33% 5.33% 2.66% 6.66%
Milestone UI Walkthroughs
1.33% 36% 6.66% 1.33% 1.33% 53.33%
Usability Studies
50.66% 12% 26.66% 0 4% 6.66%
User Correspondence
12% 28% 36% 0 4% 20%
Cognitive Walkthrough
40% 29.33% 5.33% 9.33% 2.66% 13.33%