Usability Testing ofRMAs WebsiteWhat Weve Learned So Far for www.rma.usda.govJanet Stevens RMA Website Project ManagerJanet_Stevens@wdc.usda.gov 202-690-3040

What Is Usability?The measure of the quality of a user's experience when interacting with a product or system, such as a web site, a software application, mobile technology, or any user-operated device.

Why Usability Testing?People cannot find the information they seek on web sites about 60% of the time. Could ours be that bad or worse?For 42 major sites where usability had been emphasized at the design stage, they performed 135 percent better on average. Could we see comparable traffic gains?More than 90% of users went to only 1% of the Microsoft site. Could we have a lot of content that is rarely seen? How much can be removed?

Why Now?Site statistics showed that the rapid annual growth following our first redesign in Oct. 1999 had slowed. We decided to transition the site to a portal, but what did our audiences want more of? Less of? How did they use our site? How did they want theinformation to be presented? PDAs? Cell phones?Annual Percent IncreaseHitsVisitors

Why Contract Out?Complying with Section 508 accessibility requirements for the web pages would use up existing web team resourcesThere would be no resources left to help developers ensure Section 508 compliance of existing online applicationsThere was no formal team experience with or education on web usability testingThere was no current data available on our core audiences

Our GoalsLearn about our 3 major audiences:Producers, especially limited-resource, minority, and specialty cropCommodity insurance agents who sell to and manage policies/claims for producersRMA and FSA employees, especially in the field offices, who producers rely on extensively for program information and education

Our GoalsHave an independent review of the accessibility/usability of our online applications and web site templateHave the user interface of our top two online applications completely redesignedGet usability/accessibility training and documentation for the web team and developers

What We Learned About Our Core AudiencesProducers:Who use the Internet tend to be younger, more technology savvy, more highly educated, and operators of larger farmsDo not regularly go to Web sites to get information; however, some want to conduct business with the government online in the futureIn many operations task a spouse or child to gather relevant ag information and news for the enterpriseLook for specific information: a quotation, a premium estimate, or a county production average

What We Learned About Our Core AudiencesProducers:Connect at lower bandwidths, do not scroll, and are impatient with slow downloads/screen refreshesWant lessons learned/best practices and see what other producers are doing re: insurance, planting techniques, or farm management case studiesLike finding hints, tips, and suggestions for improving their operationWant to use the Internet in the future for online trainings, rather than attending local eventsWant to learn more about insurance costs/options without an agent

What We Learned About Our Core AudiencesCommodity Insurance Agents:Connect at higher bandwidthsTend to have our site as their home pageAre familiar with agency/insurance terms, but there were some terms used on our applications that even they did not knowMonitor our site for the latest agency newsUse our daily ag weather information, but rely on producers and local contacts for local weather newsWant producers to come to them before selecting insurance products

What We Learned About Our Core AudiencesCommodity Insurance Agents:Visit Tools and Calculators extensively and most used either the Summary of Business or the Premium Calculator every week or several times a weekTended to use the proprietary version of the Premium Calculator provided by their insurance company, not our version. Sometimes because they were used to it or they felt it had more features than ours, which some discovered wasnt necessarily true

What We Learned About Our Core AudiencesRMA/FSA Employees:Use the information on our intranet unless they can only find it on the public siteLook at information on a larger scale, such as yearly trends for overall participation rates or acres enrolled in the commodity insurance programHave direct contact with producers and often help them use the siteNeed to find a specific answer in a short period of time for a congressional or media inquiry

The BasicsParticipants would scroll to here, then go back to the top. They dont tend to go below the fold.

The Basics: Summary of BusinessUsers didnt: Like the constant refreshes to load informationWant their results to come up in a new browser windowUnderstand all of the terminology

Users wanted: To be able to select multiple items and generate a custom report with subtotalsA better Help function

The Basics:Premium Calculator Producers were confused about the login, output, and terminology Since not all of the agents knew all the terms, no producer--and even some agents--could generate an accurate quote

Putting It All TogetherWe need to make our applications and web site template more usable by:Improving the terms & organization of our nav barSegregating information based on audience type; often their needs are too diverseImproving our online glossaryAdding producer case studies to teach others about best practicesImproving our search capabilities

Next Steps1. Review information from our card sorting exercises to help us understand what terms our audiences recognize and how they group them to redevelop the nav bar and site structure2. Review and approve initial rebuilds of the template and the applications in wireframes before coding

Next Steps3. Build out new Premium Calculator and Summary of Business. Use the new templates for the other online apps so they have the same look and feel4. Attend vendor-developed usability and accessibility training. Use what we learn to train other web page editors

Next Steps5. Reorganize content and provide it based on audience intentions6. Gradually phase in web content management system to ensure site consistency and content accuracy7. Participate in the ACSI customer satisfaction survey to continually monitor how were doing (see www.theacsi.org)

Thank you!

Research by User Interface Engineering, Inc., shows that people cannot find the information they seek on Web sites about 60% of the time.

Industry expert Jakob Nielsen measured traffic at 42 major sites where usability had been emphasized at the design stage and found that they performed 135 percent better on average.

Industry content expert Gerry McGovern stated that 90% of users went to only 1% of the Microsoft siteSite statistics showed that the rapid annual growth following our first website redesign in October 1999 had slowed, from a rate of 46% for hits and 43% for visitors from 2000-01 to 16% for hits and 15% for visitors from 2001-02.

Although staying at near-record levels of 3.4 million hits and nearly 80,000 visitors per month, site usage had slowed. We decided to transition the site to a portal to accommodate our diverse audiences.

To transition to a portal in the next 12-18 months under the eGov initiative, we would need audience analysis and usability/accessibility testing.They believe that the answer is somewhere on the site, but cant always find it quickly and often ask someone in the office who can point them in the right direction.On this last item, we learned that significant numbers of our users were abandoning their search or even the site before selecting even a single match to view. This pointed to a more significant problem with our search engine than we had previously thought.