PLAIN 2013 - Is it really plain? A case (and process) for content testing

PLAIN 2013 | Vancouver CA

Annetta Cheek Center for Plain Language Kath Straub Usability.org | Center for Plain Language

Is it really plain? A case for content testing


Where we are going ….

•  A parable •  A process


The Parable


We identified 10 “must-know” facts. 1.  Any veteran can join. 2.  When you join, you give access to your

VA System electronic health record, including

–  Your vital signs –  Diagnoses –  Doctor’s notes –  Medicines –  Treatment history –  Test results

3.  VA will use the data starting from the time you entered the system until 10 years after you joined Donate My Data.

4.  If you die during the program, the VA will stop using your data.

5.  VA will follow the laws about protecting your data.

6.  VA will use donated data to test software. 7.  VA uses made-up data to test software

now. 8.  Employees and contractors who build

software for the VA will have access to your health data.

9.  You can drop out at any time, by sending an email request to a specific address.

10.  If you drop out, VA will keep your authorization and withdrawal forms, and stop using your data.


The brochure didn’t test as well as we expected ….

1 2 3 4 5

6 7 9 8 10

Target = 80% recall


What we learned

•  Some key facts were not clear enough •  As a result, the brochure was not as persuasive as

desired

•  If we hadn’t tested, we would never have known.


Why am I telling you this?

•  We have entered the age of content.


In the Age of Content, there are no guides.


* Tes?ng shows that the F-‐shaped reading behavior is a symptom of the relevance of content not an determined habit of humans.

In the age of content, we will have to stop blaming the victim.



•  We have entered the age of content. •  Comprehension testing sounds harder than it is.

(Don’t tell anyone.)


Usability people

Usability people

•  Content is not our job, its their job

Content People

•  We are writers, not researchers



•  We have entered the age of content. •  Comprehension testing sounds harder than it is. •  We were surprised by our own results.

Don’t tell anyone this either, ok?


A process (Guerilla content testing)


3 types of content testing

•  “Simple” comprehension testing •  Confidence testing •  Persuasiveness testing

•  Some logistics


“Simple” comprehension testing

1.  Agree on the facts 2.  Decide which are most important 3.  Create a ques?on about each fact 4.  Agree on the answers


The challenge: Good questions are hard to write

Tips: •  Expect to revise your ques?ons several ?mes •  People remember m/c strategies from uni (e.g.,

the longer, specific answer is the right one. •  Give par?cipants an alterna?ve to guessing

(e.g., “The brochure didn’t say”)



1.  Agree on the facts 2.  Decide which are most important 3.  Create a ques?on about each fact 4.  Agree on the answer

5.  Pre-‐test your ques?ons


Comprehension testing: Have a Bake-off

Tip: No one version is going to be all be\er. By tes?ng mul?ple versions you can compare and contrast which works best (for which audiences.)



1.  Agree on the facts 2.  Decide which are most important 3.  Create a ques?on about each fact 4.  Agree on the answer

5.  Pre-‐test your ques?ons

6.  Use the test to have a bakeoff


Reporting comprehension results


Confidence testing

Could you describe the Donate my Data program to a family member or friend? a.  I can describe both the idea and the details to someone else, and get them all

right. b.  I can describe it mostly, but I might get some of the details wrong. c.  I can describe the general idea, but not the details. d.  I wouldn’t try to describe this to someone else.


Persuasiveness testing

Just because I can, doesn’t mean I will.


Persuasiveness testing •  Motivators •  Blocks/Concerns •  Barriers


Crowdsourcing science

•  A parable •  A process

Mechanical Turk (Mturk).


Some MTurk tips

•  Start with a comprehension test •  Embed ‘catch’ questions •  Create a ‘panel’ over time •  You get what you invest and what you pay for


A case for testing

•  Testing doesn’t have to be complicated to be robust •  Crowd-sourced testing can be quick, cheap and

effective. •  You don’t really know its plain language to your target

audience unless you’ve tested in your target audience

PLAIN 2013 - Is it really plain? A case (and process) for content testing

Technology

Transcript of PLAIN 2013 - Is it really plain? A case (and process) for content testing