The Survey Octopus - getting valid data from surveys

The Survey OctopusGetting valid data from surveys

Caroline Jarrett@cjforms2016 #uxinthecity

Caroline Jarrett @cjforms

What would you do for a dollar?

Caroline Jarrett @cjforms (CC) BY SA-4.03

$1 in the envelope beats $10 (guaranteed) later

Response relies on effort, reward, and trust

People will only respond if they trust you. After that, it's a balance between the perceived reward from filling in the survey compared to the perceived effort that's required. Strangely enough, if a reward seems 'too good to be true' that can also reduce the response.

Diagram from Jarrett, C, and Gaffney, G (2008) “Forms that work: Designing web forms for usability” inspired by Dillman, D.A. (2000) “Internet, Mail and Mixed Mode Surveys: The Tailored Design Method”

I’m a forms specialist

Image credit: Flickr, taxrebate.org.uk

Why do people answer questions?

“Please have a look at this survey”

The survey is a systematic method for gathering information from (a sample of) entities for the purpose of constructing quantitative descriptors of the attributes of the larger population of which the entities are members.

Groves, Robert M.; Fowler, Floyd J.; Couper, Mick P.; Lepkowski, James M.; Singer, Eleanor & Tourangeau, Roger (2004).Survey methodology. Hoboken, NJ: John Wiley & Sons.

The survey is a processfor gathering information from (a sample of) entities for the purpose of constructing quantitative descriptors of the attributes of the larger population of which the entities are members.

The survey is a processfor getting answers to questions from(a sample of) entities for the purpose of constructing quantitative descriptors of the attributes of the larger population of which the entities are members.

The survey is a processfor getting answers to questions from(a sample of) people for the purpose of constructing quantitative descriptors of the attributes of the larger population of which the entities are members.

The survey is a processfor getting answers to questions from(a sample of) people for the purpose of getting numbersof the attributes of the larger population of which the entities are members.

The survey is a processfor getting answers to questions from(a sample of) people for the purpose of getting numbersthat you can use to make decisions

To make decisions People

The survey is a process for getting answers to questions

getting numbers

The aim of a survey is to get a number that helps you to make a decision

“The numbers” or “The number”?

Fieldwork used to be expensiveso a survey was a rare event.

Image credit: http://www.census.gov/history/www/genealogy/decennial_census_records/

1950s mindset: “Ask Everything”

Survey = Big Honkin’ Survey

2016 mindset: the Light Touch survey• Choose ONE question• Find ONE person• Ask the question, face-to-face• See if you can make ONE decision• Improve, iterate, increase

Tip Interview first

One way toiterate, improve,increase

Time for new question

Caroline Jarrett @cjforms (CC) BY SA4.022

The aim of a survey is to get a number that helps you to make a decision

Asking the right question

Asking the right people

Caroline Jarrett @cjformshttp://www.bbc.com/news/10506482 27

Asking the right people is better than asking lots of people Sample:

the list you sample from

There’s often a ‘zone of indifference’

Hate it Love it

Feeling special is rewarding

Ask fewer people to get better response

Sample: the number of people to ask

Try mail, phone or face-to-face for better response rates

Fieldwork:Who answers?

Did we get answers from the right people?

Is this sample representative?

Image credit: Caroline Jarrett / CorelDraw

Population of assorted birds

Is this sample representative?

Image credit: Caroline Jarrett / CorelDraw

We don’t just want answers, we want representative answers

Response

Asking the right question

Asking the right people

Asking one person the right question

is better thanAsking 10,000 people the wrong question

In your last five days at work, what percentage of your work time do you estimate that you spent using publicly-available online services (not including email, instant messaging, and search) to do your work using a work computer or other device?

Helps a lot if you ask good questions

Questions:What are you asking about?How many questions?

Response relies on effort, reward and trust

A good question works in three ways

Obvious Interesting

Appropriate

Why did you visit our website today?

Obvious Interesting

Appropriate

Would you recommend us to a friend or family member?

In a shop, buying a baby buggy

In a hospital,having a miscarriage

Obvious YesInteresting YesAppropriate Yes Cruelly

inappropriate

Would you recommend us to a friend or family member?

In a shop, buying a baby buggy

In a hospital,having a miscarriage

Obvious YesInteresting YesAppropriate Yes Cruelly

inappropriate

TipTest your questions by interviewing in context

We don’t want any old answers. We want useful answers.

Response

In your last five days at work, what percentage of your work time do you estimate that you spent using publicly-available online services (not including email, instant messaging, and search) to do your work using a work computer or other device?

On your most recent working day,what percentage of time did you spend using the Internet?

The approximate curve of forgetting

Major life event

Noticeable, occasionalUnremarkable,repetitive

Perfectrecall

DayGone

Week Month YearHour

Can’t remember = can’t answer

Response

Insights are the answers that you use to make the decisions

Insights Caroline Jarrett @cjforms (CC) BY SA-4.053

Caroline Jarrett @cjformsCaroline Jarrett @cjforms (CC) BY SA-4.054

"Phone photography" by Petar Milošević - Own work. Licensed under CC BY-SA 3.0 via Commons - https://commons.wikimedia.org/wiki/File:Phone_photography.jpg#/media/File:Phone_photography.jpgModified by Caroline Jarrett

TipAlways allow for ‘other’

Design by @RickyBuchanan; t-shirt from nopitycity.com or zazzle.co.uk

Can we make the decision yet?

Caroline Jarrett @cjforms (CC) BY SA4.057

TipTry writing your presentation first

We’ve met the Survey Octopus

What could possibly go wrong?

Caroline JarrettTwitter @cjforms

http://www.slideshare.net/cjforms

carolinej@effortmark.co.uk

The Survey Octopus - getting valid data from surveys

Design

Transcript of The Survey Octopus - getting valid data from surveys

OCtoPus - 2CRSI · About 2crsi OCtoPus Solution Technical Specifications OCtoPus servers Remark ll specifications and photos are subect to change whitout notice. OCtoPus Solution

Fatty Acid Composition of the Common Octopus, Octopus Vulgaris

Octopus Yacht

Dumbo Octopus

Table of Contents - Octopus · i. Connect the Octopus Mobile Reader and tap “Enquiry”. ii. Put your Octopus on the “Octopus Mobile Reader”. iii. Please do not remove the Octopus

GOURMET OCTOPUS - Arista Industriesaristaseafood.com/.../2017/03/Pesfasa_Sell_Sheet_Octopus_021317.pdf · GOURMET OCTOPUS PRODUCT OVERVIEW Pesfasa® octopus (Octopus vulgaris) is

OCTOPUS DIGITAL

Portfolio Octopus

Octopus gift

Octopus brochure

Octopus 300 Octopus Perimetry you can trust 300. Octopus 900 11. Octopus ® 300 Octopus ® 900 Octopus ® 300 & 900. HAAG-STREIT AG. Gartenstadtstrasse 10. 3098 Koeniz, Switzerland

Installation: OCTOPUS 8M, OCTOPUS 16M, OCTOPUS 24M, OCTOPUS … · 2020. 2. 24. · Installation OCTOPUS 8M/16M/24M/OS32 Release 17 02/2020 3 Inhalt Sicherheitshinweise 5 Über dieses

OctOpus Vocalis

Octopus Card

The Octopus

Day Octopus - Ocean WiseDay octopus is a medium sized, diurnally active octopus that is thought to be the most common octopus species on reefs worldwide and it is the dominant octopus

Rapid Assessment, Surveys and Surveillance - ASEAN assesment.pdf · Rapid Assessment, Surveys and Surveillance ... reestablishment of infrastructure, ... - valid cluster sample survey

Octopus storytime

Octopus habitats

Octopus bay