Vision Critical’s Visual Questions

What, when and why to use these question types

Divider Page Example with Picture

Mapping Standard Question Types onto Visual Alternatives

Single Choice

3

STANDARD VISUAL

Single Choice Grid

4

VISUAL

STANDARD

Single Choice Grid #2

5

VISUAL (Magnetic Board)

STANDARD

Single Choice Grid #3

6

VISUAL (Card Sort)

STANDARD

Multiple Choice

7

VISUAL

* Combination of options with images and no images for demo purposes.

STANDARD

Multi Choice Grid

8

VISUAL

STANDARD

Multiple Choice Grid #2

9

VISUAL (Magnetic board)

STANDARD

Multiple Choice Grid #3

10

Rank Order

11

VISUAL STANDARD

Scale Question

12

VISUAL (Scale Slider)

STANDARD

Numeric Response Question

13

VISUAL (Numeric Slider)

STANDARD

Allocation (Constant Sum)

14

VISUAL (Allocation Slider)

STANDARD

Geographic Location

15

VISUAL STANDARD

Example text Go ahead and replace it with your own text. This is an example text.

Example text Go ahead and replace it with your own text. This is an example text.

Example text Go ahead and replace it with your own text. This is an example text.

16

Example text Go ahead and replace it with your own text. This is an example text.

Example text Go ahead and replace it with your own text. This is an example text.

Example text Go ahead and replace it with your own text. This is an example text.

16

Visual Questions that do what standard question types can’t

Page Turner

17

Media Player

18

Highlighter

19

Click-map

20

So how do these visual questions effect respondents?

engagement and data quality

21

We will go deeper on these question types

22

Engagement measures: Visual Question types vs. Flat Grid Questions

Multi-choice (MC) Single choice (SC)

Magnetic Board

Visual Grids (US)

Visual Grids

(Canada)

Scale Slider

Visual Grid (US)

Visual Grid

(Canada)

Easier to complete = = = = Fun to complete = = More enjoyable than most

= =

The Visual Grid and Multi Choice Magnetic Board question types were significantly more engaging than flat grid questions. Scale Sliders were equal.

Significantly higher rating, 95% CI

Equivalent rating, 95% CI =

Prior studies have also shown greater engagement

Papers presented at CASRO and ESOMAR showed a similar pattern of increased respondent engagement with visual question types, compared to flat question types.

Sliders and Visual Grids

25

Visual Grid

Scale Slider

Flat Single Choice Grid

Sliders and Visual Grids

Sliders and Visual Grids are used for rating exercises like attitudinal statements and brand ratings. Visual Grids (both single and multi choice) are rated as being more fun, easier and more enjoyable than radio button grids. Both Visual Grids and Sliders encourage people to use more of the scale, and less of the midpoint, than flat radio buttons, on a 10 point scale anchored verbally at the ends. On verbally anchored scales, much less variation is observed, suggesting the verbal anchoring makes an important difference.

Comparison of how 10 point brand association scales are used for Flat Single Choice Grid (radio buttons), Visual Grid and Slider Scales

Scale Points

Perc

ent

of

Res

po

nd

ents

US study Canadian study

Scale Points

In the US study, the differences seem to driven by a more varied use of the scale points by people answering the visual questions. With the flat grid version, many people were parking their votes on “5” and using “1” less often. In the Canadian study, the flat grid questions again gets more “5”s and shows a distribution skewed right somewhat, while the visual grid produced a more normal distribution

Note: different brands were rated in each country. These numbers represent the combination of the answers for all brands

Comparison of how 5 point verbally anchored scales are used for Flat Grids, Visual Grid and Slider Scales

With this verbally anchored scale we do not see any important variation between question types, suggesting that, in this case, the verbal anchoring drives people’s responses more than the question layout. The smaller number of scale points may also influence it.

[RICH MEDIA—SLIDER, 3a. We’re almost done. Please indicate how you feel about each of these stores, based on everything you know or have heard about them, either by clicking on the column under the rating or by dragging the checkmark to the column you choose. [FLAT—RADIO BUTTON GRID] 3b. We’re almost done. Please indicate how you feel about each of these stores, based on everything you know or have heard about them, by clicking on the column under the rating you choose.

Note: different brands were rated in each country. These numbers represent the combination of the answers for all brands

US study Canadian study

Card Sort

29

With attitude statements we see fewer “don’t knows”, with the card sort than with radio buttons

52 46

0

10

20

30

40

50

60

Card Sort Radio Buttons

3

11

0

2

4

6

8

10

12

Card Sort Radio Buttons

More people doing the card sort did not choose any “don’t know” responses

People doing the radio button survey were more likely to choose “don’t know” for all 24 statements

% with zero DK

% DK to all

Verbally Anchored Scale--Trustworthiness

With this verbally anchored rating scale we see slightly greater use of the midpoint and less use of the top box, with the radio button vs. the card sort

0

5

10

15

20

25

30

35

40

45

50

Completely Very Fairly Not Very Not at all

Card Sort Radio Button

We saw no difference in the amount of time required to do the card sort, compared to the radio button

Card Sort Usage Conclusions

Attitude statements are good candidates for use with card sorts. Ratings of single items is also a good use, as the data appears to be more varied. Card sorts also provide a welcome change of the type of task respondents are asked to do.

Multi Choice Magnetic Boards

Multi Choice Magnetic Boards

People enjoy the magnetic board exercise, rating it higher on “fun to complete” and “easier than most” compared to a radio button While conceptually similar to a visual grid or radio button grid, we find that the magnetic board question results in different, yet similar results. We conducted a head to head test of a magnetic board question vs. a radio button and a visual grid design, using three different arms of the study, each of whom used a different question type.

0%

10%

20%

30%

40%

50%

60%

70%

80%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51

Visual Grid Radio Button

Magnetic Board

Comparison of Multi-choice (binary) Brand Associations Using MC Grid (radio button), Visual Grid and Magnetic Boards

[RICH MEDIA—VISUAL GRID & SLIDER] Here is a list of words and phrases that might describe these stores, to varying degrees. Please indicate how closely you associate each word or phrase with each store, by either clicking on the column under the rating

or by dragging the checkmark to the column you choose. [FLAT—RADIO BUTTON GRID] Here is a list of words and phrases that might describe these stores, to varying degrees. Please indicate how closely you associate each word or phrase with each store,

by clicking on the column under the rating you choose.

When people used the Magnetic Board, the pattern of attributes chosen was very similar to that observed with the grids, but there were roughly half the number of attributes chosen.

These numbers represent the combination of the answers for all brands

Multi Choice Magnetic Boards: time spent and explanatory power

37

People spent as long on the magnetic type exercise as they did on the visual grid—even those they ended up choosing half the number of attributes Interestingly, when we modelled the strength of the attributes ability to predict overall liking of the brand, we found that the visual grid and magnetic type question types had the same level of explanatory power—even though there were half the number of attributes included.

Multi Choice Magnetic Boards: Conclusions

38

Multi choice magnetic boards are engaging, but people respond to them differently. So don’t use them to replace your flat multi choice grids questions in a tracker. But do think of magnetic boards for things like monadic brand association exercises, shopping exercises or the like.

Highlighter and Click Map: Unique data?

39

While the highlighter can obviously be used in in-person interviews in the real world, there is no other equivalent way of capturing this data. Click maps, however, have been compared to eye tracking, a relatively costly and laborious process.

Research on Click Maps Vs. Eye Tracking

40

When looking at total clicks versus eye movements, results are similar

Click Map Online

• Note: Heat mapping shows all clicks (up to 3 clicks per person) and all views (6 seconds). Online: Please look at the package below and click on the first three things that catch your attention. CLT: Please look at the products on the screen as you would when you’re out shopping.

High

Low

Traditional Central Location Eye Tracking

Highlighter and Click Map Conclusions

41

The data is qualitative, but the reporting is very clear

Click maps can replace eye tracking, in some uses.

When dealing with multiple languages, know that the highlighter results will need to be analysed separately

Conclusions

So what do we make of these question types?

We have seen

We see that visual question types tend to be more engaging We also see that they gather somewhat different data: data that we suggest is “better” because there is greater use of the full scale and fewer “don’t knows”. But in the case of something like magnetic type, we see that visual questions can capture data that is, in some ways, very different . And in the case of something like a click map, we find it can be very similar to something like eye tracking.

But the interesting question is: how do you see them being used? How might you use them in new and unique ways? Let’s have a go at creating a questionnaire using visual question types…

44