Individual Differences, User Perceptions and Eye Gaze in Biomedical Search Interfaces
-
Upload
ying-hsang-liu -
Category
Technology
-
view
415 -
download
0
Transcript of Individual Differences, User Perceptions and Eye Gaze in Biomedical Search Interfaces
Individual Differences, User Perceptions and EyeGaze in Biomedical Search Interfaces
7 September 2015
Ying-Hsang Liu 1,2
1School of Information Studies
Charles Sturt University
2Research School of Computer Science
The Australian National University
2Introduction
Interactive InformationRetrieval (IIR)
▶ Current IR systems designed forspecified search (Belkin, 2008)
▶ Natural search userinterfaces (Hearst, 2011)
▶ Usefulness of controlled indexinglanguages (Salton, 1972; Svenonius, 1986)
Medical Subject Headings(MeSH) terms
3Research Questions
Research questions
▶ What elements of searchinterfaces do searchers look atwhen searching for documentsto answer complex questions?
▶ What is the relation betweenuser perceptions of aninterface and the interfaceelements they look at?
▶ What is the relation betweenindividual differences and theinterface elements which arelooked at?
User experiment in alaboratory setting
8Test Collection
Selection of search topics
▶ Document test collection fromOHSUMED (Hersh, Buckley, Leone, & Hickam, 1994)
▶ MEDLINE from 1987 to 1991;348,566 records
▶ Randomly select 8 topics basedon proportion of judged relevantdocuments
▶ 2 topics from each of thequartiles (4 search topic pairs)
Sample search topic
▶ ID: 78▶ Imagine that you are 42-year-oldblack man with hypertension.
▶ You would like to findinformation about beta blockersand blacks with hypertension,utility.
9Experimental Design
Factorial design
▶ 4× 2× 2 Factorial design; 4interfaces, controlled searchtopic pairs and cognitive styles
▶ 4× 4 Graeco-Latin Square toarrange experimental conditions
▶ Power Analysis for ANOVADesign; medium effect size of.25, α < .05 and N = 256,statistical power of .93 (Cohen, 1988;
Faul, Erdfelder, Lang, & Buchner, 2007)
4× 4 Graeco-Latin Square
10Software and Hardware
Experimental system setup
▶ Experimental search systembased on Solr
▶ Gaze tracking uses FaceLabsoftware and hardware
▶ EyeWorks for data recordingand analysis
▶ Emotiv headset for EEG data▶ Search logs and mouse clicksrecorded
Gaze tracking by FaceLab
11Experimental Procedure
Experimental procedure Data collection▶ User characteristics (backgroundquestionnaire and cognitive styletest)
▶ User perceptions (exitquestionnaire)
▶ Search behaviours (search logs,mouse clicks and documentssaved)
▶ Physiological signals (eye gazeand EEG)
12Searcher Characteristics
▶ 32 subjects; male (50.0%), female(50.0%)
▶ Student: postgraduate (46.9%),undergraudate (40.6%)
▶ Age: 18–25 (59.4%), 25–35(28.1%)
▶ Online database experience: < 5years (62.5%), 5–10 years(21.9%)
▶ Search engine: every day(50.0%), several times a day ormore (37.5%)
▶ Pilot study (Liu, Thomas, Schmakeit, & Gedeon, 2012)
Biology background
13Searcher Characteristics (cont’d)
▶ Cognitive style: Individual’spreference or tendency toprocess information
▶ E-CSA-WA (Extended CognitiveStyle Analysis–WholisticAnalytic) test (Peterson, Deary, & Austin, 2003)
▶ Wholistic Analytic Ratio▶ WA ratio (M = 1.31, SD = .24);cut-off = 1.32 (Clewley, Chen, & Liu, 2010; Chen,
Magoulas, & Macredie, 2004; Yuan, Zhang, Chen, & Avery, 2011)
E-CSA-WA Test
14Data Analysis
▶ Where do people look? Area ofinterest (AOI)
▶ Logarithmic cross ratio analysisbetween individualdifferences/user perceptions andAOI (Fleiss, Levin, & Paik, 2003; Saracevic, Kantor, Chamis, &
Trivison, 1988)
▶ ANOVA between interface andsearcher characteristics, such ascognitive style and searchexperience
Heat map and AOI
15Search Interfaces and AOI
Title Author Abstract MeSH
●
●
●●
●
●●●
●● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
0
25
50
75
A B C D A B C D A B C D A B C DTypes of Interface
Prop
ortio
n of
fixat
ions i
n AO
I
16User Perceptions and AOI
Table: Summary of the relation between user perceptions and AOI
Difficulty Usefulness Notice of Keywords Use of Keywords
B C D B C D
Title m m l l l l l —Author m — m m m m m m
Abstract m l — — — — — l
MeSH m — — — — — — —
Note. The relation is not statistically significant (—), positively significant (l), ornegatively significant (m) at 95%).
17Individual Differences and AOI
Table: Summary of the relation between individual differences and AOI
Domain Knowledge Search Experience Cognitive Style
UG PG Search Engine OnlineDatabase
Title m m — — —Author — — l — —Abstract — — m — —MeSH — — l — —
Note. The relation is not statistically significant (—), positively significant (l), ornegatively significant (m) at 95%).
20Summary and Discussion
Research findings
▶ Searchers look at abstract moreoften than other interfaceelements
▶ Interfaces and user perception ofsearch task difficulty significantlyaffects elements look at
▶ Significant interaction effectbetween cognitive style/searchexperience and interface forMeSH AOI
Discussion▶ Design of Search Engine ResultsPage (SERP)
▶ Detection of search taskdifficulty
▶ Individual differences for searchuser interface design