Sentiment Visualization Widgets for Exploratory Search
-
Upload
carnby -
Category
Data & Analytics
-
view
321 -
download
1
description
Transcript of Sentiment Visualization Widgets for Exploratory Search
Sentiment Visualization Widgetsfor Exploratory Search
Eduardo Graells-GarridoUniversitat Pompeu FabraBarcelona, Spain
Mounia LalmasYahoo LabsLondon, UK
Ricardo Baeza-YatesYahoo LabsBarcelona, Spain
1st Social Personalization WorkshopSeptember 1st, 2014Santiago, Chile
Introduction: Context
ExploratorySearch
Sentiment Analysis
HCI
Our work!We use sentiment
visualization widgets for exploratory search
no concrete taskno expertise
big and complex information spaces
faceted search needs structure
positivity, negativity, ambivalence of textcan be visualizedunderstood by userscan be estimated from data
user interfacesindividual differences
information visualization
Research Questions
do visual approaches foster exploration in a sentiment-based exploratory search setting?
we propose design guidelines for visualization widgetsand we evaluate them in a pilot study
who benefits from visual approaches?not all users are equal!
we propose a simple way to identify user archetypes
Background
Sentiment Analysis
> 50% positive reviews depicts a “fresh” movie, “rotten” otherwise.Is sentiment unary(0 to 1), binary (pos/neg) or trinary (pos/neu/neg)?
Sentiment Analysis: Ambivalence
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank [Socher et al., 2005]
Sentiment Analysis: Ambivalence
Visualizing Ambivalence, showing what mixed feelings look like [Panger et al., 2013]
Faceted Search
We can have facets to enable exploratory search. Key points:1) facets need structure. 2) sentiment is not displayed with ambivalence.
We Feel Fine: Faceted Search with Sentiment
We Feel Fine [Kamvar and Harris, 2011] is mostly an art project. It’s cool! But we want to see the effects of visual interfaces in more general contexts.
Visualization Widgets for Exploratory Search
Visgets: Coordinated visualizations for web-based information exploration and discovery [Dörk et al., 2008]
Visualization WidgetsDesign Guidelines
Sentiment Visualization Widgets - Guidelines
Depict ambivalenceAmbivalence is present in many categories and genres of textual content, including public discourse, fiction and news articles. Sentiment is not a simplistic dimension!
Show sentiment distributionHow did the U.S. feel when Obama was elected?
Allow sentiment filteringHighlight/discard sentiment values of (no) interest (show me the bad reviews only*)
*I actually do that ;)
Widgets
Scatterplot Parallel Coordinates
Widget I: Scatterplot
Widget: Parallel Coordinates
Dataset
Dataset: Wikipedia and DBPedia
WikipediaOpen Encyclopedia with a “Neutral Point of View”, but not emotionless content. Positive and negative things do happen and are documented.
DBPedia OntologyShallow hierarchy of 405 classes assigned to each Wikipedia article.
Dataset737863 articles annotated with sentiment.Source: [Mejova et. al, 2013].
DBPedia: Agent > Person > Artist
Aggregated Sentiment in Biographies
Sentiment Distribution
There is sentiment in Wikipedia.We can use it as a facet for exploration!
Evaluation
Pilot User Study
HypothesisIn exploration on sentiment-based scenarios, participants perform more queries and spend more time when using visualisation widgets
Treatments: Scatterplot, Parallel Coordinates, Text Baseline
13 Users recruited from an open call in social networks (within-subjects design).(5 male, 8 female, 3.46 [of 5] average score of knowledge in web search)
Three standard exploratory search tasks with an introduced sentimentality component. In different contexts, users had to find sets of items with some specific requirements (both sentiment and not sentiment related).
Variables
System LogsNumber of queries issuedTask completion time (seconds)
User FeedbackOpen-text questions about the user interfacePerceived time of task completion (minutes)Aesthetic value of the user interface (Likert 1 to 5)
Cognitive EngagementDifference between real task completion time and perceived time of task completion. A positive value indicates a positive experience, as in “time flies!”
Treatments
Scatterplot Parallel Coordinates Text-based
Results
Results are not significant, except on task time.Post-hoc testing: when using the Scatterplot people spends more time.
Is that positive or negative? How to find the answer?
Individual Differences in BehaviorWe split users into explorers and achievers, inspired by MUD Games [Bartle, 1996]:
Explorers those who “interact” with the
worldAchievers
those who “act” in the world
We classified users considering the geometric mean of queries and time. Lower 50% -> achievers.
Explorers have positive engagement when using those approaches (and much more when using SC than BA).
Open FeedbackBaseline
boring [P8], easiest for me to find results [P4]. filters were really easy to use [P3]
I think the most useful one is the buttons one because it has more precise information reflected on it. [P10]
The one with the numbers was misleading for me [P6].
Scatterplotattractive [P8]
[BA] and [SC] are easy to use. They are helpful and easy to understand [P3]. But... needs more concentration [P6].
this is the task that I enjoy the most! I liked pretty much the graphics [P8]
this is the approach I liked the most, it was easier to filter the results [P9]
Parallel Coordinatesinteresting [P8], cooler than the other ones [P1]
helps me to know if it is positive or negative faster. I really like how [PC] worked [P6].
[PC] was not appealing nor easy to understand or use [P3]. It's confusing [P11]
Visual approaches seem to be more emotional.
Baseline seems to be more utilitarian.
Conclusions
Discussion
Do visual approaches foster exploration? Partially.
Scatterplot showed significant differences in task time (users spent more time using the system).
For whom? Explorers.
Longer task time can be explained by positive user engagement... when users are explorers.
Qualitative feedback is favorable to visual approaches. It is more emotional (joy), while the baseline is perceived as more utilitarian (easier to use, it’s familiar).
Discussion
Individual differences can be identified from user models.
Since those users can be identified (i.e., query logs and user modeling), it is possible to personalize user interfaces for explorers, i.e.,
If users are explorers, show a scatterplot as widget. Otherwise, show the text-buttons widget.
Our taxonomy was simple enough to find a significant difference, however, there are more complex taxonomies that are worth to explore in future work (fast surfers, broad scanners and deep divers [Heinström, 2002]).
Conclusions
Visualization techniques in user interfaces have potential to engage users in different ways than current text-based interfaces.
No two users are equal! This should be accounted when doing:
Experimental design which variables are likely to help discriminate between user categories?
System design which user interface is more likely to engage and enhance the user activity?
Simple rules to identify individual differences are enough to find which visualization widget is better in some cases.
Questions and Comments are Welcome!
Thanks for attending! :)
Special ThanksWorkshop Organizers
Anonymous ReviewersParticipants in the User Study
Ilaria Bordino and Yelena MejovaLuca Chiarandini
https://www.flickr.com/photos/blackham/97529032