Using Metadata to Improve Search User Interfaces Marti Hearst UC Berkeley FLINT Workshop, August...

Using Metadata to Improve Search User Interfaces

Marti Hearst

UC Berkeley

FLINT Workshop, August 2001

My Claims

Web Search is OK Gets people to the right starting points

Web SITE search is NOT ok The best way to improve site search is

NOT to make new fancy algorithms Instead …

The best wasy to improve search:

Improve the User Interface

Recent Study by Vividence Research

Spring 2001, 69 web sites The most common problems:

53% had poorly organized search results 32% had poor information architecture 32% had slow performance 27% had cluttered home pages

Organizing Search Results

There is a lot of prior work on this Cha-Cha (Chen et al. 1999)

Scatter-Gather clustering (Cutting et al. 93, Hearst et al. 1996)

Becoming more prevalent in web search too. Teoma Visimo Northern Light

Putting Results into Categories

Usually doesn’t work well when drilling down, however

Web Search Results Grouping

Drill down one category Cannot mix and match categories Probably isn’t all that useful But …what about site search?

A Taxonomy of WebSites

Catalog SitesWeb-based Information

Systems

Web-Presence Sites

Service-Oriented Sites

low

low

high

high

Complexity of Applications

Complexity of Data

From Mecca et al.,WebDB’99

An Important Trend

Web sites generated from databases Implications:

Web sites can adapt to user actions Web sites can be instrumented

Navigation on the Web

Web search engines are good at getting people to the right site.

But … what happens when the user reaches the site?

Follow Links … or …

Search

An Analogy

text searchhypertext

Goal: An All-Tertrain Vehicle

The best of both techniques A vehicle that magically lays down track to

suggest choices of where you want to go next based on what you’ve done so far and what you are trying to do

The tracks follow the lay of the land and go everywhere, but cross over the crevasses

The tracks allow you to back up easily

New interfaces are mixing and matching thesaurus-style metadata

Time/Date Topic RoleGeoRegion

The question: how to do this effectively?

Goals for Metadata Usage

Well-integrated with search Provides useful hints of where to go next Tailored to task as it develops Personalized Dynamic

The FLAMENCO ProjectFLexible Access using MEtadata in Novel Combinations

Main goal: Perform systematic studies to determine how

metadata should be incorporated into search Answer questions such as:

Given a set of user goals and a set of information: How many metadata combinations to show? What level of detail to show? How best to preview and postview choices?

Evaluation Methodology

Regression Test Select a set of tasks

Use these throughout the evaluation Start with a baseline system

Evaluate using the test tasks Add a feature

Evaluation again Compare to baseline Only retain those changes that improve results

Recipe Example

soar.berkeley.edu/recipes

www.epicurious.com

Epicurious Metadata Usage

Advantages Creates combinations of metadata on the fly Different metadata choices show the same information in

different ways Previews show how many recipes will result Easy to back up Supports several task types

``Help me find a summer pasta,'' (ingredient type with event type), ``How can I use an avocado in a salad?'' (ingredient type with dish type), ``How can I bake sea-bass'' (preparation type and ingredient type)

Metadata usage in Epicurious

PrepareCuisineIngredient Dish

Recipe



PrepareCuisineDishISelect



I >

Group by

PrepareCuisineDish



PrepareCuisineDishI >

Group by



PrepareCuisineDishI >

Group by

PrepareCuisineISelect

Metadata Usage in Epicurious

Can choose category types in any order But categories never more than one level deep And can never use more than one instance of a

category Even though items may be assigned more than one

of each category type Items (recipes) are dead-ends

Don’t link to “more like this” Not fully integrated with search

Epicurious Basic SearchLacks integration with metadata

The FLAMENCO PROJECTThe FLAMENCO PROJECTUsability Studies to determine how Usability Studies to determine how

to improve website searchto improve website searchMarti Hearst

Rashmi Sinha

Ame Elliott

Jen English

Kirsten Swearington

Ping Yee

http://bailando.sims.berkeley.edu/flamenco.html

The FLAMENCO Project

Usability studies to determine how to improve site search using metadata

Researchers: Marti Hearst, Rashmi Sinha Ame Elliott, Jennifer English, Kirstin Swearington,

Ping Yee So far:

Epicurious usability study Image search interface framework and study design

Epicurious Usability Study 9 participants so far

Independent Variables: 1) Epicurious Interface (Basic vs. Enhanced vs. Browse) 2) Task type (known-item search vs. browsing for inspiration) 3) Degree of constraint of query 4) Number of results required (1 vs. many)

Dependent Variables: 1) Time to find satisfactory recipe(s) 2) Navigation path (backtracking, starting over, revising queries) 3) Satisfaction with results of search 4) Satisfaction with individual system features (e.g. breadcrumbs,

query previews, refine by hyperlinks) 5) Likelihood of using each interface in the future.

Epicurious Usability Study

Participants were asked to: Do 3 pre-specified searches in advance In the lab:

Specify a cooking scenario of interest to them Search for 3 recipes for this recipe Search for each recipe using each of the interfaces

Complete several structured tasks Along the way, answer questions about

Getting closer or farther away from goal Satisfaction with search results Satisfaction with the interace

Usability Study: Preliminary Results, Preference Data

Site Basic Enhanced BrowseTotal "Very Likely" to Use: 7 2 4 7

Total "Likely" to Use: 0 1 1 0Total "Not Likely" to Use: 2 6 4 2

PERCENTAGES Basic Enhanced BrowseVery Satisfied 32 43 35

Satisfied 50 43 52Middle 9 4 4

Dissatisfied 9 9 9

FavoriteSubject_JG: EnhancedSubject_NS: EnhancedSubject_SP: Browse

Subject_RM: Browse

Subject_LA: Enhanced

Subject_MC: BrowseSubject_MW: BrowseSubject_NM: EnhancedSubject_CG: Browse

Query previews and navigation. Options to refine by course or season. Choose how you view the results

Searching within made all the difference. I could see how many results I was getting in each Very specific. I can choose more than 1 detail with search for recipe I'm looking for.Likes the way it narrows things down. And it gives you the numbers.

Found it simpler, more readable. Helped you hone in on the season.Liked the kid friendly, low fat optionWhy?

Can narrow down when you're stuck. You can always refine [your search].

Allowed me to make specific selections. I liked Browse too. Gave lots to choose from. Depends on what you’re looking for that day

Can limit and unlimit and limit again in a different way. Prioritize your criteria--change the first thing I clicked and go in a different direction. Easy to back up.

Usability StudyPreliminary Results: Feature Preference

Subject_SP Subject_RM Subject_LA Subject_MC Subject_MW Subject_NM Subject_CGQuery previews +1 DN +1 0 +1 +1 +1

Having a complete list of ingredients (enhanced and browse) +1 +1 +1

enhanced - 0, browse - +1 +1 +1 +1

Search within results +1 +1 0 +1 +1 +1 +1Refine using hyperlinks (browse) +1 +1 DN +1 +1 DN +1Set all criteria from one screen (enhanced) 0 +1 +1 -1 0 +1 -1"May include" and "must include" options 0 0 DN 0 +1 +1 0"All words," "any words," "exact phrase," and "Boolean" options 0 0 0 DN DN +1 DN

KEY+1 - Helpful0 - Not helpful-1 - InterferedDN - Didn't notice

Usability StudyPreliminary Results: Quantitative

Search Metrics Basic Enhanced Browse

No Facets / Keywords per search

Total per search 4.3 7.7 ~2.11

During Step 1 of Search 2.3 always 1

For Each Step of Search 2.2 3.5 ~1

Ave. times 0 Results

During Step 1 0.0% 15.8% 0.0%

Overall 12.2% 31.4% 0.0%

Ave. Time ( in seconds)

Time per Search 98.6 130.3 108.11

Time per Step 50.9 60.6 41.38

Ave. No Steps 1.86 2.1 2.81

Median No Results

Overall (excluding 0 results) 186 17 704

in Step 1 13 10 579

In Final Step 32 14 13

Usability StudyPreliminary Results: Constraint-based Preferences

# of Results High LowEnhanced (LA) Browse (LA)Enhanced (MC) Browse (MC)Browse (MW) Browse (MW)Enhanced (NM) Enhanced (NM)Basic (CG) Browse (CG)Enhanced (LA) Browse (LA)Enhanced (MC) Browse (MC)Enhanced (MW) Browse (MW)Enhanced (NM) Enhanced (NM)Enhanced (CG) Browse (CG)

Constraint

1 result needed

Many results needed

Usability Study Results: Summary

People liked the browsing-style metadata-based search and found it helpful

People sometimes preferred the search-style metadata search when the task was more constrained But zero results are frustrating This can be alleviated with query previews

People disprefer the standard simple search

Application to Image Search

Image Search: What is the task?

Illustrate my slides? “Find a crevasse” Keyword match works pretty

well Find inspiration for an

architectural design? Needs richer search support

Faceted Metadata

Planalto Palace Parti Communiste Francais Pantheon

Oscar Neimeyer Oscar Neimeyer Jacques-Gabriel Soufflot

20th Century 20th Century 17th & 18th C.

Brasilia Paris Paris

Stone Curvilinear Stone

Image:

Architect:

Period:

Location:

Concept:

Planalto Palace Parti Communiste Francais Pantheon

Oscar Neimeyer Oscar Neimeyer Jaques-Gabriel Soufflot

20th Century 20th Century 17th & 18th C.

Brasilia Paris Paris

Stone Curvilinear Stone

Image:

Architect:

Period:

Location:

Concept:

Faceted Metadata

SPIRO Query Form

SP

IRO

que

ry o

n S

ubje

ct: c

hurc

h

Results of a pilot study with Archictects: Metadata is Helpful

Very positive feedback about the general approach All 9 participants named the metadata in

the search results area as their favorite aspect of Flamenco

Metadata was successful at giving hints about where to go next Perceived as useful “These are places I

can go from here.”

Results: More Metadata Please

Participants asked for more metadata Although there were complaints about the contents of

the metadata, users still wanted more Longer lists of options (more hints) Users wanted more control to make very specific searches

Half the participants requested the ability to control order of results with metadata Juxtapose visible images 2 different ways:

Overview (one image from each project) vs. like together ( all images of a project next to each other)

Different than ranking for text retrieval (precision, recall), but ordering does matter

Results: Complaints The UI was not successful at clarifying

searching within results vs. starting a new search Only 2 of the 9 participants understood the

distinction without discussion – but they want to do both

The 1/3 of the participants who couldn’t find a treasure hunt image felt that Flamenco was slow Corroborates findings that perceived system

speed is about finding what you want (Spool ‘00)

Summary Web site search needs improvement

Users want more organized results Our approach: integrate navigation with search

Metadata is being mixed and matched in interesting ways, but there are no guidelines on what works We are investigating how to design websites containing large sets

of items Architectural images Biomedical text

Preliminary results indicate that metadata organization is useful in some situations Depends on the type of search need

SummaryOur goals

Systematically determine what works, with the following emphases:

Task-centric Integrate metadata with search Dynamic previews Easily retrace steps

Develop recommendations that reflect both the task structure and the richness of the information structure

In future: integrate with more sophisticated displays

http://bailando.sims.berkeley.edu

Using Metadata to Improve Search User Interfaces Marti Hearst UC Berkeley FLINT Workshop, August...

Documents

Transcript of Using Metadata to Improve Search User Interfaces Marti Hearst UC Berkeley FLINT Workshop, August...