Internet Resources Discovery (IRD)

Meta-Search Engines (MSEs)

T.Sharon-A.Frank2

Contents• Meta Search Engine (MSE)• Why use several SEs?• Highlighted MSEs• Hebrew MSEs• MSE comparison• When to use MSE – pros and cons• How to choose MSE?

T.Sharon-A.Frank3

Search Engines Generations

• 1st Generation - Basic SEs:

• 2nd Generation - Meta SEs:

• 3rd Generation - Popularity SEs:

T.Sharon-A.Frank4

2nd Generation SEs - MetaSEs

• Using several SEs in parallel.• The results are filtered, ranked and

presented to the user as a uniformed list.• The ranking is a combination of the

number of sources each page appeared in, and the ranking in each source.

T.Sharon-A.Frank5

Meta SE is a Meta-Service

• It doesn’t use an Index/database of its own.

• It uses other external search services that provide the information necessary to fulfill user queries.

T.Sharon-A.Frank6

Meta Search Engine

MetaCrawler

Yahoo Web Crawler Open Text Lycos InfoSeek Inktomi Galaxy Excite

Google · Yahoo · Jeeves Ask About · LookSmart · OvertureFindWhat

T.Sharon-A.Frank7

Premises of a Meta SE

• No single search is sufficient.

• Problem in expressing the query.

• Low quality references can be detected.

T.Sharon-A.Frank8

Why use Several SEs?• Search Engines differ more than we think!

T.Sharon-A.Frank9

Overlap between Google and Yahoo

Source: Jux2 analysis of 500 top search terms, April 2004

http://www.jux2.com/stats.php

T.Sharon-A.Frank10

Who Overlaps Whom?

T.Sharon-A.Frank11

Try it yourself @ jux2

T.Sharon-A.Frank12

MSE - Motivation

1. The number and variety of SEs.

2. Each SE provides an incomplete snapshot of Web.

3. Users are forced to try and retry their queries across different SEs.

4. Each SE has its own interface.

5. Irrelevant, outdated or unavailable responses.

6. Each query is independent.

7. No individual customization.8. The result is not homogenized.

T.Sharon-A.Frank13

Problems of MSEs

• No advanced search options.

• Using the lowest common denominator.

• Sponsored results from the SEs are not highlighted.

T.Sharon-A.Frank14

Highlighted MSEs

T.Sharon-A.Frank15

Mamma

T.Sharon-A.Frank16

Dogpile

T.Sharon-A.Frank17

Dogpile Advanced (1)

T.Sharon-A.Frank18

Dogpile Advanced (2)

T.Sharon-A.Frank19

Dogpile Advanced (3)

T.Sharon-A.Frank20

Dogpile Advanced (4)

T.Sharon-A.Frank21

Dogpile Preferences (1)

T.Sharon-A.Frank22

Dogpile Preferences (2)

T.Sharon-A.Frank23

Vivisimo• Vivísimo supports the most advanced features

of the major search engines.• Need to just use Vivísimo syntax, which

follows the most standard conventions. • Vivísimo translates your query into the

corresponding syntax of each underlying search engine.

• Also, Vivísimo only queries the search engines that support your chosen syntax.

T.Sharon-A.Frank24

Vivisimo Advanced (1)

T.Sharon-A.Frank25

Vivisimo Advanced (2)

T.Sharon-A.Frank26

Clusty

T.Sharon-A.Frank27

Ixquick (1)

T.Sharon-A.Frank28

Ixquick (2)

T.Sharon-A.Frank29

Ixquick (3)

T.Sharon-A.Frank30

KartOO – Visual MSE

T.Sharon-A.Frank31

MetaSEs in Hebrew

T.Sharon-A.Frank32

When to use a MSE?• When single Basic-SE fails to provide good

results.

• One-stop shopping - prefer to search multiple SEs/sites at once to get blended ranked results (so as to save effort/time).

• Searching for multi-faceted topics.

• Want to get clustered results to focus search on the relevant keywords.

• Looking for current events/news.

T.Sharon-A.Frank33

MSE pros

• Useful when you want to retrieve a relatively small number of relevant results.

• An excellent choice for obscure topics.

• A good option when you are not having luck finding what you want when you search.

• Appropriate when you want to get an overall picture of what is available on the Web on your topic.

T.Sharon-A.Frank34

MSE cons

• Use is limited primarily to simple queries.

• Little or no field searching is available.

• Most services return a limited number of results that do not represent the total results from any source engine.

• Sponsored results are not highlighted (even though probably not first).

T.Sharon-A.Frank35

How to Choose your MetaSE

• Search engines used

• Operators supported

• Special features

• Speed

• Presentation

T.Sharon-A.Frank36

Meta-SEs Features Chart

Red – not working

T.Sharon-A.Frank37

Practical RecommendationsUse Ixquick for fast results and maximal

syntax flexibility.Use Vivisimo/Clusty (as a start) for

Clustering and/or Hebrew.Use Dogpile to include Google, date

range, or spelling corrections.Use none for non-MSE tasks

(see MSE cons)…

T.Sharon-A.Frank38

Bibliography• http://www.cs.washington.edu/homes/etzioni/papers/m

etacrawler.pdf• http://www.cs.washington.edu/homes/etzioni/papers/ie

ee-metacrawler.pdf• http://searchenginewatch.com/links/article.php/2156241

• http://vivisimo.com/advanced?form=Advanced• http://vivisimo.com/help.html• http://searchenginewatch.com/searchday/article.php/

2226841