The Future of Vertical Search Engines

Ted DRAKE

Yahoo! France

WWW2009, Madrid

22 April 2009

Yahoo! BOSS

More Freedom - More Data - More Control

Vik Singh - Yahoo! BOSS Architect

Yahoo! Query Language

• SQL-like syntax

• Treats any API as a data table

• RSS, XML, HTML, Spreadsheets, and more

• oAuth gives access to personal information

Google App Engine

• Development Platform

• Python, Java

• Free for most uses

• Fast Product Development

oAuth

• Open Standard

• Grant Limited Access to Private Information

• Username/Password are NOT Shared

• Yahoo!, Google, Twitter, Pownce, Bebo, Flickr, Digg, MySpace…

Yahoo! Developer NetworkDeveloper.Yahoo.Com

Documentation:

• SDK

• API

• YUI shared Libraries

• Design Patterns

• Yahoo! And Open Standards

Past - Present - Futureof Search Construction

0

10

20

30

40

50

60

70

80

Past Present Future

DiscoveryRelevancyDesign

Future of Relevancy

Location Based Relevancy

•Where am I?

•Where am I going?

•What can I find?

Map generated by FirePin application on iPhone

Location Based Relevancy

• Fire Eagle: Standardized location and sharing platform

• Live location tracking• Shared locations with friends• Mining Interesting Locations and Travel

Sequences from GPS Trajectories for Mobile Users by Yu Zheng, Lizhu Zhang, Xing Xie and Wei-Ying Ma

1. Blah2. Foo3. Blah Blah

Secondary SourcesWikipedia, Craigslist, Government Data…

1. Baz2. Bar3. Foo

1. Foo

• Multiple sources to increase relevance

• DuckDuckGo.com = BOSS + Wikipedia (and other services)

• Understanding User's Query Intent with Wikipedia by Jian Hu, gang wang, Fred Lochovsky and Zheng Chen

•OpenData: DataMob.org, TheInfo.org, InfoChimps.org

Real Time Events

• Tweet News: Twitter + News Search

• Twitter users share most timely articles

• Relevancy highlights tweeted stories

Internal + External Data Sources

BOSS

• Tech Crunch Search: BOSS + Access to proprietary data

• Create custom tables in YQL

• Quicklink Selection for Navigational Query Results by Deepayan Chakrabarti, Ravi Kumar and Kunal Punera

BOSS “Vertical Lens” defines what internal data BOSS should index as well as your preferred external sources.

Offline Analysis

Coloralo requests extra images, caches them, and analyzes them for relevancy.

Coloralo finds coloring book images.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Vertical Focus

• Vertical Search Engines already have a niche audience.

• Limit searches to appropriate sites: InsiderFood

• Truevert creates a model of word relations in context to its niche: environmental.

Go Beyond the Web Site

• Desktop: Xobni for Outlok

• Tools: Zemanta finds related information for blogs and emails

• Modular: Create an application for Facebook, Yahoo, MySpace and more with the Open Social standard.

Go from Search to Action

• Keyword Finder uses BOSS keyterms to return the top 10 keywords used by successful sites for a query

• Bossy returns a single answer to questions. Where is the Prado? Madrid.

Resources

• Yahoo! BOSS: http://developer.yahoo.com/boss• YQL: http://developer.yahoo.com/yql• Fire Eagle: http://developer.yahoo.com/fireeagle/• Google App Engine: http://appengine.google.com• Amazon Web Services: http://aws.amazon.com • oAuth: http://oauth.net/• Open Social: http://www.opensocial.org/• Open Data: http://theinfo.org • Alt Search Engines: http://www.altsearchengines.com/