Web Search Update 2007

The Future Of Internet SearchChris Sherman (@cjsherman)Founding Editor

1

What Your Mom Thinks You Do

2

What You Really Do

3

To StartLets look back at search history, and see where the strikes came from4

Search In 19985

Googles Original Name

6

PerspectiveWeve entered the age of big dataThis is having an impact both on how we search, and also, perhaps more importantly, what we end up seeing as resultsNet, a good thing, but ironically may now subject us to information underload7

Its More Than Rocket Science NowIt takes about the same amount of computing to answer one Google search query as all the computing done - in flight and on the ground for the entire Apollo space programNot just the actual flights, but all the computing done throughout the planning and execution of the 11-year, 17 mission Apollo program- Udi Manber and Peter Norvig8

Google Data Centers: Then

Google Data Centers: Now

10

The Big TrendsGoogle continues to improveBing continues to gain (slight) share, but is still far from a threat to Big GSocial and big data are now paramount in searchThus, beware the filter bubbleAnd, maybe, a promising new beginning for Yahoo

11

No QuestionGoogle is getting smarter But has it lost its mind?Google+ integrated into everythingGoogle- : more than 75 product eliminations over the past 2 yearsNet: CEO Larry Page is reshaping Google from something we know and love -into something we cant know and may not love

And Search Is ChangingSearch isnt just about finding documents any moreSearch has evolved into an interactive process that helps:Answer a questionSolve a problemAccomplish a task13

The Major ChangesSmall, but helpfulChanges to search result pagesAuthor profilesHuge, and maybe (or maybe not) helpfulSocial now woven deeply into all things GoogleA knowledge graph that seeks to make Google truly intelligent

14

How Search WorksCrawling the webCreating an indexApplying algorithms to organize indexed contentUsing signals to surface the most probable results for your query15

Ranking SignalsLinks from other, authoritative sourcesSearch Engine Optimization (SEO)Your web history & locationMatching keywords in the title, URL & body of a web pageEmphasis: Lists, italics or boldface, headers, etc.400+ more16

Search Engines Are Pruning ResultsThe big data problemBased on game theory (think chess), pruning seeks to eliminate rather than expand choicesWhile this can lead to really good results (depending on the query), it also necessarily excludes millions (or even billions) of potential results17

For Perspective

18

Some ConsiderationsIf the web is made up of more than 60 trillion pages, that means that 99.999999% is effectively invisible, even if included in Googles indexGoogle cannot crawl and index some things that are accessible online:Some Facebook/Twitter dataMost database powered web sites19

Social: The ImplicationsGoogle is putting emphasis on trust and identityThis is a new type of ranking signal, and one that will grow in strength over timeIts already having a major impact on both search results and ads - Up to 20% of results for some queries may be social results20

PersonalizationGoogle now personalizes all search results, whether youre logged in or notIn addition, at any given moment there are 50-200 different versions of the core algorithm out in the wildConclusion: There are no standard Google search results any more21

Conflicting Signals?Google+ and personalization are creating filter bubbles for all of usThis is changing/confounding the whole notion of authorityAt the same time, Google is aggressively developing a knowledge graph which seeks to have definitive information for everything 22

The Filter BubbleCoined by Eli Pariser, and a worthy read:

23

What Is The Filter Bubble?An algorithmselectively guesses what information a user would like to see, effectively separating them from information that disagrees with his or her viewpointsExamples?Google's personalized search resultsFacebook's personalized news feed24

How To Pop Googles BubbleLog out of all instances of Google accounts (including Gmail, Google+, etc.)Deliberately use different terms than those suggested by Google as you typeUse a different search engine25

What Is The Knowledge Graph?A seriously cool approach to taking search to the next levelEssentially, its a map that does two fundamental things:It attempts to disambiguate multiple meanings of wordsIt also connects the relationships between entities 26

Disambiguation27

Connecting Entities28

The Knowledge Graph Is True AIGoogle has taught its systems about things, in multiple waysWeb search taught Google about real world relationshipsBook search taught Google about language and how its usedThe knowledge graph is correlating these and other learnings29

Knowledge Graph Results

30

Surfacing Related Links

31

Carousel Results

32

The Knowledge Graph Vs. AdsAmazingly, knowledge graph results in some cases displace ads on the right side of the screenNonetheless, Googles ad revenue has actually increased since its introductionDarwinian conclusion: More relevant web results make surviving ads more compelling and/or useful33

Important: Author ProfilesAs part of its identity push, Google now identifies authors in resultsDone primarily via HTML markup, but also associations with Google Profile pages and other trustworthy sites on the web (Major Media sites, Wikipedia, LinkedIn, etc)Benefit: Increased visibility of authoritative authors in search results

34

Another Key ShiftRelevance used to be determined by web authority essentially authoritative web pages from trusted publishersToday, social proof is equally if not more important35

What Is Social Proof?The original version: Click popularity (observing what pages most searchers click on and stay on)Ver. 2.0: PageRank - relying on links to other pages, e.g. publisher votes (in the offline world known as citation analysis)Today: Likes, +1s, tweet mentions, other social recommendations

36

The Social Proof ConundrumWhos the most trusted authority?Britannica or Wikipedia?Your parents or your friends?Your friends or 500 5-star ratings on Yelp?500 5-star ratings on Yelp or one Michelin review?Your teachers or op-ed writers of major newspapers?

37

The Problem With Social ProofWho delivered the Gettysburg Address?

Abraham LincolnPeter NorvigEdward Everett

Google: Gettysburg Address

39

Who Was Edward Everett?The specially invited speaker who delivered THE Gettysburg orationFormer Governor of Massachusetts, U.S. Secretary of State, Harvard President, Minister to Great BritainAbraham Lincoln was a last-minute addition to the ceremony, who spoke for just two minutes after Everetts two hour official oration40

The New Query RealityWith all of the changes to Google, traditional keyword search isnt the most effective approach any moreLonger queries are betterHow long? Full sentences or even paragraphs or full pages of textTip: Voice search is really good with longer queries, & its easy to try many variations41

Boolean Is DeadIts NOT whether you use Boolean operators OR alternate keywords AND hope that Google overrides ~500 other signalsIts how you phrase your question, what youve searched for in the past, and what other people have found useful.42

Advanced Search Isnt DeadAdvanced search is still really useful; accessible three ways:For each type of search result, contextual search tools availableFrom settings button on google.com, select advanced searchUse search operators in your queryWait isnt Advanced Search just Boolean? Sorta/not really

43

About Those Queries20% of queries are unique and new to Google every day 450 billion uniques since 2003!Every query has to travel on average 1,500 miles to get results back to a user44

The Next Big Trend: Predictive SearchIt started with query autofillThen it was Google instant for search resultsNext: Always on search agents that observe your behavior and location, know your history and have access to key assets like your calendar, contacts, social connections and more

Key Question, For YouThe Professional Searcher:Do you want search to be descriptive or predictive?Traditional search focused on precision vs. recallBy definition, and in the way search engines evolved, search results were descriptive

Predictive SearchNew (predictive) search uses machine learning, constant scanning of news, social media and the web, combined with our search history & behavior to intuit whats new and perhaps more interesting than traditional searchGoogle NowBing Satori47

Google NowFor now, very mobile centricAssumes you have your connected mobile device (notice, I didnt say smartphone) with you at all timesHelps with work tasks but skewed toward tasks where you might use your phone to engage with a mobile ad

Bing Predictive SearchRemember this guy?

Hes Baaaaacckkk

Bing Code Name: SatoriUsing Bing as a knowledge base, monitors your activities on apps, and intervenes when it has suggestionsWriting a paper on GalileoChatting with a friend about which movie to seeLike Siri or Google Now, but less intrusive and smarter thanks to content partnerships

Satori PotentialSatori has an intriguing inherent capability: it can search a real time graph of more than 1BN nodes instantlyTranslation: Possible real-time Facebook/Twitter search

Hope For Yahoo Search?

Search is far from over. Its physics in the 1600s or biology in the 1800s. Theres miles to go before you get to quantum physics or even a microscope.

- Marissa Mayer, August 1, 2013

Shifting GearsHeres some stuff to help you search more effectively & creatively on a daily basis53

A People Search Strategy, Part 1Write a mini-biography - schools attended, professional history, hobbies, etc.In search settings select never show instant results and select 100 results per pageNow do a simple search for the persons name54

A People Search Strategy, Part 2Cut & paste results into wordcounter.comMake a list of all non-relevant frequent termsIn Advanced Search:Put must-have terms in exact phrase and at least one of the words boxesPut must-not-have terms in without the words box55

Bill Who?

56

Setting Up The Search

57

Two Simple People Search TacticsUse fill in the blank querieswork for company nameemployed bydegree in/fromexpert inUse the wildcard operator"a * saved is a * earned"

58

People Search & PatentsPatents can provide some really useful search avenues to exploreTry Googles patent search, and use some of the advanced optionsExample:ininventor:"Sebastian Thrun

Patent Search Results

60

Some Additional Useful SitesGoogle is like a crack dealer, one frustrated startup founder told me recently. They give you something that gets you hooked, but you end up strung out. Youre so dependent on somebody that you cant do anything about it.GigaOM, March 16, 201261

Awful Name, Great Results: BlekkoA new search engine, built from scratch over the past six yearsFrom the gang who built Newhoo (the Open Directory Project)A bunch of new, original features you wont find elsewhereEmphasis on openness, community, privacy and customization

62

Blekko SlashtagsSlashtags are filters that customize standard web resultsBlekko has hundreds of pre-built slashtagsYou can also create your own, or use the slashtags created by other users63

Blekko Slashtags

64

Another Fowl NameDuck-Duck-Go. Yes, thats its real name.Lots of useful tools for direct resultsZero click results similar to Google one box but sometimes more usefulDirect search/integration with unique content sources65

Microsoft Academic SearchAn index of millions of academic publicationsAlso displays the key relationships between and among subjects, content, and authors, highlighting the critical links that help define scientific research66

Microsoft Academic Search

67

Wolfram AlphaA computational knowledge engineGreat for authoritative facts that can be computedA little known side benefit: Your personal Facebook report68

Facebook Report

69

QuoraA continually improving collection of questions and answers created, edited, and organized by everyone who uses itSo far, pretty high-quality stuffComing soon: Ask to answer suggestions e.g. recommendations of topical expert Quorans70

YouTube: An Increasingly Rich Resource100+ hours of video are uploaded to YouTube every minuteYouTube is localized in 61 countries and across 61 languagesNot just silly cat videos companies, universities and other important organizations (e.g. governments) now recognize the value of video71

SlideshareBecoming the defacto YouTube for Powerpoint presentationsA great resource for investigating a persons communication skillsA great serendipitous way to find people with similar skills/interests72

A Word About PrivacyGoogle has come under fire regarding its privacy policies, especially in some European countriesAlso from advocacy groups (coincidentally partially funded by Microsoft) here in the US73

Dont Be Evil?

74

Mr. Schmidt Goes To Washington

75

Should You Trust Google?Google was forced to provide some information to the NSAIt gave far less than othersIt also has taken steps to make it harder for others to snoop.However Google also collects (and uses) a ton of personal data every time you use it76

The Bottom Line?Google works very hard to protect privacyIt provides free services in exchange for capturing personal informationIf you feel the usefulness is worth the price, theres little harm in trusting them with your personal details77

Confucius SaysThe hardest thing to find is a black cat in a dark room especially if there is no cat.78

Wait For It!

79