Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon [email protected]...
-
Upload
alberta-stevens -
Category
Documents
-
view
219 -
download
1
Transcript of Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon [email protected]...
![Page 2: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/2.jpg)
www.sharon-it.com2
Contents• Expanding/Learning terms• Categorization/Clustering engines• Google Suggest• SurfWax FocusWords• When you don’t know where to
start
![Page 3: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/3.jpg)
www.sharon-it.com3
Make Longer Queries
•Yahoo
•Harvest Digital
Average Search Terms per Query
Overall Experienced
![Page 4: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/4.jpg)
www.sharon-it.com4
Adding Words• Holocaust
23,200,000• holocaust memorial
836,000• holocaust memorial budapest
42,300• holocaust memorial budapest danube
4,910• holocaust memorial budapest danube promenade
692
![Page 5: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/5.jpg)
www.sharon-it.com5
Classification/Categorization
• Classification: the process of deciding the appropriate category for a given document.
• Examples:– deciding what newsgroup an article belongs
to. – what folder an email message
should be directed to.– what is the general topic of an essay.
![Page 6: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/6.jpg)
www.sharon-it.com6
Clustering
• The process of automatically grouping documents.
![Page 7: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/7.jpg)
www.sharon-it.com7
Search Categorization/Clustering
• The result documents are ordered according to categories.
• The searcher can select the relevant category to display the related documents.
• Examples:– Vivisimo/Clusty– Excite– Teoma– Exalead
![Page 8: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/8.jpg)
www.sharon-it.com8
Clusty
![Page 9: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/9.jpg)
www.sharon-it.com9
Excite
![Page 10: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/10.jpg)
www.sharon-it.com10
Ask
![Page 11: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/11.jpg)
www.sharon-it.com11
Exalead
![Page 12: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/12.jpg)
www.sharon-it.com12
Google Suggest
• As you type – you get query suggestions and number of results per query.
![Page 13: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/13.jpg)
www.sharon-it.com13
Google Suggest (2)
![Page 14: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/14.jpg)
www.sharon-it.com14
Google Suggest
![Page 15: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/15.jpg)
www.sharon-it.com15
![Page 16: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/16.jpg)
www.sharon-it.com16
Yahoo Search Assist
![Page 17: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/17.jpg)
www.sharon-it.com17
SurfWax FocusWords• SurfWax has an option “Focus”• This option invokes the FocusWords
mechanism• You get suggestions to make your
query:– Broader– Similar– Narrower
• http://www.surfwax.com
![Page 18: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/18.jpg)
www.sharon-it.com18
SurfWax FocusWords
![Page 19: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/19.jpg)
www.sharon-it.com19
When you don’t know how to start
• Reverse Dictionary• Glossaries and
Dictionaries• Taxonomy/
Folksonomy• Pearl Culturing• Analyzing pages• Finding similar pages
– Google’s related:– Alexa www.alexa.com
![Page 20: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/20.jpg)
www.sharon-it.com20
Reverse Dictionary• OneLook reverse dictionary:
http://www.onelook.com/reverse-dictionary.shtml
• Example: “bird of prey” => raptor• Example: economical measure of a
nation’s wealth => Gross Domestic Product
![Page 21: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/21.jpg)
www.sharon-it.com21
![Page 22: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/22.jpg)
www.sharon-it.com22
Glossaries and Dictionaries• Google search:
– <topic>– glossary OR thesaurus OR dictionary OR taxonomy
• Example 1: agriculture glossary– http://www.cnie.org/nle/AgGlossary/AgGlossary.htm– http://www.cahe.nmsu.edu/news/aggloss.html– http://agriculture.house.gov/info/glossary.html
• Example 2: agriculture thesaurus– http://agclass.nal.usda.gov/agt/agt.shtml– http://www.fao.org/aims/ag_intro.htm (multilingual)
• http://www.glossarist.com– http://www.glossarist.com/glossaries/business/primary-in
dustry/agriculture.asp
![Page 23: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/23.jpg)
www.sharon-it.com23
![Page 24: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/24.jpg)
www.sharon-it.com24
Taxonomy/Folksonomy• Taxonomies
– Found via directory search (example DMOZ): http://search.dmoz.org/cgi-bin/search?search=taxonomy
– www.taxonomywarehouse.com (paid)
• Folksonomy– Use tags in
• Technorati www.technorati.com• Delicious www.del.icio.us
![Page 25: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/25.jpg)
www.sharon-it.com25
del.icio.us
![Page 26: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/26.jpg)
www.sharon-it.com26
del.icio.us
![Page 27: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/27.jpg)
www.sharon-it.com27
Pearl Culturing• What to do when you don’t have the
category nor the right keywords?• Find one good relevant website• Look it up in directories• You will find:
– the category/main keywords– authoritative websites
• Useful search engine: Exalead
![Page 28: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/28.jpg)
www.sharon-it.com28
Analyze Pages• Distilling: what is problematic in a bad page?
– what is wrong? Is there an interfering keyword/term appearing.
– Remove interfering terms (using “-”).• Identifying clues and patterns in a good page.
– Read the document, what are the clues? – Look for new keywords, word combinations and
other things differentiating between it to non-authoritative documents.
– Use frequency counter:• http://www.wordcounter.com/• http://www.georgetown.edu/faculty/ballc/webtools/web_
freqs.html
![Page 29: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/29.jpg)
www.sharon-it.com29
Frequency Counter
![Page 30: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/30.jpg)
www.sharon-it.com30
![Page 31: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/31.jpg)
www.sharon-it.com31
Wordcounter
![Page 32: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/32.jpg)
www.sharon-it.com32
![Page 33: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/33.jpg)
www.sharon-it.com33
References• http://hacks.oreilly.com/pub/ht/2 • www.batesinfo.com• http://www.searchtools
.com/info/classifiers.html
![Page 34: Www.sharon-it.com1 Refining – Finding Words/expanding Taly Sharon taly@sharon-it.com sharont@alum.mit.edu.](https://reader036.fdocuments.us/reader036/viewer/2022081514/56649e845503460f94b869f8/html5/thumbnails/34.jpg)
www.sharon-it.com34
Exercises1. How are bad user interfaces called (hint: try Google suggest)2. Reverse dictionary
a. Find relevant keywords for chemistryb. What is the terminology for when menstruation stops?c. How was the separation between the west and the soviet union
called?3. What are the related terms to Competitive Intelligence?4. Check suggestions from Google Suggest for a query starting
with biofuel.5. Using SurfWax, learn options to focus or broaden the query:
biodiesel.6. Identify the most relevant terms in the website: www.uspto.gov 7. Identify the most relevant terms in the Biofuel Wikipedia entry
http://en.wikipedia.org/wiki/Biofuel.8. Search in Onelook reverse dictionary and in other glossaries
terms: fuel, natural energy, geothermal, and other terms. Look at the results.