Discriminating Among Word Meanings by Identifying Similar...
Transcript of Discriminating Among Word Meanings by Identifying Similar...
1
Discriminating Among Word Meanings by Identifying Similar Contexts
Amruta PurandareAdvisor: Dr. Ted Pedersen
July 27, 2004
Research Supported by National Science FoundationFaculty Early Career Development Award (#0092784)
2
Word Sense Discrimination
shells exploded in a US diplomatic complex in Liberiashell scripts are user interactive
artillery guns were used to fire highly explosive shellsthe biggest shop on the shore for serious shell collectors
shell script is a series of commands written into a file that Unix executesshe sells sea shells by the sea shore
sherry enjoys walking along the beach and collecting shellsfirework shells exploded onto usually dark screens in a variety of colors
shells automate system administrative taskswe specialize in low priced corals, starfish and shells
we help people in identifying wonderful sea shells along the coastlinesshop at the biggest shell store by the shoreshell script is much like the ms dos batch file
3
sherry enjoys walking along the beach and collecting shellswe specialize in low priced corals, starfish and shells
we help people in identifying wonderful sea shells along the coastlinesshop at the biggest shell store by the shore
she sells sea shells by the sea shorethe biggest shop on the shore for serious shell collectors
shell script is much like the ms dos batch fileshell script is a series of commands written into a file that Unix executes
shell scripts are user interactiveshells automate system administrative tasks
shells exploded in a US diplomatic complex in Liberiafirework shells exploded onto usually dark screens in a variety of colors
artillery guns were used to fire highly explosive shells
4
Our Approach
Strong Contextual HypothesisSea Shells => (sea, beach, ocean, water, corals)Bomb Shells => (kill, attack, fire, guns, explode)Unix Shells => (machine, OS, computer, system)
Corpus—Based Unsupervised Clustering
Knowledge—LeanPortable – Other languages, domainsScalable – Large Raw TextAdaptable – Fluid Word Meanings
5
FeatureSelection
ContextRepresentation
MeasuringSimilarities
ClusteringEvaluation
Methodology
6
Surface Lexical Features
Unigrams in today’s world the scallop is a popular design in architecture and iswell known as the shell gasoline logo
Bigramsshe sells sea shells on the sea shore
Co-occurrencesbivalve shells are mollusks with two valves joined by a hingeshells can decorate an aquarium
7
Local TrainingPectens or Scallops are one of the few bivalve shells that actually swim. This is accomplished by rapidly opening & closing their valves, sending the shell backward.
Fire marshals hauled out something that looked like a rifle with tubes attached to it, along with several bags of bullets and shells.
If you hear a snapping sound when you’re in the water, chances are it is the sound of the valves hitting together as it opens and shuts its shell.
Teenagers tried to make a bomb or some kind of homemade fireworks by taking the bullets and shotgun shells apart and collecting the black powder.
Bivalve shells are mollusks with two valves joined by a hinge. Most of the 20,000 species are marine including clams, mussels, oysters and scallops.
There was an explosion in one of the shells, it flamed over the top of the other shells and sealed in the fireworks, so when they ignited, it made it react like a pipe bomb."
These edible oysters are the most commonly known throughout the world as a popular source of seafood. The shell is porcelaneous and the pearls produced from these edible oysters have little value.
8
Global Training
U.S. researchers said sea shells may be the product of a geological accident that flooded ancient oceans with calcium, thereby diversifying marine life. Researchers at the U.S. Geological Survey have found the amount of calcium in sea water shot up between the end of the Proterozoic era (about 544 million years ago) and the early Cambrian period (515 million years ago). This increase, they suggested, allowed soft-bodied marine organisms to create hard shells or body parts from the calcium minerals. The researchers studied the chemical composition of liquids trapped in the cavities of salty rocks called halites, which provide samples of prehistoric oceans.
John Kerry is a man who knows how to keep a secret. The Democratic White House hopeful was so obsessed with making sure the name of John Edwards, his vice presidential running mate, remained under wraps until the announcement that he had vendors who printed up placards and T-shirts sign a non-disclosure agreement. Kerry himself telephoned his plane charter company at 6 p.m. on Monday night to let them in on his decision in time to have the red, white and blue aircraft's decal changed to read "Kerry-Edwards A Stronger America." Edwards did not travel to Pittsburgh to attend the rally at which his name was announced, which also might have alerted the media. After months of speculation, first reports began emerging less than 90 minutes before Kerry made his public announcement at 9 a.m.
9
1st Order Context Vectors
C1: if she sells shells by the sea shore, then the shells she sells must be sea shore shells and not firework shells
C2: store the system commands in a unix shell and invoke csh to execute these commands
2
0
commands
101100C2
010022C1
unixfireworkexecutesystemshoresea
10
2nd Order Context VectorsThe largest shell store by the sea shore
06272.852.913362.608420.0321176.8451.021O2context
018818.55000205.5469134.5102Store
000136.044129.57600Shore
008.739951.781230.5203324.9818.5533Sea
ArtillerySalesBombsSandyNorth-West
WaterSells
11
2nd Order Context Vectors
12
052.2700.9204.210
28.7203.2401.2802.53
WeaponMissileShootFireDestroyMurderKill
17.77014.646.222.1034.2
19.232.36072.701.282.56
ExecuteCommandBombPipeFireCDBurn
13
Latent Semantic Analysis
Singular Value Decomposition
Resolves Polysemy and Synonymy
Captures Conceptual Similarities
Converts Word Space to Semantic Space
14
Before SVD …
0
0
0
0
2
0
0
Shoot
3020003312
2030002202
0222210000
0210120010
3002322030
2231321020
3020000102
MachineFireExecuteMurderGunsBombPipeUnixKillComputer
15
After SVD …
0.054
0.02
0.471
0.386
0.761
0.75
0.028
Shoot
3.23-0.112.6750.0050.075-0.012.2622.450.592.292
2.67-0.152.2-0.05-0.03-0.081.872.070.421.934
0.8131.030.8121.0131.841.350.45-0.281.25-0.27
0.5520.850.57120.8341.5121.110.289-0.321-0.306
2.041.611.9061.6082.932.161.240.112.110.102
2.1811.582.0221.5872.8992.111.340.242.120.222
2.073-0.091.713-0.0120.021-0.031.451.590.361.48
MachineFireExecuteMurderGunsBombPipeUnixKillComputer
16
Clustering
UPGMAHierarchical : Agglomerative
Repeated BisectionsHybrid : Divisive + Partitional
17
Similarity Coefficients
(X П Y)_____________________
√ |X| * |Y|
Cosine
(X П Y)_______________________
min (|X|,|Y|)
Overlap
(X П Y)_______________
|X U Y|
Jaccard
2 (X П Y)___________________
|X| + |Y|
Dice
(X П Y)Matching
18
Evaluation
2015212C417
110
55111215
10612C310171C2152310C1
19
DataLine, Hard, Serve
4000+ Instances / Word60:40 training:test Split3-5 Senses / Word
SENSEVAL-272 words = 28 V + 29 N + 15 AApprox. 50-100 Test, 100-200 Training8-12 Senses / Word
Associated Press Worldstream NewswireNov 1994-June 2002 by LDC, U Penn539,665,000 words
20
Experiments
TrainingLocal versus Global
FeaturesUnigrams, Bigrams, Co-occurrences
Context Representations1st order versus 2nd order
ClusteringRBR versus UPGMA
21
Conclusions
Smaller Data2nd Order + RBR
Larger Local Data1st Order + UPGMA
Global TrainingHelps 1st Order Bigrams, UPGMAOverall Local Training Better
Corpus + DictionaryHelps only with smaller training data
22
Applications
Word Sense DisambiguationSynonymy IdentificationEmail/News ClassificationName DiscriminationOntology AcquisitionText Summarization
23
Contributions
Systematic ComparisonPedersen & Bruce (1997)Schütze (1998)
Discrimination ParametersFeaturesContext RepresentationsClustering Approaches
24
Contributions
Training VariationsLocal Global
Relative ComparisonRaw CorpusCorpus + Dictionary
Softwarehttp://senseclusters.sourceforge.net
25