What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond,...
-
Upload
sylvia-jencks -
Category
Documents
-
view
215 -
download
0
Transcript of What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond,...
![Page 1: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/1.jpg)
What Did We See?& WikiGIS
Chris PalUniversity of Massachusetts
A Talk for Memex DayMSR Redmond, July 19, 2006
![Page 2: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/2.jpg)
Research Questions
1. How do personal and community photo-journals and blogs interact?Spectrum from personal blogs – community portals (bliki’s) – Wiki articles (most public) User Interface & Social Computing Research
2. Can we ‘mine’ information in Blogs ?Find Blog entries that look like Wiki entries, extract information, encourage contributions?Document and Text Processing Research
3. What is the role of computer vision for location and object recognition?Can we use these methods to provide the user with relevant information?
![Page 3: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/3.jpg)
Search Blogs and Wiki Entries
![Page 4: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/4.jpg)
Questions About Observations
![Page 5: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/5.jpg)
Search and Social Computing
I Discover that my friend Justin also found an interesting mushroom
Have I been here as well?
![Page 6: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/6.jpg)
1. Object RecognitionFrom Images and Text
2. Location RecognitionFrom Images and Text
Object and Location Recognition
![Page 7: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/7.jpg)
Conditional Random Fields
yt -1
yt
xt
yt+1
xt +1
xt -1
. . .
yt+2
xt +2
yt+3
xt +3
said Ling a Microsoft VP …
OTHER PERSON OTHER ORG TITLE …
Named Entities
(SFSM states)
Binary Features
Input Sequence
• Widely applicable, many positive results e.g. speech recognition
• Fact Extraction (from Blogs and Wikis)
• Address extraction
• Information Extraction Example
![Page 8: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/8.jpg)
Research Result - Training a CRF
• Define the vector of feature values a time t
• Define the global feature function as
• The gradient of the conditional log likelihood
Model expectation, i.e.Empirical expectation
![Page 9: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/9.jpg)
Results: CRF Training
NetTalk text-to-speech: Linear-chain CRF training using sparse inference
75% less training time than exact training, with no loss in accuracy
Accuracy:
Fixed: 85.7KL: 91.6Exact: 91.6
![Page 10: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/10.jpg)
SenseCam Enhanced Blogs
Produce Lots of Data for Location Recognition
![Page 11: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/11.jpg)
Multi-Conditional Learning
• Motivation - Simple GMM Example Joint Conditional Multi-Conditional
![Page 12: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/12.jpg)
Multi-Conditional Learning
• One motivation: Conditional Random Fields can be derived from a traditional joint model
• But, there are many other conditional distributions that could be defined
• What do we gain if we model those as well?
• Other combinations possible
![Page 13: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/13.jpg)
Image Segmentation/Pixel Classification
MSR Cambridge / Berkeley Data
![Page 14: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/14.jpg)
Mixtures of Factor Analyzers
• Generative model for simultaneous dimensionality reduction and clustering
• We wish to obtain a discriminative version of this type of model discriminatively
![Page 15: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/15.jpg)
Performance vs. Model Complexity
Interesting ?
Joint Optimization benefits more substantially from additional data.
![Page 16: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/16.jpg)
Performance with More DataTraining Set Accuracy Test Set Accuracy
hmm…
![Page 17: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/17.jpg)
Search Blogs of Friends
![Page 18: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/18.jpg)
Detect and Find Expert Knowledge
![Page 19: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/19.jpg)
Simple Exponential Family Models for Documents
![Page 20: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/20.jpg)
Results: Document Classification
![Page 21: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/21.jpg)
New Graphical Models for Email and Blogs
xb
y
Nb
xsNs
xrNr-1
Body Title FriendsWords Words discussed
PredictedRecipient
Nr
- function - random variable
- N replicationsN
Email Model: Nb words in the body, Ns words in the subject, Nr recipients
The graph describes the joint distribution of random variables in term of the product of local functions
• Scenario: Predict which friends might be interested in your new Blog entry
• New Idea: Plated Factor Graphs
![Page 22: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/22.jpg)
Detect Quality Content and Encourage Knowledge Contributions
![Page 23: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/23.jpg)
Conclusions, Present & Future Work
• WikiGIS – Merged Blogs, Blikis and Wikis with Microsoft Virtual Earth
• Merge the SenseCam with a smart Phone- Enable Intelligent Digital Assistants - Output to the television
• Next Steps: Location and object recognition enabling information retrieval
• Other Uses: Assistive Technology for the Elderly
![Page 24: What Did We See? & WikiGIS Chris Pal University of Massachusetts A Talk for Memex Day MSR Redmond, July 19, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022070307/551b0f24550346f70d8b5e03/html5/thumbnails/24.jpg)
References & Results so Far
• with Charles Sutton and Andrew McCallum. Sparse Forward-Backward using Minimum Divergence Beams for Fast Training of Conditional Random Fields. In proceedings of ICASSP 2006.
• with Michael Kelm and Andrew McCallum. Combining Generative and Discriminative Methods for Pixel Classification with Multi-Conditional Learning To appear in the proceedings of ICPR 2006.
• with Andrew McCallum, Greg Druck and Xuerui Wang. Multi-Conditional Learning: Generative/ Discriminative Training for Clustering and Classification To appear in the proceedings of AAAI 2006.
• CC Prediction with graphical models To appear in the proceedings of CEAS 2006.