An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

6
An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing via the Dependency Model with Valence (DMV)

description

An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing. via the Dependency Model with Valence (DMV). Motivation. Dependency Parsing: Search Query Refinement Statistical Machine Translation Unsupervised Learning: Availability of Large Quantities of Data. DMV. - PowerPoint PPT Presentation

Transcript of An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

Page 1: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

An Attempt at Unsupervised Learning of Hierarchical Dependency

Parsing

via the Dependency Model with Valence (DMV)

Page 2: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

Motivation

• Dependency Parsing:• Search Query Refinement• Statistical Machine Translation

• Unsupervised Learning:• Availability of Large Quantities of Data

Page 3: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

DMV

• Pick a Direction (left or right)• Generate the first child, or stop;• Generate more children, until stop.• Repeat in the other direction.• Recurse…

• Porder• Pstop• Pattach

Page 4: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

EM

• Inside-Outside Algorithm:• Inside: Pi(i,X,j) = P(X derives i…j)• Outside: Po(i,X,j) = P(S derives 0…iXj…l)

• Re-Estimation:• Frequency of sub-tree (i,X,j)=Pi(i,X,j)*Po(i,X,j)

Page 5: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

Evaluation

• Head-percolation of Penn Treebank parses;• % edges correct (directed or undirected) in the

best (P)CFG parse…

• Zero Knowledge: 14.4 (29.9)• Adjacent Word Heuristic: 33.6 • Klein & Manning: 43.2 (63.7)• Oracle: 75.5 (77.5)• - Pattach: 60.0 (63.3) - Pstop: 53.9 (57.7)• - PstopA: 50.0 (54.8) - PstopN: 12.5 (30.8)

Page 6: An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

EM

• Didn’t work out… always made things worse, even when initialized with very good solutions.

• If started using Zero Knowledge, then after 1 iteration already gets 18.4 (38.4), then worsens.

• If started using an Ad-Hoc Harmonic for Pattach, then 21.5 (47.1) after 1 iteration, then worse, and similarly even for the Oracle solution…

• Summary:• - DMV – useful, simple, extensible model;• - EM – more thorough debugging needed.


Dependency Parsing - cs.princeton.edu

Dependency Parsing - cs.princeton.edu

Scene Graph Parsing as Dependency Parsing · 2018-05-15 · Scene Graph Parsing as Dependency Parsing Yu-Siang Wang National Taiwan University b03202047@ntu.edu.tw Chenxi Liu( ) Johns

Scene Graph Parsing as Dependency Parsing · 2018-05-15 · Scene Graph Parsing as Dependency Parsing Yu-Siang Wang National Taiwan University [email protected] Chenxi Liu( ) Johns

Dependency Parsing of Turkish

Dependency Parsing of Turkish

Dependency Grammar and Dependency Parsing - …stp.lingfil.uu.se/~nivre/docs/05133.pdfDependency Grammar and Dependency Parsing Joakim Nivre 1 Introduction ... Thus, in the sentence

Dependency Grammar and Dependency Parsing - …stp.lingfil.uu.se/~nivre/docs/05133.pdfDependency Grammar and Dependency Parsing Joakim Nivre 1 Introduction ... Thus, in the sentence

Exploiting Reducibility in Unsupervised Dependency Parsing

Exploiting Reducibility in Unsupervised Dependency Parsing

Recent Advances in Dependency Parsingnivre/docs/eacl4.pdf · unsupervised dependency parsing with richer contexts and smoothing. In Proceedings of Human Language Technologies: The

Recent Advances in Dependency Parsingnivre/docs/eacl4.pdf · unsupervised dependency parsing with richer contexts and smoothing. In Proceedings of Human Language Technologies: The

Unsupervised Dependency Parsing

Unsupervised Dependency Parsing

Bare-Bones Dependency Parsing - Uppsala Universitynivre/docs/BareBones.pdf · I Parsing methods for bare-bones dependency parsing I Chart parsing techniques ... [Kuhlmann and Satta

Bare-Bones Dependency Parsing - Uppsala Universitynivre/docs/BareBones.pdf · I Parsing methods for bare-bones dependency parsing I Chart parsing techniques ... [Kuhlmann and Satta

Unsupervised Dependency Parsing David Mareček Institute of Formal and Applied Linguistics Charles University in Prague Doctoral thesis defense September.

Unsupervised Dependency Parsing David Mareček Institute of Formal and Applied Linguistics Charles University in Prague Doctoral thesis defense September.

Unsupervised Partial Parsing: Thesis defense

Unsupervised Partial Parsing: Thesis defense

Punctuation: Making a Point - Stanford NLP GroupPunctuation: Making a Point in Unsupervised Dependency Parsing Valentin I. Spitkovsky withDaniel Jurafsky (StanfordUniversity) andHiyan

Punctuation: Making a Point - Stanford NLP GroupPunctuation: Making a Point in Unsupervised Dependency Parsing Valentin I. Spitkovsky withDaniel Jurafsky (StanfordUniversity) andHiyan

Statistical Dependency Parsing - Uppsala Universitynivre/docs/StatState.pdf · I Methods for statistical dependency parsing I Chart parsing techniques ... Eisner 2000]: I Split head

Statistical Dependency Parsing - Uppsala Universitynivre/docs/StatState.pdf · I Methods for statistical dependency parsing I Chart parsing techniques ... Eisner 2000]: I Split head

Unsupervised Semantic Parsing - microsoft.com · Semantic networks: SNE [Kok & Domingos 2008] ... unsupervised semantic parsing

Unsupervised Semantic Parsing - microsoft.com · Semantic networks: SNE [Kok & Domingos 2008] ... unsupervised semantic parsing

Concavity and Initialization for Unsupervised Dependency ...kgimpel/talks/gimpel... · Unsupervised Dependency Parsing Kevin Gimpel Noah A. Smith 1. lti Unsupervised learning in NLP

Concavity and Initialization for Unsupervised Dependency ...kgimpel/talks/gimpel... · Unsupervised Dependency Parsing Kevin Gimpel Noah A. Smith 1. lti Unsupervised learning in NLP

Dependency Parsing - Indian Institute of Technology Bombay · Dependency Parsing GaneshBhosale-09305034 NeelamadhavG.-09305045 NileshBhosale-09305070 PranavJawale-09307606 undertheguidanceof

Dependency Parsing - Indian Institute of Technology Bombay · Dependency Parsing GaneshBhosale-09305034 NeelamadhavG.-09305045 NileshBhosale-09305070 PranavJawale-09307606 undertheguidanceof

Dependency Parsing (3) - University Of Maryland · Dependency Parsing: what you should know •Transition-based dependency parsing •Shift-reduce parsing •Transition systems: arc

Dependency Parsing (3) - University Of Maryland · Dependency Parsing: what you should know •Transition-based dependency parsing •Shift-reduce parsing •Transition systems: arc

Dependency Parsing - Oregon State Universityclasses.engr.oregonstate.edu/.../notes/dependency-parsing-lecture.pdf · Outline Introduction Dependency Parsing Formal definition Parsing

Dependency Parsing - Oregon State Universityclasses.engr.oregonstate.edu/.../notes/dependency-parsing-lecture.pdf · Outline Introduction Dependency Parsing Formal definition Parsing

Dependency Parsing by Belief Propagation

Dependency Parsing by Belief Propagation

Feature Embedding for Dependency Parsing

Feature Embedding for Dependency Parsing

Dependency Parsing - Universitetet i oslo · Data-driven dependency parsing MaltParser: transition-based dependency parsing MaltParser is a language-independent system for data-driven

Dependency Parsing - Universitetet i oslo · Data-driven dependency parsing MaltParser: transition-based dependency parsing MaltParser is a language-independent system for data-driven