Patent Summarization and Paraphrasing · Patent Summarization I Patent Summarization is the...

Patent Summarization and Paraphrasing

David Cinciruk

February 25, 2015

1

Table of Contents

Overview of Patent Summarization

Patent Claim Language

Claim Dependency Structuring

SimplificationThree Different Simplification Experiments

Deep Paraphrasing and SummarizationSidebar - Meaning-Text TheoryPreprocessingDeep Syntactic SummarizationGeneration Proper

Evaluation of their System

2

Patent Summarization

I Patent Summarization is the technique of summarizing and/orparaphrasing patent claims into a more readable format.

I Patent Summarization is a harder technique than regular textsummarization especially when one considers dependentclaims and how to relate them with the summary

3

PATExpert Patent Summarization Methodology

4

Overview of their System

I Claim Dependency Structuring: Relating dependentclaims to the claims they depend on

I Simplification: Segments the original sentence andreconstructs them to obtain grammatical sentences

I Regeneration: Generates a deep syntactical structure fromthe above and summarizes them and/or combines them

5

Table of Contents







6

Patent Claims

I Only legal portion of a patent

I Used to determine the extent of protection a patent provides

I Legally, the rest of the patent is only used to support theterms used in the claims

7

Sample Patent Claim

1. A personal digital assistant (PDA), comprising: a calendar function;an address book function; a processor operably coupled to amemory and a display; wherein the PDA dynamically generates aroute path using the processor and the memory from a moveablelocation associated with the PDA to a destination of the PDA byrepetitively dynamically expanding one or more adjacent locationsand inserting the adjacent locations into a first data structurewherein one or more first locations of the first data structure areassociated with a then existing least cost location in the route path,the route path is dynamically generated from the moveable location,the first locations of the first data structure, and the destination;and wherein at least a portion of the route path is dynamicallycommunicated to the display.

2. The PDA of claim 1, further comprising an interface device operableto audibly communicate the route path.

3. The PDA of claim 1, wherein the first locations are removed fromthe first data structure as the first locations become part of theroute path and the removed first locations combine to form asecond data structure.

8

Grammatical Problems

I Claims are very dense structures that barely resemble standardEnglish at times

I The language is filled with words that don’t match the typicaluses of words or which have very explicit meaning compare tohow they are used in real life

9

Relating Dependent Claims

I Dependent claims take itemsfrom the claim they aredependent on and narrowdown the scope to a moreimplementable form.

I They are not part of themain scope of the inventionbut instead used to describewhat the patenter wants thepatent to be used for.

I Dependent claims mayconflict with one anothershowcasing different ideasthat the inventor had withhis invention

Independent Claim

Dependent Claim

Items in Claim

10

Table of Contents







11

Claim Structures

I Program determines theportion of the parent claiman independent claim refersto and maps them together

I Able to set how much of thisinformation is to be kept.

Independent Claim

Dependent Claim

Items in Claim

13

Table of Contents







14

Steps of Simplification

I POS Tagging: Standard POS Tagging with Tree-Tagger

I Segmentation of Claim Sentences: Uses Rule Based andMachine Learning based segmentation

I Coreference Resolution: Determine if two NPs that referto the same object

I Building a Clause-Discourse Tree: Identifies clausestructures

I Reconstruction of Sentences: Creates grammaticallycorrect independent sentences

16

Clause-Discourse Structuring

17

Clause-Discourse Structuring

I Clause Structuring: Output is a binary tree whose terminalnodes are sentence segments and whose intermediate nodesspecify whether the subtree is a subordination, coordination,or juxtaposition

I Clause Tree Flattening: Binary Tree is flattened to accountfor n-ary constraints such as coordinates

I Projection of the Clause Structure onto the DisclosureStructure: Intermediate nodes are enriched with discourseinformation based on Rhetorical Structure Theory labeling thediscourse units (spans) as nucleus/satellites and determiningthe relationship between them according to a set of rules thatuse the type of constructs and lexical information

18

Clause Structuring

I Search for the best clause structure in a space restricted by aset of weighted rules and constraints on the rules

I Rules encode fundamental features for the identification ofcoordination, subodination, and juxtaposition relationsbetween spans

I Constraints ensure syntactical correctness and globalcoherence

I Rules, constraints, and weights were developed and adjustedin a series of iterative trials based on evaluation of a smalldevelopment corpus consisting of 8 manually structured claims(featuring 104 segments)

19

Clause Structuring Rules

Rule R is a quadruple < F1,F2,W0,Wc >

I F1 and F2 are feature tuples describing the two spans S1 andS2 - consists of < punct, coord , subord , syntagm, colon >

I punct - clause delimiter punctuation preceding the segmentI coord - coordination markerI subord - subordination markerI syntagm - span’s syntactical group (mainly S, NP, or VP)I colon - whether the clause contains a colon

I W0 is tha rule’s initial weights

I Wc is the set of weighted constraints (weights determine ifit’s a hard or soft constraint)

20

Constraint Table

21

Search Problems

Two Points

Three Points

Four Points

2 trees

12 trees

96 trees

I As number of segments grow thenumber of rules to link them growsexponentially

I A variation of local beam search isused to search among the varioustrees with the metrics calculatedfor each application of a ruleserving as an objective function

I Goal of algorithm is a rooted treeso the algorithm can backtrack andexplore alternatives

22

Sentence Reconstruction

I Set of rules to convertsegments into sentences

I Adds missing nouns or verbs

I Conjugates verbs innon-finite clauses

I Removes initial markers

I Change the order ofconstituents

23

OPTIONAL - Discourse Structure Based SummarizationPruning

I Depth: all spans below agiven depth are preserved

I Discourse Relation: a listof ordered disclosurerelations can be provided,pruning occurs from lowestto highest

I Purpose > Means >Elaboration-Object-Attribute >Elaboration-Location

I Discourse Markers: anordered list of discoursemarkers can be provided forpruning

I When > By > For

24

Table of Contents







25

Segmentation Experiments

I Manually constructed database of about 1500 patents

I Machine Learning - Weka’s J48 Decision Tree Learner

I Rule Based - Best result utilized semi-colons, commas, andabout 20 lexical markers and expressions of the patent domain

I Evaluation counts 1:1 alignment between machinesegmentation and hand segmentation as truth

26

Coreference Resolution Experiments

I Database of 30 claims

I Precision of 83% and Recall of 79%I Types of Errors

I Chunking Errors - failed to mark a chunk as an NPI NP Modifier - identical NPs used as modifiers inside larger and

more complex NPs that aren’t the same objectsI Abstract NPs - abstract nouns are often repeated but do not

corefer to each otherI Partial Match - some NPs may contain another NP inside it

but they do not match completely

27

Clause Structuring Experiments

I Comparing automatic segmentation and coreferencing tomanual segmentation and coreferencing and runningexperiments on both

I Baseline performs clause structuring based on right branchinggiven the number of segments in the gold standard

I Evaluation involves counting the number of identical spansbetween automatic and manual

I Automatically map each segment of the raw input to itscorresponding gold standard segment

I Development: F-score of 64%, Test: F-score of 42%

I This experiment was also useful for determining the mostimportant rules and constraints

28

Table of Contents







29

Stages for this Step

I Preprocessing: Dependency parsing and mapping to DeepSyntactical Structure

I (Optional) DSyntS Summarization

I Structure Transfer: Converts from one language model toanother

I Generation Proper

31

Table of Contents







32

Meaning-Text Theory

I Theoretical linguistic framework, put forward in Moscow byAleksandr Zolkovskij and Igor Mel’cuk, for the construction ofmodels of natural language

I Mapping of a semantic meaning to phoentics via intermediatestages.

33

Semantic Representation

I Web like structure that consists of a network of predications -nodes with arrows running from predicate nodes to argumentnodes

I Nodes correspond to lexical and grammatical meanings asthese are expressed directly by items in the lexicon or byinflectional means

34

Semantic Representation

[The media] [harshly criticized the government for its decision toincrease income taxes]

35

Syntactic Representation

I Uses Dependency TreesI Consists of two levels - Deep Syntactic Structure (DSyntS)

and Surface Syntactic Structure (SSyntS)I Deep Syntactic - trees represent dependency relations between

lexemes and lexical functions. More about relating wordsdirectly with verbs. Two sentences may have the same deepsyntactic relation but different other structures

I Surface Syntactic - represents the language-specific syntacticstructure of an utterance and includes nodes for all the lexicalitems

36

Deep vs Surface Syntactic Representation

37

Morphological representation

I Strings of mophemes arranged in fixed linear order reflectingthe ordering of elements in the actual utterance

I Deep Morphological - consists of strings and morphemes

I Surface Morphological - converts morphemes into theappropriate morphs and performing morphological operations

38

Table of Contents







39

Minipar Dependencies vs. Stanford Dependencies

Minipar vs. Stanford

41

Minipar to Surface Syntactic Structures

I Uses Minipar to generate dependency models

I Develop a set of 137 rules to map Minipar dependencies toSSyntS

42

How to map Surface Syntactic to Deep Syntactic

I Verbal Tense Auxiliary Forms are mapped onto attribute-valuepairs

I Determiners are removed and definiteness/indefiniteness isplaced into an attribute

I Governed Prepositions are removed from the structure

I Some lexical units are reduced to abstract lexical labels

43

Table of Contents







44

Deep Syntactic Summarization

I Small set of summarization criteria is based on specificpatterns within the DSyntS input

I Takes into account discourse and dependency structureinformation

I Follows specific rules

46

Example Rules

I A noun has a postponed attribute:I The optical component is a shading member [arranged near the

optical axis around the aperture plane of the optical system]

I A definite noun is modified by a full statement:I An automatic focusing apparatus comprises the actuator

[which controls the focusing means depending upon the outputof the phase detector]

I A noun in a dependent claim is modifed by a has-part relation(in an independent claim, it can bear important information):

I A unitary ridge is formed on the top face [having side surfacesconstituting the first and second side chip deflector surfaces].

I A noun in a dependent claim is modified by a PURPOSErelation (for + Gerund):

I The apparatus comprises a lens [for converting the light fromthe signal plane].

47

Table of Contents







48

Stages of Generation

I Aggregation of repetitive and isolated fragments of theDSyntS

I Introduction of discourse markers to the SSyntS

I Generation of referring expressions in the SSyntS

I Syntactic Generation

50

Aggregation

I Fusion of several smaller sentence or phrase structures thatshare common parts into one structure

I Uses the coreferences determined in the previous partI Example:

I 1 - [An optical disk drive] [comprises] [a laser light source].I 2 - [An optical disk drive] [comprises] [an optical system].I (1+2) - An optical disk drive comprises a laser light source and

an optical system.

51

Discourse Marker Insertion

I Some rules add discourse markers to the top verb of theSSyntS

I Depending on the discourse relation to which is introduced,the marker can be retrieved from a discourse-marker dictionary

I Examples of Discourse Markers: For, By, When

52

Referring Expression Generation

I Bunch of rules that the program uses to try to make a moreunderstandable summary/paraphrase

I Example: Introducing a relative pronoun as subject of asentence if it is coreferring with the object of the previoussentence.

I The phrase to become the relative clause must not besyntactically “too heavy” (i.e., contain “too many” nodes); ifit is, the introduction of a deictic is preferred

I The rule does not apply either if the object of the firstsentence has undergone aggregation so as not to rebuildsentences that would be very long and potentially ambiguous

53

Syntactic Generation

I Stage responsible for completing the generation

I Realizes agreement, word order, and other surfacemorphological structures to convert the deep syntacticalstructures back to regular sentences.

54

Table of Contents







55

System Setup

I Uses a software package ROUGE to automatically evaluatequality of summarizations

I Chose 30 patents at random to be summarized using theirmethod as well as with Microsoft Word’s automaticsummarizer

I Also ran experiments with Multilingual patents to determinehow well translation work - native speakers rate translationsgiven by their software versus Google Translate

56

Results and Explanations

I For summarization, their system achieved an F-Score of 61%while MS Word achieved 43%

I Low score can be partially explained by the object/methoddichotomy in some patent claims, which cannot reliably beidentified in an automatic way. If a patent claim sectioncontains claims referring to both the invented object and themethod of applying this object, both kinds of claims tend tocontain largely the same information

I Multilingual Results - no meaningful information is lost duringsimplification

57

Example Summarization

An optical disk drive comprising: a laser light source for emitting alaser beam; an optical system for conversing the laser beam fromthe laser light source on a signal plane of optical disk on whichsignal marks are formed and for transmitting the light reflectedfrom the signal plane; one or more optical components arranged inthe optical path between the laser light source and the optical diskfor making the distribution of the laser beam converged by theconversing means located on a ring belt just after the passage ofan aperture plane of the optical system; a detection means fordetecting the light reflected from the optical disk; and a signalprocessing circuit for generating a secondary differential signal bydifferentiating the signals detected by the detection means and fordetecting the edge positions of the signal marks by comparing thesecondary differential signal with a detection level.

58

Example Summarization

An optical disk drive comprises a laser light source, an opticalsystem, a detection means, and a signal processing circuit. Thelaser light source emits a laser beam. The optical system conversesthe laser beam from the laser light source on a signal plane ofoptical disk and transmits the light. The optical disk drive alsocomprises one or more optical components. It is arranged in theoptical path between the laser light source and the optical disk.The detection means detects the light. The signal processingcircuit generates a secondary differential signal and detects theedge positions of the signal mark.

59

Patent Summarization and Paraphrasing · Patent Summarization I Patent Summarization is the...

Documents

Transcript of Patent Summarization and Paraphrasing · Patent Summarization I Patent Summarization is the...