Attivio - What's New in AIE 2 and Dictionary Management

Proprietary & Confidential

WHAT'S NEW IN AIE 2AND

DICTIONARY MANAGEMENT

Traction User Group10/14/2010


Overview2

• What’s New In AIE 2• Dictionary Management


WHAT'S NEW IN AIE 2.1.1



What’s New In AIE 2.14

• Relevancy Improvements• Core Indexing and Search Improvements• Dictionary Management



Relevancy Improvements

• Field length normalization enhancement for improved title matching

• Proximity boost enhancement



Core Indexing and Search Improvements

• Real time fields redesigned to reduce disk IO• Memory capped queues for content processing• Per segment caching• Faster commits• Numeric range improvements• Scoring only done when requested



Dictionary Management

• The ability to manage the search experience dynamically to improve query matching

• Synonyms, Acronyms and Lemmas


DICTIONARY MANAGEMENT



Dictionary Management Overview

• Query Side Dictionaries• Enrich the query• Benefits

• GUI• Real time, change on the fly – no reindexing

• vs Ingest Side Dictionaries• Requires reindexing

9


Query Enrichment – Query Lifecycle10

• Query Lifecycle Presentation

QUERY LIFECYCLE

Prepared by:Daryl Gies, Director Professional Services

David Basham, Engineer Professional Services

10/13/2010

1


Dictionary Admin Overview

• Synonyms, Acronyms, Lemmas

11

•Generic Infrastructure•Support Hyponyms, Acronyms, etc.


Dictionary Admin - Synonyms12

• Synonym dictionary entries make associations between words of similar meanings or usage.

• For example, car and automobile are terms that can be used to reference the same item as are the terms blouse and shirt. With synonyms, you may wish to define entries as unidirectional rather than bidirectional.


Dictionary Admin - Acronyms13

• Acronym dictionary entries make associations between abbreviations and their expanded words or phrases.

• For example, CIA is an abbreviation for Central Intelligence Agency.

• For example, IBM is an abbreviation for International Business Machine.


Dictionary Admin - Lemmas14

• A Lemmatization dictionary entry allows you to make an association between a root word and its variations.

• Lemmatization is the inverse of stemming in that stemming reduces words to a root and lemmatization expands words to include all variants.

• For example, take the verb "jump". This word can appear in the form of "jumps", "jumped", and "jumping". The word "run" could appear as "runs", "running", or "ran".


Setting up a Synonym Dictionary

• Create dictionary• Add synonyms• Publish dictionary

15


Dictionary Admin – Dictionary Properties

• Name • Locale (Language) – corresponds to the language specified at query time.• Bidirectional – controls term expansion behavior.

• Unidirectional: When you encounter this term also search for these terms.

• Bidirectional: When you encounter any of these terms also search for these terms.

• Active – enable or disable a dictionary.

16


Dictionary Admin – Dictionary Properties - Boosting17

• Use a boost of 0 (zero) so that expansions are not factored in when relevancy scores are calculated.

• Use a boost of 50 so that documents found using the expansions count half as much as documents found with the original term.

• Use a boost of 100 so that documents found using the expansions count the same as documents found with the original term.

• Use a boost higher than 100 if you want documents that match the expanded terms to count more than the user-entered term.


Dictionary Admin – UI18


Dictionary Admin – UI19

Teaser - Abstract


THANK YOU

Attivio - What's New in AIE 2 and Dictionary Management

Technology

Transcript of Attivio - What's New in AIE 2 and Dictionary Management