Proprietary & Confidential

WHAT'S NEW IN AIE 2AND

DICTIONARY MANAGEMENT

Traction User Group10/14/2010

Proprietary & Confidential

Overview2

• What’s New In AIE 2• Dictionary Management

Proprietary & Confidential

WHAT'S NEW IN AIE 2.1.1

Traction User Group10/14/2010

Proprietary & Confidential

What’s New In AIE 2.14

• Relevancy Improvements• Core Indexing and Search Improvements• Dictionary Management

Proprietary & Confidential

What’s New In AIE 2.15

Relevancy Improvements

• Field length normalization enhancement for improved title matching

• Proximity boost enhancement

Proprietary & Confidential

What’s New In AIE 2.16

Core Indexing and Search Improvements

• Real time fields redesigned to reduce disk IO• Memory capped queues for content processing• Per segment caching• Faster commits• Numeric range improvements• Scoring only done when requested

Proprietary & Confidential

What’s New In AIE 2.17

Dictionary Management

• The ability to manage the search experience dynamically to improve query matching

• Synonyms, Acronyms and Lemmas

Proprietary & Confidential

DICTIONARY MANAGEMENT

Traction User Group10/14/2010

Proprietary & Confidential

Dictionary Management Overview

• Query Side Dictionaries• Enrich the query• Benefits

• GUI• Real time, change on the fly – no reindexing

• vs Ingest Side Dictionaries• Requires reindexing

9

Proprietary & Confidential

Query Enrichment – Query Lifecycle10

• Query Lifecycle Presentation

QUERY LIFECYCLE

Prepared by:Daryl Gies, Director Professional Services

David Basham, Engineer Professional Services

10/13/2010

1

Proprietary & Confidential

Dictionary Admin Overview

• Synonyms, Acronyms, Lemmas

11

•Generic Infrastructure•Support Hyponyms, Acronyms, etc.

Proprietary & Confidential

Dictionary Admin - Synonyms12

• Synonym dictionary entries make associations between words of similar meanings or usage.

• For example, car and automobile are terms that can be used to reference the same item as are the terms blouse and shirt. With synonyms, you may wish to define entries as unidirectional rather than bidirectional.

Proprietary & Confidential

Dictionary Admin - Acronyms13

• Acronym dictionary entries make associations between abbreviations and their expanded words or phrases.

• For example, CIA is an abbreviation for Central Intelligence Agency.

• For example, IBM is an abbreviation for International Business Machine.

Proprietary & Confidential

Dictionary Admin - Lemmas14

• A Lemmatization dictionary entry allows you to make an association between a root word and its variations.

• Lemmatization is the inverse of stemming in that stemming reduces words to a root and lemmatization expands words to include all variants.

• For example, take the verb "jump". This word can appear in the form of "jumps", "jumped", and "jumping". The word "run" could appear as "runs", "running", or "ran".

Proprietary & Confidential

Setting up a Synonym Dictionary

• Create dictionary• Add synonyms• Publish dictionary

15

Proprietary & Confidential

Dictionary Admin – Dictionary Properties

• Name • Locale (Language) – corresponds to the language specified at query time.• Bidirectional – controls term expansion behavior.

• Unidirectional: When you encounter this term also search for these terms.

• Bidirectional: When you encounter any of these terms also search for these terms.

• Active – enable or disable a dictionary.

16

Proprietary & Confidential

Dictionary Admin – Dictionary Properties - Boosting17

• Use a boost of 0 (zero) so that expansions are not factored in when relevancy scores are calculated.

• Use a boost of 50 so that documents found using the expansions count half as much as documents found with the original term.

• Use a boost of 100 so that documents found using the expansions count the same as documents found with the original term.

• Use a boost higher than 100 if you want documents that match the expanded terms to count more than the user-entered term.

Proprietary & Confidential

Dictionary Admin – UI18

Proprietary & Confidential

Dictionary Admin – UI19

Teaser - Abstract

Proprietary & Confidential

THANK YOU