Hibernate Search - GOTO...

55

Transcript of Hibernate Search - GOTO...

Page 1: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database
Page 2: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Hibernate SearchHardy Ferentschik, Red Hat

Page 3: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

The toolbox

Page 4: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Build tool Ant/Maven

Container Tomcat/JBoss

MVC Struts/Seam

Domain model JPA/Hibernate

Search ?

The toolbox

Page 5: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Build tool Ant/Maven

Container Tomcat/JBoss

MVC Struts/Seam

Domain model JPA/Hibernate

Search ?

The toolbox

Page 6: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Build tool Ant/Maven

Container Tomcat/JBoss

MVC Struts/Seam

Domain model JPA/Hibernate

Search ?

The toolbox

Page 7: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Build tool Ant/Maven

Container Tomcat/JBoss

MVC Struts/Seam

Domain model JPA/Hibernate

Search ?

The toolbox

Page 8: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Build tool Ant/Maven

Container Tomcat/JBoss

MVC Struts/Seam

Domain model JPA/Hibernate

Search ?

The toolbox

Page 9: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

“LIKE” queriestitle = (title == null) ? "%" : "%" + title.toLowerCase() + "%";actor = (actor == null) ? "%" : "%" + actor.toLowerCase() + "%";

em.createQuery( "select distinct p from Product p JOIN p.actors a " + "where lower(p.title) like :title " + "and lower(a.name) LIKE :actor order by p.title") .setParameter("title", title) .setParameter("actor", actor));

Page 10: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

SQL shortcomings

Page 11: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

SQL shortcomingsWildcard / word

search%hibernate%

Approximation hybernat

Proximity ‘Java’ close to ‘Persistence’

Result scoring

Multi-”column” search

Page 12: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

SQL shortcomingsWildcard / word

search%hibernate%

Approximation hybernat

Proximity ‘Java’ close to ‘Persistence’

Result scoring

Multi-”column” search

Page 13: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

SQL shortcomingsWildcard / word

search%hibernate%

Approximation hybernat

Proximity ‘Java’ close to ‘Persistence’

Result scoring

Multi-”column” search

Page 14: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

SQL shortcomingsWildcard / word

search%hibernate%

Approximation hybernat

Proximity ‘Java’ close to ‘Persistence’

Result scoring

Multi-”column” search

Page 15: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

SQL shortcomingsWildcard / word

search%hibernate%

Approximation hybernat

Proximity ‘Java’ close to ‘Persistence’

Result scoring

Multi-”column” search

Page 16: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Lucene

Powerful fulltext search engine

Open Source

In the TOP 10 of downloaded Apache projects

Page 17: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Lucene DIY

Structural mismatch

Retrieval mismatch

Synchronization mismatch

Page 18: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Structural Mismatch

Page 19: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Structural Mismatch

Page 20: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Retrieval Mismatch

• Index contains Documents not Objects

• Even if you re-hydrate you don’t have managed objects

Page 21: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Synchronization mismatch

Page 22: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Synchronization mismatch

Page 23: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database
Page 24: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Configure

• Enable Search via event listeners

• Add Backend options

Page 25: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Configure

• Enable Search via event listeners

• Add Backend options

<property name="hibernate.search.default.indexBase" value="/var/lucene/indexes"/>

Page 26: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Annotate@Entity @Indexedpublic class Essay { ... @Id public Long getId() { return id; }

@Field public String getSummary() { return summary; }

@Lob @Field public String getText() { return text; }

@ManyToOne @IndexedEmbedded public Author getAuthor() { return author; }}

Page 27: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Annotate@Entity @Indexedpublic class Essay { ... @Id public Long getId() { return id; }

@Field public String getSummary() { return summary; }

@Lob @Field public String getText() { return text; }

@ManyToOne @IndexedEmbedded public Author getAuthor() { return author; }}

Page 28: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Annotate@Entity @Indexedpublic class Essay { ... @Id public Long getId() { return id; }

@Field public String getSummary() { return summary; }

@Lob @Field public String getText() { return text; }

@ManyToOne @IndexedEmbedded public Author getAuthor() { return author; }}

Page 29: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Annotate@Entity @Indexedpublic class Essay { ... @Id public Long getId() { return id; }

@Field public String getSummary() { return summary; }

@Lob @Field public String getText() { return text; }

@ManyToOne @IndexedEmbedded public Author getAuthor() { return author; }}

Page 30: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Annotate@Entity @Indexedpublic class Essay { ... @Id public Long getId() { return id; }

@Field public String getSummary() { return summary; }

@Lob @Field public String getText() { return text; }

@ManyToOne @IndexedEmbedded public Author getAuthor() { return author; }}

Page 31: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

SearchString searchQuery = "Hibernate Search";String[] productFields = {"summary", "author.name"};

// LuceneQueryParser parser = new MultiFieldQueryParser(productFields, new StandardAnalyzer());Query luceneQuery = parser.parse(searchQuery);

// Hibernate SearchFullTextEntityManager ftEm = Search. getFullTextEntityManager((EntityManager)em);

FullTextQuery query = ftEm.createFullTextQuery( luceneQuery, Essay.class );

List<Essay> items = query.getResultList();

Page 32: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Database

Lucene Directory(Index)

Hibernate +

Hibernate Search

Search requestIndex update

Hibernate +

Hibernate Search

Search requestIndex update

Architecture

Page 33: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Database

Lucene Directory(Index)

Hibernate +

Hibernate Search

Search requestIndex update

Hibernate +

Hibernate Search

Search requestIndex update

Architecture

Page 34: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

The goodies

Page 35: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Analyzers• Take text as an input, chunk it into individual

words and optionally applying a chain of filter operations on the tokens.

Page 36: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

2009Welcome to JAOO

Document

title

java

welcomjaoo

Page 37: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

2009

Welcome

to JAOO

Document

title

java

welcom

jaoo

Page 38: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

2009

Welcome

to

JAOO

Document

title

java

welcom

jaoo

Page 39: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

2009

Welcome

to

JAOO

Document

title

java

welcomjaoo

Page 40: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

2009

Welcome

to

JAOO

Document

title java

welcomjaoo

Page 41: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

2009

Welcome

to

JAOO

Document

title java

welcomjaoo

Page 42: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

@Entity@Indexed@AnalyzerDef(name = "customanalyzer", tokenizer = @TokenizerDef(factory=StandardTokenizerFactory.class), filters = { @TokenFilterDef(factory = LowerCaseFilterFactory.class), @TokenFilterDef(factory = SnowballPorterFilterFactory.class, params = {@Parameter(name = "language", value = "English")}) })public class Book { ... @Field(index=Index.TOKENIZED, store=Store.NO) @Analyzer(definition = "customanalyzer") private String title; ... }

Analyzer Example

Page 43: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Filters

Page 44: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Filters

Page 45: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Filters

Page 46: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

@Entity@Indexed@FullTextFilterDefs( { @FullTextFilterDef(name="bestDriver", impl=BestDriversFilter.class), @FullTextFilterDef(name="security", impl=SecurityFilterFactory.class) })public class Driver { ... }

Filter Example

...fullTextQuery = s.createFullTextQuery( query, Driver.class );fullTextQuery.enableFullTextFilter("bestDriver");fullTextQuery.enableFullTextFilter("security"). setParameter( "login", "andre" );fullTextQuery.list();...

Page 47: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

@Entity@Indexed@FullTextFilterDefs( { @FullTextFilterDef(name="bestDriver", impl=BestDriversFilter.class), @FullTextFilterDef(name="security", impl=SecurityFilterFactory.class) })public class Driver { ... }

Filter Example

...fullTextQuery = s.createFullTextQuery( query, Driver.class );fullTextQuery.enableFullTextFilter("bestDriver");fullTextQuery.enableFullTextFilter("security"). setParameter( "login", "andre" );fullTextQuery.list();...

Page 48: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

@Entity@Indexed@FullTextFilterDefs( { @FullTextFilterDef(name="bestDriver", impl=BestDriversFilter.class), @FullTextFilterDef(name="security", impl=SecurityFilterFactory.class) })public class Driver { ... }

Filter Example

...fullTextQuery = s.createFullTextQuery( query, Driver.class );fullTextQuery.enableFullTextFilter("bestDriver");fullTextQuery.enableFullTextFilter("security"). setParameter( "login", "andre" );fullTextQuery.list();...

Page 49: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Projection

Projection on metadata (SCORE, BOOST, ID, ...)

No DB access

Page 50: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

FullTextQuery query = s.createFullTextQuery(luceneQuery,Book.class );query.setProjection( "id", "summary", "body", "mainAuthor.name" );

List results = query.list();Object[] firstResult = (Object[]) results.get(0);

Integer id = firstResult[0];String summary = firstResult[1];String body = firstResult[2];String authorName = firstResult[3];

Projection Example

Page 51: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Clustering

DatabaseHibernate +

Hibernate Search

JMS queue

Lucene Directory(Index)Master

Hibernate +

Hibernate SearchProcessIndex update

Index update order

Lucene Directory(Index)Copy

Search request

Copy

Slave

Master

Page 52: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

And even more

Index sharding

Automatic index optimization

Manual indexing and purging

Shared Lucene resources

Access to native Lucene

Page 53: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Hibernate Search

Fulltext search without the hassle!

Page 54: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

Q&A

Page 55: Hibernate Search - GOTO Conferencegotocon.com/dl/...SearchedAndFoundFullTextSearchWithHibernate.pdf · Hibernate + Hibernate Search Search request Index update Architecture. Database

More InfoHibernate Search- http://search.hibernate.org- Hibernate Search in ActionApache Lucene- http://lucene.apache.org- Lucene In Actionhttp://in.relation.tohttp://forum.hibernate.org/[email protected]