Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014
-
Upload
grid-dynamics -
Category
Technology
-
view
693 -
download
0
Transcript of Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014
![Page 1: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/1.jpg)
![Page 2: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/2.jpg)
Faceting with Lucene
Block Join Query
![Page 3: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/3.jpg)
Agenda
1. Why we need special faceting for
Block Join queries?
1. Proposed Block Join facet component.
![Page 4: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/4.jpg)
PRIVILEGED AND CONFIDENTIAL
Introducing myself
Oleg Savrasov, PhD
A programmer
Working for Grid Dynamics
(griddynamics.com)
Work and live in Saint-Petersburg,
Russia
![Page 5: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/5.jpg)
Online shopping
![Page 6: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/6.jpg)
Jerrica is looking for a dress
![Page 7: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/7.jpg)
Huge amount of dresses
![Page 8: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/8.jpg)
Facet filters help
Facet
filters
Reduced
amount
![Page 9: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/9.jpg)
Tasks to be solved
● Performant Search
● Facet
calculation/filtering
FacetComponent ?
![Page 10: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/10.jpg)
Product has many SKU
![Page 11: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/11.jpg)
Aggregated facet counts
Facets should
count products,
not SKU.
Expected
facets:
COLOR
Blue : 1
Red : 1
SIZE
S : 1
M : 1
![Page 12: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/12.jpg)
Flat documents don’t help
False positive match for
+COLOR:Blue +SIZE:M
![Page 13: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/13.jpg)
Separate SKU documents
q = *:*
facet.field = COLOR
facet.field = SIZE
COLOR
Blue : 1
Red : 2
SIZE
S : 2
M : 1
Wrong
numbers!
There is
only one
product
![Page 14: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/14.jpg)
Search products only
q = *:*
fq = scope:product
facet.field = COLOR
facet.field = SIZE
COLOR : 0
SIZE : 0
No such
fields in
product
documents
![Page 15: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/15.jpg)
Aggregated facet counts
Facets should
count products,
not SKU.
Expected
facets:
COLOR
Blue : 1
Red : 1
SIZE
S : 1
M : 1
![Page 16: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/16.jpg)
Solr Block Join Support (since Lucene 3.4.0)
Gre
en
Blu
e
Yello w
Ye
llo w
Blu
e
Gre
en
Pro
du
ct
Gre
en
Ye
llo w
Pro
du
ct
Gre
en
Blu
e
Yello w
Ye
llo w
Pro
du
ct
docId
1 1 1
Query: {!parent which="scope:product"}COLOR:Blue
1 1
scope:product
COLOR:Blue
ToParentQuery 1 1
Child docs Parent doc
Block1
![Page 17: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/17.jpg)
SOLR-5743 Faceting with Block Join support
● Create BlockJoinFacetComponent
● Only DocValues fields are
supported
● Facet counts should correspond to
amount of parent documents
● ToParentQuery is expected
![Page 18: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/18.jpg)
Faceting over DocSet slicesG
reen
Blu
e
Ye
llo w
Ye
llo w
Blu
e
Gre
en
Pro
du
ct
Gre
en
Ye
llo w
Pro
du
ct
Gre
en
Blu
e
Ye
llo w
Ye
llo w
Pro
du
ct
docId
10 1 0 0 1 0
DocSet Slice
DocSet Slice counts
COLOR Blue : 2
Aggregated counts
COLOR Blue : +1
![Page 19: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/19.jpg)
Block Join Facet Component
![Page 20: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/20.jpg)
BlockJoinFacetCollector
![Page 21: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/21.jpg)
Facets counting
![Page 22: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/22.jpg)
It works!
q =
{!parent
which="scope:product"}COLOR:Blue
child.facet.field = SIZE
<response>
...
<lst name="facet_counts">
<lst name="facet_fields">
<lst name="SIZE">
<int name="S">14</int>
<int
name="L">22</int>
<int
name="XL">17</int>
</lst>
</lst>
</lst>
</response>
![Page 23: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/23.jpg)
The dress is found
![Page 24: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/24.jpg)
Further improvements
● Thorough profiling
● Performance improvements
● Algorithmic improvements
![Page 25: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/25.jpg)
References
http://www.slideshare.net/MarkHarwood/proposal-for-nested-document-support-
in-lucene
http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html
http://blog.griddynamics.com/2013/09/solr-block-join-support.html
![Page 26: Faceting with Lucene Block Join Query - Lucene/Solr Revolution 2014](https://reader031.fdocuments.us/reader031/viewer/2022013105/55a6c9a61a28ab4e1d8b4758/html5/thumbnails/26.jpg)
Big thanks!
Do you have any questions?
Please vote for SOLR-5743.