Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

67
Search Intelligently Building the Digital Experience with Liferay DXP, new queries and Elasticsearch André Ricardo Barreto de Oliveira ("Arbo") Search Infrastructure Lead - Liferay, Inc. Chicago, USA September 27, 2016

Transcript of Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Page 1: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Search IntelligentlyBuilding the Digital Experience with

Liferay DXP, new queries and Elasticsearch

André Ricardo Barreto de Oliveira ("Arbo")Search Infrastructure Lead - Liferay, Inc.

Chicago, USASeptember 27, 2016

Page 2: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Searchin the age of

Digital Transformation

Page 3: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

The classic approach to Search

● A tool for content retrieval, and little else

● Analyze - Index - Query - Display - Repeat

● Website finished: "add a Search Bar" as an afterthought

● No real correlation between Search and Business Case

@arbocombr

Page 4: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Meanwhile, in the Search landscape...

● Big Data

● Analytics and Statistics

● Post-text content

● Maps and Geolocation

@arbocombr

● Natural Language Processing

● Bots

● Machine Learning

● Artificial Intelligence

Page 5: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Search meets Digital Transformation

● Well-designed Search, now central to digital business

● A search is often the starting point for the User Journey

● Results with nonlinear interaction- and action

● Impossible to tell apart: Search and the Digital Experience

@arbocombr

Page 6: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

TheSearch Centric

Digital Experience

@arbocombr

Page 7: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Business assets designed for Search

@arbocombr

Page 8: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 9: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 10: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 11: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 12: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Turn chaotic into manageable

@arbocombr

Page 13: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 14: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 15: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 16: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 17: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Find the needle in the haystack

@arbocombr

Page 18: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 19: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 20: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 21: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 22: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Predict user intention

@arbocombr

Page 23: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 24: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 25: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 26: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 27: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Show what they need to see

@arbocombr

Page 28: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 29: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 30: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 31: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 32: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Liferay DXP Demo:What's new in Search

@arbocombr

Page 33: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 34: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 35: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 36: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 37: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 38: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 39: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 40: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 41: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 42: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 43: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Liferay DXP Search: Demo Time

New and upcoming features:

building your own Search experience

@arbocombr

Page 44: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Buildingintelligent

queries

@arbocombr

Page 45: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

The classic approach to queries

● Full text "bag of words" → all results with same relevance

● Substring (*wildcards*) → performance hit in large indices

● Limited flexibility for special parsing cases (e.g. emails)

● Manual configuration for fields with custom analyzers

@arbocombr

Page 46: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Intelligent queries in Liferay DXP

● Modular OSGi extension points for query builders

● Construct sophisticated, compound queries per field

● Add custom analyzers and type mappings programmatically

● Design searchable assets and fine tune relevance

@arbocombr

Page 47: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Rich data requires tailored search

@arbocombr

TITLE(Short, autocomplete aware)

Description(Lengthy, full text)

Email address(Special formats, symbols)

Geolocation(Coordinates)

Page 48: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Description field style - ready to use

● Classic full text search

● Match by any number of words (or phrases in quotes)

● Proximity: words near each other become top results

@arbocombr

Page 49: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Title field style - ready to use

● Full text search like classic Description- with tweaks

● Autocomplete ready: match to start will boost relevance

● A perfect match to the exact title becomes a top result

● Ignore proximity, since titles are short- best performance

@arbocombr

Page 50: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Substring field style (discouraged)

● Find anywhere in "keyword" field: "[email protected]"

● Not actually analyzed- full scan- kills performance

● Kept for backward compatibility only ("wildcard", "like")

● Use intelligent mappings, analyzers and queries instead

@arbocombr

Page 51: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Page 52: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

An intelligent platform

● A well designed User Journey- an increase in Search volume

● Content suggestion- incremental filters- advanced queries

● Liferay DXP: focus on application and content management

● Search engine: external scalability, dynamic advantages

@arbocombr

Page 53: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Elasticsearch:the Liferay DXP

official search engine

@arbocombr

Page 54: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

● Match

● Multi Match

● Match All

● Query String

● Term / Terms

Elasticsearch queries for any use case

@arbocombr

● Regexp

● Fuzzy

● Type

● Ids

● DisMax

● REST API for compound queries- Lucene with better syntax

● Range

● Exists

● Missing

● Prefix

● Wildcard

● Geo Distance

● Geo Distance Range

● Geo Bounding Box

● Geo Polygon

● More Like This

Page 55: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Fine tune relevance rapidly...

… then bring it back into your Liferay search

@arbocombr

GET /cars/transactions/_search{ "query" : { "constant_score": { "filter": { "range": { "price": { "gte": 10000 } } } } }, "aggs" : { "single_avg_price": { "avg" : { "field" : "price" } } }}

Page 56: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Similarity suggestion: More Like This

● Liferay DXP: MoreLikeThisQuery

● User viewing blogs, documents, your own custom entities

● Automatically suggest related assets, based on content

● Full text and specific fields

@arbocombr

Page 57: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Massive queries need great scalability

@arbocombr

Page 58: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

A seamless transition

@arbocombr

Production modeDev mode

Page 59: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Enterprise-grade security: Shield

@arbocombr

Page 60: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Cluster monitoring: Marvel

@arbocombr

Page 61: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Visualization: Kibana

@arbocombr

Page 62: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Takeaways

@arbocombr

Page 63: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

The Digital Experience and Search

● Digital Transformation: not just index-and-find anymore

● Your User Journey will often start with a search

● Effective matches and refinements generate business

● With more user searches, underlying platform must scale

@arbocombr

Page 64: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Liferay DXP: innovations in Search

● Elasticsearch: Lucene at core, improvements at every level

● Enterprise-grade Search with Shield, Marvel and Kibana

● Maximum scalability, decoupled from the DXP footprint

● Flexibility with modular API and OSGi extension points

@arbocombr

Page 65: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Intelligent queries for all use cases

● DXP: many new filters and queries to mix and match

● Ultimate relevance with per-field analyzers, queries, boosts

● Geolocation and more native field type mappings

@arbocombr

Page 66: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Intelligent Search for your User Journey

● Leverage modularity for fast, extensible development

● Small components that modify queries and share results

● Aggregations, filters, boxes, lists, maps, custom UI portlets

● Search Pages tailored to the needs of your business

@arbocombr

Page 67: Search Intelligently - Liferay Symposium North America 2016, Chicago, USA

Thank you- The Liferay Search Infrastructure Team -

@arbocombr

André de Oliveira ➤ Lead, Engineering (USA)

Tibor Lipusz ➤ SME, Support (Hungary)

Felipe Pires, Vitor Fernandes ➤ Design (Brazil)

Rodrigo Paulino ➤ Back-end (Brazil)

Jonathan Mak, Kevin Tan ➤ Front-end (USA)

Albert Lee, Brian Lee ➤ QA (USA)

Russell Bohl, Rich Sezov ➤ Tech writing (USA)

David Truong, Michael Han ➤ Product Management (USA)

http://j.mp/SearchLiferayNorthAmerica2016

[email protected]

github.com/arboliveira