Sharing content between hippo and solr

Post on 16-May-2015

815 views 2 download

Tags:

description

Presentation I have given at the hippo gettogether august 2010 about the lucene connector framework

Transcript of Sharing content between hippo and solr

Sharing content between Hippo and Solr

Jettro Coenradie [jettro@jteam.nl]Chief Architect JTeam B.V.

vrijdag 20 augustus 2010

Why ?

vrijdag 20 augustus 2010

Why ?

• Multiple content repositories, one access point

vrijdag 20 augustus 2010

Why ?

• Multiple content repositories, one access point

• More search options than out of the box available in hippo

vrijdag 20 augustus 2010

JTeam involvement

• Experience with Solr

• Experience with Hippo

vrijdag 20 augustus 2010

JTeam involvement

• Experience with Solr

• Experience with Hippo

• Customer demand

vrijdag 20 augustus 2010

JTeam involvement

• Experience with Solr

• Experience with Hippo

• Customer demand

• Ralph Ruijs internship

vrijdag 20 augustus 2010

[LCF] LuceneConnector Framework

• Data synchronization framework

• Originated from MetaCarta project

• Apache incubator project

• http://incubator.apache.org/connectors/

vrijdag 20 augustus 2010

[LCF] Architecture

vrijdag 20 augustus 2010

• Start example that comes with solr

• Change the schema [add fields]

• Start the Solr explorer

http://www.jteam.nl/news/solrexplorer.htmlhttp://lucene.apache.org/solr/

vrijdag 20 augustus 2010

LCF Crawler ui

• configure repository connectors

• configure output connectors

• configure jobs

• Start the job

vrijdag 20 augustus 2010

vrijdag 20 augustus 2010

Building LCF

vrijdag 20 augustus 2010

Building LCF

• Build from source using ant

vrijdag 20 augustus 2010

Building LCF

• Build from source using ant

• Deploy artifacts to artifactory

vrijdag 20 augustus 2010

Building LCF

• Build from source using ant

• Deploy artifacts to artifactory

• Use maven to create war with connectors

vrijdag 20 augustus 2010

Building LCF

• Build from source using ant

• Deploy artifacts to artifactory

• Use maven to create war with connectors

• Use maven to build agent with connectors

vrijdag 20 augustus 2010

Building LCF

• Build from source using ant

• Deploy artifacts to artifactory

• Use maven to create war with connectors

• Use maven to build agent with connectors

• Use special classes to prepare environment.

vrijdag 20 augustus 2010

Create an LCF Connector

• UI related methods

• Connection related methods

• Runner related methods

• UI related methods

• Connection related methods

• Runner related methods

vrijdag 20 augustus 2010

Create an LCF Connector

IConnector

IOutputConnector IRepositoryConnector

IConnector

IOutputConnector IRepositoryConnector

vrijdag 20 augustus 2010

Create an LCF Connector

installdeinstallconnectdisconnectcheckpollgetConfigurationviewConfigurationoutputConfigurationHeaderoutputConfigurationBodyprocessConfigurationPost

IConnectorinstalldeinstallconnectdisconnectcheckpollgetConfigurationviewConfigurationoutputConfigurationHeaderoutputConfigurationBodyprocessConfigurationPost

IConnector

vrijdag 20 augustus 2010

Create an LCF Connector

executeCommandaddOrReplaceDocumentremoveDocumentviewSpecificationoutputSpecificationHeaderoutputSpecificationBodyprocessSpecificationPost

IOutputConnectorexecuteCommandaddOrReplaceDocumentremoveDocumentviewSpecificationoutputSpecificationHeaderoutputSpecificationBodyprocessSpecificationPost

IOutputConnector

vrijdag 20 augustus 2010

Create an LCF Connector

executeCommandaddSeedDocumentsprocessDocumentsremoveDocumentviewSpecificationoutputSpecificationHeaderoutputSpecificationBodyprocessSpecificationPost

IRepositoryConnectorexecuteCommandaddSeedDocumentsprocessDocumentsremoveDocumentviewSpecificationoutputSpecificationHeaderoutputSpecificationBodyprocessSpecificationPost

IRepositoryConnector

vrijdag 20 augustus 2010

[Bad] code base

vrijdag 20 augustus 2010

[Bad] code base

• Build without dependency management

vrijdag 20 augustus 2010

[Bad] code base

• Build without dependency management

• No standard project layout

vrijdag 20 augustus 2010

[Bad] code base

• Build without dependency management

• No standard project layout

• Large classes with to many concerns

vrijdag 20 augustus 2010

[Bad] code base

• Build without dependency management

• No standard project layout

• Large classes with to many concerns

• Positive - working on improving it

vrijdag 20 augustus 2010

Thumbs up or down

vrijdag 20 augustus 2010

Thumbs up or down

• Code base can be a lot better

vrijdag 20 augustus 2010

Thumbs up or down

• Code base can be a lot better

• Good idea

vrijdag 20 augustus 2010

Thumbs up or down

• Code base can be a lot better

• Good idea

• Good start

vrijdag 20 augustus 2010

Questions

vrijdag 20 augustus 2010