Post on 18-Aug-2015
Scalable Search and AnalyticsRavi Krishnamurthy, VP Technical Services, ravi@lucidworks.com
Yann Yu, Systems Engineer, yann.yu@lucidworks.com
• Motivation: Why Search AND Analytics?
• Apache Solr and Lucidworks SILK
• Solution Architectures
• Demo(s)
• Q & A
• Resources
Agenda
Secure access to all your data through one interface, empowering everyone in your organization to access the data they need.
Search is the key to unlocking big data.
vSearch anything.
Search can be smarter.
location search history query permissions context
Personal, contextual, relevant results: consumer-like simplicity and power in the enterprise.
Solr in a nutshell
8M+ total downloads
Solr is both established & growing
250,000+monthly downloads
Largest community of developers.
2500+open Solr jobs.
Solr most widely used search solution on the planet.
LucidworksUnmatched Solr expertise.
1/3of the active committers
70%of the open source code is committed
Lucene/Solr Revolutionworld’s largest open source user
conference dedicated to Lucene/Solr.
Solr has tens of thousands of applications in production.
You use Solr everyday.
• Search-first NoSQL store
• Distributed, Horizontally Scalable
• Stable and Robust
• Deep Paging
• Accurate Facets and Stats
• Stats on Pivots (5.0)
• Easier to start-up; run as a service on Linux (5.0)
• Your Content, Your Way (5.0)
Solr and Analytics
• Solr - Logstash - Kibana
• http://lucidworks.com/product/integrations/silk/
• Open source at:
• https://github.com/LucidWorks/banana
• https://github.com/LucidWorks/solrlogmanager
SiLK
data enrichment
your business
your app
your datamachine learning
recommendations landing pages relevancy tuningsecurity
connector framework signal processing
api reporting admin
Lucidworks FusionEverything your team needs to rapidly design and deploy next-generation search apps to your entire organization.
Enterprise Search
Lucidworks connectors processes documents and
sends to SolrCloud
Standard document storage and search
Log record search
Machine generated log records are sent to Flume.
Flume forwards raw log record to Hadoop for archiving.
Flume simultaneously parses out data in record into a Solr document,
forwarding resulting document to Solr
Lucidworks SiLK exposes real-time statistics and analytics to end-users,
as well as full-text search
High volume indexing of many small records
• Solr: http://lucene.apache.org/solr
• Company: http://www.lucidworks.com
• Our blog: http://www.lucidworks.com/blog
• Blog on stats and facets: http://lucidworks.com/blog/you-got-stats-in-my-facets/
• Fusion: http://www.lucidworks.com/products/fusion
• Solr for Data Science code: http://github.com/lucidworks/solr-for-datascience
• Email: ravi@lucidworks.com; yann.yu@lucidworks.com
Resources