Administering and Monitoring SolrCloud Clusters
-
Upload
rafal-kuc -
Category
Entertainment & Humor
-
view
1.020 -
download
1
description
Transcript of Administering and Monitoring SolrCloud Clusters
![Page 1: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/1.jpg)
Administering and Monitoring SolrCloud
Rafał Kuć – Sematext Group, Inc.@kucrafal @sematext sematext.com
![Page 2: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/2.jpg)
Ta me…
Sematext consultant & engineerSolr.pl co-founderFather and husband
![Page 3: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/3.jpg)
Solr Server
SolrCloud Concepts
Solr Server
Solr Server Solr Server
Shard1 Replica
Shard2 Replica
Shard2Shard1
Application
![Page 4: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/4.jpg)
Local SolrCloud Cluster
java -Dbootstrap_confdir=./solr/revolution/conf -Dcollection.configName=revolution -DzkRun -DnumShards=1 -jar start.jar
Runs embedded ZooKeeperBootstraps collection with 1 shardsStarts Solr
![Page 5: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/5.jpg)
Starting Solr Cluster
ZooKeeper ZooKeeper ZooKeeper
Solr Server Solr Server
-DzkHost=192.168.1.2:2181,192.168.1.1:2181,192.168.1.3:2181
Solr Server Solr Server
-DzkHost=192.168.1.1:2181,192.168.1.2:2181,192.168.1.3:2181
-DzkHost=192.168.1.3:2181,192.168.1.1:2181,192.168.1.2:2181
-DzkHost=192.168.1.3:2181,192.168.1.1:2181,192.168.1.2:2181
No Collection
No Collection No Collection
No Collection
![Page 6: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/6.jpg)
Uploading Collection Configuration
./zkcli.sh -cmd upconfig -zkhost 192.168.1.1:2181 -confdir ./conf/ -confname revolution
ZooKeeper
ZooKeeper
ZooKeeper
Collection configuration Solr
![Page 7: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/7.jpg)
Collections APICreate
Delete
Reload
Split
Create Alias
Delete Alias
Shard Creation/Deletionhttp://wiki.apache.org/solr/SolrCloud
![Page 8: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/8.jpg)
Collection Creation
curl 'http://solrhost:8983/solr/admin/collections?action=CREATE&name=revolution&numShards=3&replicationFactor=4'
name
numShards
replicationFactor
maxShardsPerNode
createNodeSet
collection.configName
![Page 9: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/9.jpg)
Collection Split Example
$ curl 'http://solr1:8983/solr/admin/collections?action=CREATE&name=collection1&numShards=2&replicationFactor=1'
![Page 10: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/10.jpg)
Collection Split Example
$ curl 'http://localhost:8983/solr/admin/collections?action=SPLITSHARD&collection=collection1&shard=shard1'
![Page 11: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/11.jpg)
Getting Deeper – CoreAdmin API
curl 'http://solrhost:8983/solr/admin/cores?action=CREATE&name=newcore&collection=revolution&shard=shard2'
collection
shard
numShards
collection.configName
![Page 12: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/12.jpg)
Schema – the API
Reading (Solr 4.2)FieldsDynamic fieldsTypesCopy fieldsName (4.3)Version (4.3)Unique Key (4.3)Similarity (4.3)
Writing (Solr 4.4)Adding new fieldsAdding copy fields
![Page 13: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/13.jpg)
Reading Your Schema
curl -XGET 'http://solrhost:8983/solr/rev/schema/fields/name'
Full reference: http://wiki.apache.org/solr/SchemaRESTAPI
{ "responseHeader" : { "status" : 0, "QTime" : 5 }, "field" : { "name" : "name", "type" : "text_general", "indexed" : true, "stored" : true }}
![Page 14: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/14.jpg)
Dynamic Schema Modifications<schemaFactory class="ManagedIndexSchemaFactory"> <bool name="mutable">true</bool> <str name="managedSchemaResourceName">managed-schema</str> </schemaFactory>
curl -XPUT 'http://solrhost:8983/solr/rev/schema/fields/content' –d'{ "type" : "text", "stored" : "false", "copyFields" : ["catchAll"]}'
curl -XPOST 'http://solrhost:8983/solr/rev/schema/copyFields' -d '[ { "source" : "name", "dest" : [ "text", "personal" ] }]'
![Page 15: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/15.jpg)
The Right Directory
_0.fdt _0.fdx _0.fnm _0.nvd
_1.fdt _1.fdx _1.fnm _1.nvd
StandardDirectory
SimpleFSDirectory
NIOFSDirectory
MMapDirectory
NRTCachingDirectory
RAMDirectory <directoryFactory name="DirectoryFactory" class="solr.NRTCachingDirectoryFactory" />
![Page 16: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/16.jpg)
Segment Merging
a b c d e
Level 0 Level 1
cf g
![Page 17: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/17.jpg)
Segment Merge Under Control
Merge policy
Merge scheduler
Merge factor
Merge policy configuration
https://cwiki.apache.org/confluence/display/solr/IndexConfig+in+SolrConfig
![Page 18: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/18.jpg)
Autocommit or Not?
<autoCommit> <maxTime>15000</maxTime> <maxDocs>1000</maxDocs> <openSearcher>false</openSearcher></autoCommit>
<autoSoftCommit> <maxTime>1000</maxTime> </autoSoftCommit>
Automatic data flush (hard commit)
Automatic index view refresh
![Page 19: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/19.jpg)
Caches
q=lucene+revolution
fq=city:Dublin
Solr Cache
Refreshed with IndexSearcher
Configurable
Different purposes
Different implementations
![Page 20: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/20.jpg)
Monitoring Importance
![Page 21: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/21.jpg)
What to Pay Attention to?
![Page 22: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/22.jpg)
Cluster State
Health
Shards and replica status
Shard placement
Failing nodes
![Page 23: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/23.jpg)
Indexing Related Metrics
Index throughput
Document distribution
I/O subsystem metrics
Merging
![Page 24: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/24.jpg)
Search - related Metrics
Count
Latency
Distribution among nodes
Anomalies and spikes
![Page 25: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/25.jpg)
Monitoring Memory and GC
Heap details
Pool size
Pool utilization
Garbage collection count
Garbage collection time
![Page 26: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/26.jpg)
Monitoring OS Related Metrics
CPU details
Load
I/O activity
Network usage
![Page 27: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/27.jpg)
Solr Administration Panel
![Page 28: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/28.jpg)
Solr & JMX<jmx />
java -Dcom.sun.management.jmxremote –jar start.jar
![Page 29: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/29.jpg)
Solr & JMX
![Page 30: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/30.jpg)
SPMIndex statistics
Request # and latency
Caches and warmup
CPU
JVM Memory and OS Memory
Garbage collector
OS related statistics
![Page 31: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/31.jpg)
SPM Dashboard
![Page 32: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/32.jpg)
Other Monitoring Tools
Ganglia http://ganglia.sourceforge.net/
New Relic http://www.newrelic.com/
Opsview http://www.opsview.com
![Page 33: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/33.jpg)
Too much is too much
![Page 34: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/34.jpg)
Too hot
![Page 35: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/35.jpg)
Caches
![Page 36: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/36.jpg)
We Are Hiring !
Dig Search ?Dig Analytics ?Dig Big Data ?Dig Performance ?Dig working with and in open – source ?We’re hiring world – wide !
http://sematext.com/about/jobs.html
![Page 37: Administering and Monitoring SolrCloud Clusters](https://reader035.fdocuments.us/reader035/viewer/2022062616/54b67ab44a795962088b45f9/html5/thumbnails/37.jpg)
Rafał Kuć @kucrafal [email protected]
Sematext @sematext http://sematext.com http://blog.sematext.com
SPM discount code: LR2013SPM20
Thank You !
@ Sematext booth ;)