Managing & Troubleshooting SharePoint Search
-
Upload
thomas-vochten -
Category
Technology
-
view
125 -
download
5
description
Transcript of Managing & Troubleshooting SharePoint Search
![Page 1: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/1.jpg)
Managing & Troubleshooting
SharePoint Search
#SPSJEThomas Vochten@thomasvochten
IT Pro - Level 200
![Page 2: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/2.jpg)
About Me
Thomas Vochten SharePoint MVP. Platform architect. Speaker. Trainer. Involuntary DBA. Architect at Xylos. V-TSP at Microsoft.
@thomasvochtenhttp://[email protected]
![Page 3: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/3.jpg)
Objectives
• Understanding the search architecture• Creating a search service application• Managing the search topology• Troubleshooting the search topology
IT Pro - Level 200
![Page 4: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/4.jpg)
Search ArchitectureBack to basics
![Page 5: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/5.jpg)
Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
![Page 6: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/6.jpg)
Services & Processes
• SharePoint Search Host Controller Service
hostcontrollerservice.exe
Controls & checks other components
• SharePoint Server Search Service
mssearch.exe & mssdmn.exe
Crawl component
![Page 7: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/7.jpg)
Services & Processes
• All other components
noderunner.exe
• Search Query and Site Settings ServiceWCF Service in IIS
searchservice.svc
![Page 8: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/8.jpg)
Crawl Component
Crawl
ContentProcessing Index
Crawl
HTTP
File Shares
SharePoint
User Profiles
Exchange
Lotus Notes
Documentum
Custom (BCS)
...Admin
mssearch.exe
SearchAdmin
FAST Search Index
![Page 9: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/9.jpg)
Crawl Component
ULS Category: CrawlerProcesses: mssearch.exe & mssdmn.exe
• Manages the crawl queue• Protocol handlers• State lives in the crawl database• Uses a local cache • Full/Incremental/Continuous
![Page 10: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/10.jpg)
Crawl Database
![Page 11: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/11.jpg)
Content Processing Component
Crawl
ContentProcessing Index
Crawl AdminAnalyticsProcessing
Link
Content EnrichmentWeb Service
SearchAdmin
FAST Search Index
![Page 12: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/12.jpg)
Content Processing Component
ULS Category: Content ProcessingProcess: noderunner.exe
• Document parsing • Metadata extraction• Stateless• Extensible
![Page 13: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/13.jpg)
Index Component
Replica
ReplicaReplica
Replica
Content Processing
Replica
Partition #1
Replica
Replica
Replica
Partition #2
Replica
Replica
Replica
Partition #3
Replica
Each replica is an index component
![Page 14: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/14.jpg)
Index Component
ULS Category: IndexProcess: noderunner.exe
• Inserts content into binary index• Also stores security info & metadata• Indexes live on disk only• Uses partitions & replica’s
![Page 15: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/15.jpg)
Query Component
UXQueryProcessing WFEIndex
SearchAdmin
Admin
FAST Search Index
API
![Page 16: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/16.jpg)
Query Processing Component
ULS Category: QueryProcess: noderunner.exe
• Handles user queries• A lot of business logic• Stateless• Query Analyzer• Security Trimming
![Page 17: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/17.jpg)
Query Path
• Web Server
• Search Query & Site Settings Service (HTTP endpoint)
• Query Processing Component (net.tcp endpoint)
• Index Component (net.tcp endpoint)
![Page 18: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/18.jpg)
Analytics Component
Crawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Link
Analytics Reporting
Event Stream
![Page 19: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/19.jpg)
Analytics Component
ULS Category: AnalyticsProcess: noderunner.exe
• Intensive computations• Links, social interactions, views, recommendations, …• Gets better over time• Also handles health statistics in central admin
![Page 20: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/20.jpg)
Admin Component
SearchAdmin
Admin
![Page 21: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/21.jpg)
Admin Component
ULS Category: AdminProcess: Administration
• Stores configuration objects• Web service for administering configuration objects• Management via PowerShell only!• Only one active but can be made redundant• Application Server Admin Service timer job
![Page 22: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/22.jpg)
Components Placement
Search is probably the most resource intensive SharePoint service
Good Practice:
• CPC & APC together• Admin & Crawl together• Index & Query together
![Page 23: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/23.jpg)
Start Here
Minimal highly available search architecture
![Page 24: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/24.jpg)
Scale Out
![Page 25: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/25.jpg)
Resource Requirements
Component CPU Network Disk Memory
Search administration * * *Crawler ** *** *** **Content processing (CPC) *** ** ***Analytics processing (APC) ** *** * **Index *** ** *** ***Query processing (QPC) * ** *
![Page 26: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/26.jpg)
Databases Recap
• Administration DB• Crawl DB• Links DB• Analytics Reporting DB
![Page 27: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/27.jpg)
SetupBuilding a Search Service Application
![Page 28: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/28.jpg)
What you need
• A managed account• An application pool
Consider other service applications:• Managed Metadata• User Profiles• State Service• Usage & Health
![Page 29: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/29.jpg)
PowerShell to the rescue
• Central Admin will give you nothing by GUIDs• New-SPEnterpriseSearchServiceApplication
![Page 30: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/30.jpg)
Creation Process
• Check system requirements• Create Search Service Application• Create Search Topology• Create Components• Activate Search Topology
![Page 31: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/31.jpg)
Default Topology
• 1 admin component• 1 crawl component• 1 content processing component• 1 index component• 1 query component• 1 analytics component
All components will be installed on a single server
![Page 32: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/32.jpg)
Demo
Creating a search service applicationCreating a basic search topology
![Page 33: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/33.jpg)
Manage the Search ComponentsGet to know your search topology
![Page 34: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/34.jpg)
General
• Topology is stored in the Search Service Application• There is no user interface as in SharePoint 2010• You can deploy all components at once across the
network
![Page 35: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/35.jpg)
Typical Operations
• Add search components• Move search components• Remove search components• Add additional search servers
New-SPEnterpriseSearch...ComponentRemove-SPEnterpriseSearchComponent
![Page 36: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/36.jpg)
Modify the existing topology
• Make sure crawls are in idle state• Clone Search Topology• Add Components• Activate Search Topology (can take a while!)• Wait for a green light
![Page 37: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/37.jpg)
About Cloning
Clone the active search topology• You cannot modify an active topology
Topology Activation Process• Check for valid topology first
Topology Activation in progress• Topology A is marked as Activating• Topology B is marked as Deactivating • The deactivating topology is the one serving search.• At this point your active current topology is still Topology B• Topology A is marked Active
![Page 38: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/38.jpg)
(Re)moving Components
• Clone Search Topology• Add new components• Activate Search Topology• Wait for the green light• Clone Search Topology again• Remove components• Activate Search Topology• Wait for the green light
![Page 39: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/39.jpg)
Index Operations
• Adding a replica to existing partition 0
New-SPEnterpriseSearchIndexComponent … -IndexPartition 0
• Adding a new partition (can’t remove!)
New-SPEnterpriseSearchIndexComponent … - IndexPartition 1
![Page 40: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/40.jpg)
Demo Scenario
Server A has all search components.We want to make all components redundant
1. Clone the topology on server A2. Start search service instance on server B3. Modify the cloned topology4. Activate the cloned topology5. Verify the new topology
![Page 41: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/41.jpg)
Demo
Make all search components redundantMove components to dedicated servers
![Page 42: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/42.jpg)
Troubleshooting 101Topology & Component Failures
![Page 43: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/43.jpg)
Search Validation
• Windows Services
Host Controller (on every server)Search Service (on crawl servers)
• Search Topology page
Verify everything is green
![Page 44: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/44.jpg)
Search Validation
• Crawl Test
Add documentsFull crawl
• Query Test
Search CenterTry “*”
![Page 45: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/45.jpg)
Search Validation
• Get-SPEnterpriseSearchStatus• Custom health scripts
Get-SPSearchTopologyState.ps1
http://thvo.me/spsearchstatus
![Page 46: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/46.jpg)
Search Health
• Green (ok) – sit back and relax• Yellow (degraded) – sit back and wait• Red (failure) – manual intervention needed
![Page 47: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/47.jpg)
Approach when it breaks
• Asking questions• Looking at the crawl/query logs• Looking at the ULS logs• Health Status in Central Administration• Validate processes & services• Diagnostics scripts
![Page 48: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/48.jpg)
Useful tools
• ULSViewer http://thvo.me/ulsviewernew
• SharePoint 2013 Search Query Toolhttp://thvo.me/querytool
• Fiddlerhttp://thvo.me/getfiddler
![Page 49: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/49.jpg)
Some Tips
• Don’t be overwhelmed
• Always make components redundant, especially Admin
• Admin component redundancy should be priority
• Pauze & Stop search before patching SharePoint!http://thvo.me/slowpatch
![Page 50: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/50.jpg)
Some Tips
• Boosting logging levels to Verbose of VerboseEx
• You can follow crawled documents in the ULS
• You can follow issued queries in the ULS (single correlationid)
• Get to know logging categories
![Page 51: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/51.jpg)
Importance of the Admin ComponentDown?
• No search administration possible• No search monitoring possible• No backup or restore• No topology changes possible• No new crawls will start• Queries will still work
![Page 52: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/52.jpg)
Common Problems
• Disk space on the index servers (master merge)• Disk space on the analytics servers (temp space)• Disk space on the crawl servers (temp space)• Proxy servers preventing crawl• Domain Administrator service accounts• Missing Antivirus Exclusions• Not indexing the default zone• Stuck or long running crawls• Database problems (blocking, perf, space)
![Page 53: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/53.jpg)
Common Problems
• Networking problems• Endpoints not being available• Problems with custom code (e.g. content enrichment
svc)• WCF endpoints
![Page 54: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/54.jpg)
Search Data Locations
C:\Program Files\Microsoft Office Servers\15.0\Data
Analytics Temporary StorageCrawl Temporary StorageIndex files
![Page 55: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/55.jpg)
Demo
Get-SPSearchTopologyState.ps1Something went wrong… when querying
![Page 56: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/56.jpg)
Summary
• Get to know the individual search components• Start with a redundant implementation from the start• Learn the basic PowerShell commands• Keep your favorite tools around when troubleshooting• First things first: don’t neglect the logfiles• Rebuilding your service application should be your last
resort, not your first
![Page 57: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/57.jpg)
Questions?
DisclaimerMost of the graphics in this presentation were created by Microsoft
![Page 58: Managing & Troubleshooting SharePoint Search](https://reader036.fdocuments.us/reader036/viewer/2022081716/547e8b01b379594e2b8b54aa/html5/thumbnails/58.jpg)
Thank You!
Raffle Prizes at 16:00
1st #SharePint in the Hotel Bar sponsored by Cloudshare 16:30