Post on 31-Dec-2015
Search Architecture in SharePoint 2013Rune Zakariassen Thomas MolbachProgram Manager Architect, Search COE
SPC202
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
SharePointSP AppsDevicesNon-SP UX
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public API
Unit of scale/role boundary
Feeding ChainIndex Core
Query Chain
Analytics Service
What is SharePoint 2013 search?
SharePoint 2013 Search
SharePoint 2010 Search
FAST Search
For SharePoi
nt
FAST Technolog
y
Office Search Scenarios
Search 2013 Platform
Exchange 2013 platform SharePoint 2013 platform
Search 2013 platform
`
Exchange
SharePoint
Custom solution
CSOM/REST API
Outlook Web App (OWA) search eDiscovery
Intranet search
Custom app (Internet/intranet)
Marketplace app
All
resu
lts
Peop
le
Sit
e s
earc
h
Vid
eo
Cust
om
searc
h
vert
ical
Internetsearch
Content sources
Public API
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public API
Unit of scale/role boundary
Feeding Chain
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
Crawl Component• OOB connectors
• Extensible through BCS
• Local disk cache
• Crawled items tracked in Crawl database
• Configurations stored in Admin database
• Crawl modes• Full Crawl• Incremental Crawl• Continuous Crawl
Crawl
ContentProcessing Index
Crawl
FAST Search Index
HTTP
File Shares
SharePoint
User Profiles
Exchange
Lotus Notes
Documentum
Custom (BCS)
...SearchAdmin
Admin
mssearch.exe
Content Processing Component• Stateless node
• Analyzes content for indexing
• Processing flow
• Dictionaries
• Schema mapping
• Stores links and anchors in Link database (analytics)
• Extensible through web service call-outs
• Configurations stored in admin database
Crawl
ContentProcessing Index
Crawl
FAST Search Index
SearchAdmin
Admin
AnalyticsProcessing
Link
ContentEnrichmentWeb Service
Content Processing Component
Content Processing Flow (conceptual)
Detect language
Document
summary
Map to managed propertie
s
Parse documents and extract metadata
Custom Entity
Extraction
Phonetic name
variations
Word breaking
Web Service Callout
Ifilter sandbo
x
Security Descriptor
s
Delete item
Update security
Crawler
Index
Delete Links
Security Descriptor
s
Insert new or updated item
Each CrawlerFlow processes one item at time.
Demo 1
Pipeline – smart title/author extractor
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public API
Unit of scale/role boundary
Index Core
• Partitions and replicas• No rows and columns
• Index system• n Partitions• 1 primary replica• ≥ 0 secondary replica
• All nodes performs indexing• Journal shipping from primary to replicas
Index Partition details
Index
Index
Index
Index
Index
Content Processing
Index
Index
Index
Index
Index
Index
Index
Append sectionCheckpoint section
Merging Index PartsMaster partLevel
M
Level 0
Level 1
Level 2
New Master part
+
Disk
Memory
Checkpoint
Master merge
= 1 generation
Demo 2
Index Schema and Managed Properties
• Crawled property managed property• Site collection schema mapping allows• Custom/managed property mapping• Aliasing of managed properties• Enhanced functionality with managed property:• Data type• Queryable• Searchable• Retrievable• Refinable• Sortable• Safe for anonymous access
Understanding Index Schemas
With Schema Mapping
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search IndexDocId Partition
Content Title Author Modified
RefinableText1
111 0 Xyz... My lab ... Rune ... Some value
222 0 Abc... My Word ... Rune ... ...
333 1 .... ....
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
SharePointSP AppsDevicesNon-SP UX
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public API
Unit of scale/role boundary
Query Chain
SharePointSP AppsDevicesNon-SP UX
Web Front-End
SearchAdmin
UXQueryProcessing WFE
API
Index
FAST Search Index
Public API
Unit of scale/role boundary
• REST/OData API• Client-Side object model
(CSOM)• Server-Side object
model (SSOM)• Search center• Portals and publishing• ContentWebPart• RefinerWebPart• Result templates
Search
SharePoint 2013
SharePoint 2013 Remoting API
CSOM
http://<spsite>/_api
JavaScriptLibrary
SilverlightLibrary
.NET CLRLibraryREST
ODataJSON
Custom Client Code
Server
Client
SharePointSP AppsDevicesNon-SP UX
Query Processing Component
SearchAdmin
UXQueryProcessing WFE
API
Index
FAST Search Index
Public API
Unit of scale/role boundary
• Stateless node• Processing flows• Query analyzer• Linguistics• Dictionaries• Result sources• Schema mapping• Query rules• Query federation • Configurations
stored in admin database
Admin
Query Execution Flow (conceptual)
WFE
Query Processing Component
Index
Index
IndexP0
Index
Index
IndexP1
Index
Index
IndexP2
Index
Index
IndexP3
Search Provider Flows
Best Bets
Exchange Search
People Search
Personal Favorites
Remote SharePoint
Local SharePoint
Keyw
ord
Query
FlowInput
QueryRuleMatching
QueryTransformer
QueryRouter
FlowOutput
ResultTableAssembler
LayoutSelector
Query Rules
Captures Search IntentReplaces search keywords in 2010Create query rules at Farm, Site Collection, and Site levels
Composed of 3 top level elementsQuery Conditions
What queries should be handled?
Query Actions
What happens when a rule matches?
PublishingOptions
Is the rule active and for how long?
Demo 3
The query sent to the index
Search
INTERNAL ITEM CATALOGSITE COLLECTION
Electronics
DESKTOP WEB CHANNEL MOBILE CHANNEL
REST API
NATIVE APP
CAMPAIGNPROMOTION
Editorial Control with Query Rules
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
SharePointSP AppsDevicesNon-SP UX
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public API
Unit of scale/role boundary
Analytics Service
Analytics Processing Component
Crawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Link
Analytics Reporting
• Map-reduce• Learns by usage• Search Analytics• Usage Analytics• Enriches index by
updating index items
• Usage reports in Analytics database
Event Stream
Up sell / cross sellConsumers
Light-up with Recommendations
SharePoint 2013 Analytics Processing
Purchased Products
Viewed Products Liked Products
Trending Products
Recommendations
Built-in recommendationsEvent stream analysisItem-to-item recommendationsStored in “recommendedfor:” managed propertyUp to 12 configurable, weighted, events
Demo 4
Analytics and Recommendations
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
Crawl
Admin
Link
Analytics Reporting
FAST Search Index
ContentEnrichmentWeb Service
CustomConnectors
Public API
Unit of scale/role boundary
Extensibility Points
Search HOLs and events @ SPCHOL031 – Introduction to Search in SharePoint 2013HOL034 – Exploring Search Query Rules in SharePoint 2013HOL032 – Extending the Search experience in SharePoint 2013HOL033 – People Search in SharePoint 2013HOL035 – SharePoint Server 2013 Search Connectors and Using BCS
Meet a Search SME
Ask questions, meet the community and share knowledge!
Mon-Thu @ Exhibit Hall
Hands on Labs
Daily 10:30am-6:30pm @ HOL Lab Lounge
Ask the Experts
Discuss search!
Wed 6:15PM @ Ask the Experts Lounge
Related Search Sessions @ SPCTue 10:30am - SPC044 - Crawl and Index all Enterprise Content with SharePoint 2013 Search - Speaker: Vaidy Raghavan
Tue 5:00pm - SPC172 - Overview of Capacity Planning, Sizing and High Availability for Search in SharePoint 2013 - Speakers: Barry Waldbaum, Olaf Birkeland
Wed 9:00am - SPC143 - Making Great Search Based Applications with Query Rules in SharePoint 2013 - Speaker: Pedro DeRose
Tue 5:00pm - SPC203 - Search Content Enrichment and Extensibility in SharePoint 2013Speakers: Runar Olsen, Kathrine Hammervold
Wed 3:15pm - SPC049 - Custom Security Trimming for Search in SharePoint 2013Speakers: Morgan Larsson, Sveinar Rasmussen Wed 5:00pm - SPC145 - Optimize Search Relevance in SharePoint 2013Speakers: Jan Inge Bergseth, Victor Poznanski
Questions?
Evaluate this session now on MySPC using your laptop or mobile device: http://myspc.sharepointconference.com
MySPC
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.