Building Search Driven Applications in SharePoint 2010

Nik Patel

Lead Solutions Architect, Slalom Consulting, Chicago Practice

Session Goals

Audience – SharePoint Architects, SharePoint Developers, and Power Users

Main Goal - Understanding Search from Application Development Perspective

What this Session will cover

Out of Box Search Configurations

On-Premises Enterprise Search API

What this Session won’t cover

FAST Search API

Search in SharePoint Online (First half partially applies to Office 365)

2

About Me

Nik Patel • Lead Solutions Architect, Slalom Consulting

• Working with SharePoint 2010 since Summer 2009 TAP

• Master in Computer Science from IIT, Chicago

Contact Info • Email – patenik2@yahoo.com

• Blog – Nik Patel’s SharePoint World -

http://nikspatel.wordpress.com/

• Twitter - @nikxpatel

• LinkedIn - linkedin.com/in/nikspatel

• Slideshare - slideshare.net/patenik2

3

Session Agenda

Why Search is Important in SharePoint Information Architecture?

Why Search is Important as SharePoint Data Access Method?

Overview of SharePoint Enterprise Search Architecture

Out of Box Customization Options for SharePoint 2010 Search Applications

Demo – Custom Search Solutions using OOB Web Parts

Programming Options for SharePoint 2010 Search Applications

Demo – Custom Search Solutions using Keyword Search API

Questions and Answers

4

Why Search and Metadata Driven Information Architecture necessary? Information is everywhere, Needs foundational architecture to find it efficiently

Information Islands

Modern intranet has Information and Applications Islands and Needs to find information efficiently

Find and Target Relevant Information

User can receive information if they are tagged property. Search can leverage metadata to target relevant information

Aggregate information from Distributed SharePoint Information Architecture

Rollup information across Multiple Sites, Site Collections, Web Applications, and Farms

Next time when you design systems, think about Search + Metadata Driven Information Architecture

5

Why Search is Important as SharePoint Data Access Method? Overview and Limitation of Traditional SharePoint 2010 Data Access APIs

6

Farm Site List Data External Lists

Data Platform

Client OM REST APIs SOAP APIs

• Server Side APIs

• SPList, SPQuery, and SPDataQuery

• LINQ to SharePoint

• Client Side and Remote APIs

• CSOM and REST APIs

• SOAP APIs and WCF Services

• Targeted at List or Library

• Scoped at the Site or Site Collection

• 5000 List Items threshold

• CSOM has slow performance over the network

• Data Aggregation and Rollup would require complex custom programming

Why Search is Important as SharePoint Data Access Method? Enterprise Search delivers Data Rollups and Data Aggregations efficiently

7

• Use OOB Web parts, Search APIs, or Search ASMX Services to query data against Search Index

• Major Benefits

• Perfect for Data Rollups and Data Aggregations across Cross-farm, Cross-Web Application, or Cross-Site

Collection

• Can query across large data corpuses (because it query against index)

• Queries are fast even with large data corpuses (because it query against index)

• Most scalable data access solution out there, All other techniques are mostly SQL-based (because it query

against index)

• Major Limitations

• Content needs to be in the indexed (dependency), Avoid this option if real time data access are requirements

• Requires advanced search service administration and configuration (dependency)

• Improve search by tagging with metadata (to be effective)

Typical Search Application Scenarios and Business Cases

Typical Business Cases

De-centralized content, data, and documents

Large amount of data, document libraries, sites, or site collections

IA Governance – Metadata, Content Types, and Search

Typical Application Scenarios

Cross Site Collection Portal Navigation

My Team Sites Dashboard

Corporate Knowledge Center - Rolling up documents

Customer Documents Dashboard - Cross Web Application or Cross-Farm Documents Sharing for Extranets

8

Overview of SharePoint Enterprise Search Architecture

9

Enterprise Search Terminology for Developers – Major Puzzle Pieces Understanding out of box crawling and indexing pipeline would benefit

Content Sources

Crawling Schedules

Crawl Rules

Content Scopes

Crawled Properties

Managed Properties

Search Query Object Model

Enterprise Search Center

10

Search by default is Security Trimmed

Out of Box No-Code SharePoint Search Customization

Audience – Power Users & SharePoint Architects

11

Why would you Customize Search? Out of box Search configuration doesn’t cover all business cases

Typical complains with OOB Search Center

Too many results – They are not relevant

Missing information on search page

Improving how Users View results

OOB interface may not provide ideal UI

Tailoring search center may greatly increase productivity or usability

Tip - Try using Out of box Options before exploring Custom Coding Options

12

Out of Box Search Capabilities – Search Web Parts Search Web Parts can be used by Power Users to build Search Applications

Advanced Search Box Allows users to create detailed searches against Managed Properties

Federated Results Displays results from a federated search location

People Refinement Panel Presents facets that can be used to refine a people search

People Search Box Allows users to search for people using a keyword

People Search Core Results

Displays the primary result set from a people search

Refinement Panel Displays a summary of search results and enables

users to filter results.

Related Queries Displays the related searches for a particular user

query.

Search Action Links Displays links for RSS, alerts, and Windows Explorer

Search Best Bets Displays best-bets results

Search Box Allows users to enter keyword query searches

Search Core Results Displays the search results for a user's query

Search Paging Allows a user to page through search results

Search Statistics Presents statistics for search results like total

search results

Search Summary Displays the summary for a user's query.

Top Federated Results Displays top results from a federated location

Major components in configuring No-Code browser based Search Applications

Out of Box Search Capabilities – Core Results Web Part Displays search results based on user query or predefined queries

Use in conjunction with Refiner Panel, Search Box, Search Paging, and Search Statistics web parts

Major features

Displays search results

Specify a scope or content source to limit results

Customize what fields returned by Search

Results customizable using XSL

Can hard code query on behalf of the user

Use the KeywordQuery API syntax

Fixed Keyword Query

Append Text to Query

More Info - http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/04/05/a-quick-guide-to-coreresultswebpart-configuration-changes-in-sharepoint-2010.aspx

Out of Box Search Capabilities – Refiner Panel Web Part Refines search results in Core Results web part

Use in conjunction with Core Results, Search Box, Search Paging, and Search Statistics web parts

Major Features

Refine results based upon categories, metadata, managed metadata or tags

Customizable using XSL

Filter categories configured using XML

Accuracy Index – Refiners results based on first 50 results by default,

Beware of Use Default Configuration – Uncheck for custom filter category definitions

More Info: http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/03/15/get-to-know-the-refinement-web-part-in-sharepoint-2010-enterprise-search.aspx

Improve Search with Queries - Keyword Query Syntax Refines and filter query results by Keyword Query Syntax

Improved Keyword Syntax Operators in SharePoint 2010

Property Operators: :, =, <, >, <=, >=, <>

Boolean Operators: AND, NOT, and OR

Wildcard Operator: *, Only supports prefix matching e.g. account* returns data with account, accounts, accountant etc.

Proximity Operators: x NEAR y (up to 8 terms away)

Synonym Operators: WORDS(TV, Television), As opposed to “TV OR Television”, WORDS ranks them equivalently

Inclusion/Exclusion Operators: +/-

Parenthesis Support for Complex Grouping of queries

Improve Search with Queries - Keyword Query Syntax (Continued) Refines and filter query results by Keyword Query Syntax

Streamline with Managed Properties

Search by either built-in or custom managed properties

Managed properties must be configured in Search Center against crawled properties

Sample OOB Managed Properties

ContentSource:"Local SharePoint Sites“

Scope:”Products”

Site:”http://sp2010vm/products”

ContentType:”Policy Type”

isDocument:0 (0 or 1)

ContentClass:STS_ListItem_Tasks (e.g. STS_Site, STS_Web, STS_List, STS_ListItem, STS_List_{Type}, STS_ListItem_{Type})

FileType:“docx"

Demo

18

No-Code Customized Search Application

Centralized Policy Center

Programming Search - Custom SharePoint Search Applications

Audience – SharePoint Architects & SharePoint Developers

19

Why would you Extend Search or Build Custom Search Solutions? Building Custom Search Applications using Search APIs

Typical Scenarios

Out-of-the-box search works great, but users want a rich interface

Out-of-the-box search doesn’t meet all the business requirements and requires advanced custom logic

Accessing SharePoint Search Index remotely from the ASP.NET, Silverlight, or Mobile Applications

Typical Options

Extend OOB Search Results Web Part

Programming against Server Side Object Model

Keyword Query API

FullTextSQL Query API

Programming against Search Web Services (Not Covered Today)

SharePoint Search SOAP API - http://sp2010vm/_vti_bin/spsearch.asmx

Enterprise Search SOAP API: http://sp2010vm/_vti_bin/search.asmx

NOTE: Search is not supported in SharePoint 2010 CSOM API

More info - Enterprise Search Query Architecture - http://msdn.microsoft.com/en-us/library/ee558338.aspx

20

Extending the OOB Core Results Web Part Simplest way to extend out of box functionality

Why?

Dynamically set search query on behalf of the user

Specify Fixed query and Additional query

User interactions (e.g. toggling custom views based on user friendly search queries)

Reading query from non-traditional source (e.g. Query string, custom formula)

Dynamically update look and feel

Search by default sort results by relevance, Instead apply custom sorting

Apply different custom XSLTs for different layouts based on user queries

How?

Create new web part, extend CoreResultsWebPart

Set key properties on derived class

Location (Content Source) & Scope (Specify Scope)

FixedQuery & AppendedQuery

SelectColumns – All rendered columns must be in this XML string

Xsl & XslLink

Modify the Query Pipeline

Override GetXPathNavigator Method

QueryManager class allows us to intercept query pipeline before and after query is issued

More Info - How to: Use the QueryManager class to query SharePoint 2010 Enterprise Search -http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/08/15/how-to-use-the-querymanager-class-to-query-sharepoint-2010-enterprise-search.aspx

Server Side Query Object Model - Search Query APIs KeywordQuery API, FullTextSQLQuery API, and Fast Query Language (FQL)

KeywordQuery API

Preferred Approach for Search Queries in SP2010

Executes Keyword Query Syntax queries

Query executes against Search Index

Requires content must be crawled, crawled properties must be in the index, and managed properties must be mapped to crawled properties.

FAST doesn’t support SQL Syntax, use Keyword Query Syntax to seamlessly support code in both Enterprise Search and FAST

22

FullTextSQLQuery API

Executes Search SQL Syntax queries

Executes queries against Search Scopes

Doesn’t require managed properties

Puts load on SQL rather than Query servers

Use if you prefer the SQL syntax and execute full text query programmatically

Unlike Keyword Query API, SQL Syntax support CONTAINS parameter

Step by Step – Using Search Query APIs Initial steps are same for KeywordQuery API and FullTextSQLQuery API

Step 1 - Create SharePoint 2010 Visual Studio Solution based on SharePoint 2010 Project Template

Step 2 - Add SharePoint Project Item for Search Application UI & Logic

Custom Web Part or Visual Web Part

Custom Application Page or Site Page

Step 3 - Reference two Search DLLs from the ISAPI directory

~SharePointRoot\ISAPI\Microsoft.Office.Server.dll

~SharePointRoot\ISAPI\Microsoft.Office.Server.Search.dll

Typical SharePoint Root is - C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14

Step 4 - Import Search Namespaces

using Microsoft.Office.Server;

using Microsoft.Office.Server.Search;

using Microsoft.Office.Server.Search.Query; [KeywordQuery Class defined in this namespace]

using Microsoft.Office.Server.Search.Administration;

Step by Step – Using KeywordQuery API

• Step 1 - Instantiate the KeywordQuery object

• You have two options, pass current site collection (Classic MOSS 2007 way), or pass the Search Query and

Site Settings Service proxy (New in SP2010) and instantiate KeywordQuery object

1. SearchQueryAndSiteSettingsServiceProxy settingsProxy = SPFarm.Local.ServiceProxies.GetValue

<SearchQueryAndSiteSettingsServiceProxy>(); SearchServiceApplicationProxy searchProxy = settingsProxy.ApplicationProxies.GetValue <SearchServiceApplicationProxy>("Search Service Application"); KeywordQuery q = new KeywordQuery(searchProxy);

2. SearchServiceApplicationProxy proxy = (SearchServiceApplicationProxy)SearchServiceApplicationProxy.GetProxy(SPServiceContext.GetContext(SPContext.Current.Site));

KeywordQuery q= new KeywordQuery(proxy);

Step by Step – Using KeywordQuery API (Continued…)

• Step 2 - Set KeywordQuery Object Properties

• QueryText – This is keyword query text (e.g. LastName:”Patel”)

• ResultTypes – Search result types (default is Relevant Results)

• HiddenContraints – Filter by search scope

• RowLimit – By default, it returns only 50 records, Se this property to increase number

• RowsPerPage - Paging

• IgnoreAllNoiseQuery

• TrimDuplicates – Trims duplicate results

• EnableNicknames

• EnablePhonetic

• EnableStemming

• EnableSpellCheck

• EnableFQL

Step by Step – Using KeywordQuery API (Continued…)

• Step 3 – Use SelectProperties string collection to return custom managed properties

• Out of the box, KeywordQuery API will return only default search properties such as Title, Author etc.

• To return managed properties, use SelectProperties string collection

• Step 4 - Run KeywordQuery Object Execute() method and return ResultTableCollection which can be

loaded in ADO.NET DataTable Object

• Step 5 - Bind ADO.NET DataTable Object data into ASP.NET or SharePoint Controls for UI

More Info from Corey Roth’s blog

How to: Use the MOSS Enterprise Search KeywordQuery class - http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2008/02/19/how-to-use-the-moss-enterprise-search-keywordquery-class.aspx

How to: Use the SharePoint 2010 Enterprise Search KeywordQuery Class -http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/08/12/how-to-use-the-sharepoint-2010-enterprise-search-keywordquery-class.aspx

Step by Step – Using KeywordQuery API (Complete Example)

Demo

28

Custom Application using KeywordQuery API

Cross Site Collection Navigation

Step by Step – Using FullTextSQLQuery API

• Step 1 – Just like KeywordQuery API, Instantiate the FullTextSqlQuery object

• Just like KeywordQuery API, You have two options, pass current site collection (Classic MOSS 2007 way), or

pass the Search Query and Site Settings Service proxy (New in SP2010) and instantiate FullTextSqlQuery object

1. SearchQueryAndSiteSettingsServiceProxy settingsProxy = SPFarm.Local.ServiceProxies.GetValue

<SearchQueryAndSiteSettingsServiceProxy>(); SearchServiceApplicationProxy searchProxy = settingsProxy.ApplicationProxies.GetValue <SearchServiceApplicationProxy>("Search Service Application"); FullTextSqlQuery q = new FullTextSqlQuery(searchProxy);

2. SearchServiceApplicationProxy proxy = (SearchServiceApplicationProxy)SearchServiceApplicationProxy.GetProxy(SPServiceContext.GetContext(SPContext.Current.Site));

FullTextSqlQuery q= new FullTextSqlQuery(proxy);

Step by Step – Using FullTextSQLQuery API (Continued…)

• Step 2 - Set FullTextSQLQuery Object Properties

• QueryText – This is SQL Query text, You can specify scopes in the query

• ResultTypes – Search result types

• RowLimit – By detault, it returns only 50 records, Se this property to increase number

• RowsPerPage - Paging

• IgnoreAllNoiseQuery

• TrimDuplicates

• EnableNicknames

• EnablePhonetic

• EnableStemming

• Step 3 - Run FullTextSQLQuery Object Execute() method and return ResultTableCollection which can

be loaded in ADO.NET DataTable Object

• Step 4 - Bind ADO.NET DataTable Object data into ASP.NET or SharePoint Controls for UI

• Step 5 – Unlike KeywordQuery Object, you must dispose FullTextSQLQuery object

How to: Use the MOSS Enterprise Search FullTextSqlQuery Class - http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2008/03/06/how-to-use-the-moss-enterprise-search-fulltextsqlquery-class.aspx

Step by Step – Using FullTextSQLQuery API (Complete Example)

32

• Thanks for attending the session!! • Follow SharePoint Fest Chicago

• Twitter - @SharePointFest - https://twitter.com/sharepointfest

• Website - http://www.sharepointfest.com/Chicago/

© 2011 Slalom Corporation. All rights reserved. The information herein is for informational purposes only and represents the current view of Slalom Corporation as of the date of this presentation.

SLALOM MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.