Customizing and Extending Microsoft Office SharePoint Search

Tom RizzoDirector, SharePointthomriz@microsoft.com

Session Objectives

Session Objectives: Modify the SharePoint Server search UIConsume SharePoint Search within external applicationsExtend the reach of SharePoint searchBuild a custom security trimmerLearn more about Microsoft Search Server and Federation

Agenda

Customizing the built in user experienceExtending the OOB experience

MatchingKeywordsandBest Bets

HighConfidence

ActionLinks

SearchTabsSearch

BoxPaginationStatistics

CoreResults

Search Center

Sources, Scopes and Tabs

All Sites and People tabs provided OOBAdd your own Search Tabs

Tabs can be added for additional scopesSpecific site addressSpecific propertySpecific content sourceInclude/Exclude

Consists of a layout page and a results page

Search Web Parts9 OOB web parts

Search BoxCore ResultsHigh ConfidenceStatisticsPaginationAction LinksMatching Keywords and Best BetsSearch Summary (Did you mean?)Advanced Search2 New ones for MSS – Federated Results and Top Federated Results

Web part properties such asFormattingTurning stemming on/offNumber of results returned XSL

Thesuarus and SynonymsCustomize Thesaurus and Noise Words

Located in drive:\Program Files\Microsoft Office Servers\12.0\Data\Configtsxxx.xml and noisexxx.txt where xxx is the language codeThesaurus supports Replacement or Expansion setsSubstitution weights and stemming are supported

Thesaurus Example

<replacement><pat>WinNT</pat>_{Windows Server}

</replacement><expansion>

<sub weight="1.0">Office SharePoint Server</sub><sub weight="0.6">SharePoint</sub><sub weight="0.8">MOSS</sub>

</expansion>

Tuning ResultsBest bets

Setup best bets at the site collection level

Simple things you can do for relevanceAuthoritative pagesDemoted sitesExclude specific items

Managed PropertiesAdd your ownAdd them to Advanced SearchChange weighting of properties (via the OM, requires a bit of code)

Customize UI With XSLTCreate a custom look and feel or results layoutModified through web part propertiesLeverage the power of XSLT

FormattingLogic Math

Walk through of the XSLTParameter Section

Parameters passed by results, web part properties

No Keyword SectionYou didn’t enter a keyword

No Results SectionNo results were returnedCustomize with your own message to help users

Main Body Result SectionWhere it all happens

Results Template SectionMatches results in XMLSelected column matching – Title, URL, etc.Hit Highlighting, Auto definition

CUSTOMIZING THE SEARCH UI

demo

Recap of Options

Add/Remove/Change TabsAdd/Remove OOB Web PartsAdd custom web partsTune resultsChange Web Part PropertiesChange XSL

Agenda

Customizing the built in user experienceExtending the OOB experience

Extending Search

Customizing the Query and ResultsQuery Object Model

Customizing the IndexIndex custom data - Protocol Handlers, IFilters, BDCCustom Query Time Security Trimmer

Query SyntaxURL Syntax

Use CaseLaunching a URL from a custom applicationWindows Desktop SearchSave searches

Keywordsresults.aspx?k=fish

Scopesresults.aspx?k=fish&s=BDC

Sort/Viewresults.aspx?v=date results.aspx?v=relevance

Pageresults.aspx?start=21

Property matchingresults.aspx?k=“author:Tom –site:http://tom.com”

Query Object Model

Use query OMTo build custom search UI web partsTo have direct access to query and results propertiesTo invoke custom queries

2 types of query syntaxesKeywordSQL

Query SyntaxKeyword Syntax

Use CaseBusiness user

Simple and easy to useRequired and excluded term support

Bike –fitness SharePoint search -author:”Tom Rizzo”

Filter by any string or integer propertyAuthor:Tom RizzoEmployees:30

Implicit “AND” behavior

Consistent property:value syntax across Office, Windows and Live search

gallery hinges –brass site:http//supportdesk scope:Products

Query SyntaxSQL Syntax

Use CaseAdvanced search solutions

Complex queriesConsistent SQL across enterprise and desktop

SELECT URL, Title, DescriptionFROM portal..SCOPE()WHERE FREETEXT(‘gallery hinges’)AND SITE = “http://supportdesk”AND SCOPE = “Products”AND NOT CONTAINS(‘brass’)

Query SyntaxSQL Syntax

Full strength, complete coverageFREETEXT()Arbitrary groupings for AND, OR, NOTCONTAINS()LIKEORDER BY ASC | DESCRemoved in MOSS 2007

UNION ALL MATCHESSELECT *COALESCE TABLE  

Query Object ModelFeatures

Managed code APISingle request – multiple results

Result Types• Relevant results• High confidence

results• Special terms• Definitions

Optional parameters• # of Sentences in

Summary• Implicit - AND/OR• Number of results• Ignore noise words• Enable stemming• Language

Query OMInput Output

SQLQuery

QueryEngine

Result Table CollectionResultTable:IDataReader

Relevantresults

Highconfidence

Specialterms

Definitions

Site UI

CustomClientExecute()

KeywordQuery

OptionalParameters

Local

Query Object Path

However…

Adding Custom Web PartsOOB Web Parts use sealed hidden objectYou’ll need your own query object to share with other Web PartsYou *can* use URL request parameters we pass between pages

Query Web Service

Use CaseOffice Research PaneWindows Desktop SearchRemote application

ASP.Net 2.0, Win Forms, Java apps

Similar features to query object modelResults in XML or data table formatshttp://o12server/_vti_bin/search.asmx

Query Web Service

MethodsQuery – returns results as XML format

Research and Reference Service-compatible XMLUsed in the Office Research pane

QueryEx – Returns .NET DataSetDifferent result types, features

GetSearchMetaData – Retrieve available properties and scopesRegistrationStatus

QUERY WEB SERVICE

demo

Finding People

View byrelevance or

social distanceRefine results using various

properties

Finding People

People Search Implements the core resultsCustomizes the search experience

Refine by Job Title/ DepartmentSort by social distanceDisplay results differently

Indexing Custom Data Protocol Handler

MOSS ships with support forWeb Content, NTFS File Shares, Exchange Public Folders, Lotus Notes DBs, SharePoint Content, SharePoint Profiles, Business Data Catalog

Develop a Protocol Handler to gather data from custom repositories

e.g. Vorsite for DocumentumConnects to a content source and enumerates the documentsInterfaces are the same as in SPS 2003New registry path:

Register PH under HKLM\Software\Microsoft\Office Server\12.0\Search\Setup\ProtocolHandlers

Indexing Custom Data IFilter

Install OneNote IFilter (requires OneNote on the server )Install PDF IFilter (64 bit now available)Develop IFilter for custom file formats

Reads a document and extracts the plain text content and properties for the indexer to indexSupport reading from streams

Indexing Custom DataBusiness Data Search

Integration without writing codeInformation in LOB systems is often hard to accessMOSS 2007 can bring that data to your users

Data is accessed through the Business Data CatalogExposed through Search, Meta Data, BDC web parts

Once a web service or a relational data source is registered with the BDC, search can easily index the data

No need to write codeHighly customizable resultsIntegrated with scopes and search center

Security

Built-in Query-time security trimming (Same as SPS 2003)

File shares, WSS/SPS 2003, Exchange, Lotus Notes(via mapping)

New! Support for result time custom security trimming

ISecurityTrimmer interface (Microsoft.Office.Server.Search.Query namespace)Implement Initialize() and CheckAccess()Add the custom security trimmer assembly to the GACRegister the security trimmer using stsadm command

New Federation Capability

Federation Overview

Enables the display of results from other search engines or applications to be displayed alongside local resultsSearch Server will

Send a query to other search engines or applicationsFormat and render the resultsWork out of the box or with minimal effort with many existing search engines and applications

Search Server will notAggregate multiple result sets into a single result setRelevance rank results from other search engines or applications

Federated Locations

Each search engine or application must be configured as a Federated Search LocationEach Federated Search Location

Defines how and when queries will be federated to the locationControls the formatting of resultsDefines authentication settingsCan be connected to one or more Federation Web Parts to render resultsIs easily imported and exported as a Federated Location Definition file (.FLD)

Supported Location Types

OpenSearch 1.0/1.1Query is sent in a parameterized URL

http://search.live.com/results.aspx?q={searchTerms}{searchTerms} is replaced with query termsResults must be returned as XML

Local Search IndexQuery is sent through the search object model

{searchTerms} scope:customers{searchTerms} is replaced with query termsResults are returned as XML

A Note On OpenSearch

Standard created by Amazon A9 for search syndication and aggregationAn exported Federated Location Definition file is >= an OpenSearch Description DocumentSearch Server extends the OpenSearch schema to include properties such as;

TriggersXSLLocation TypeMore Results Link…

See www.opensearch.org for more info

Location Triggers

Control when to send query terms to a location based on the terms submitted

Always - Will always send all the query terms to the locationPrefix - If the prefix matches will only send the query terms after the prefix to the locationPattern - Will send some or all of the query terms based on the regular expression match. A pattern trigger can be used to build complex triggers including “OR” statements

Federation Web Parts

Federated Results Web PartConnected to a single Federated LocationDisplays the top X results returned from the location

Top Federated Results Web PartConnected to multiple Federated locationsDisplays the top X results from the first location to return a result in the order of the configured locations

Neither Web Part will render if the location is triggered but it returns zero results

Rendering Results

Built in XSL for OpenSearch Locations that return results as RSS or ATOMLocal Search Index results

Built in XSL can be customized or replacedConfigured in the Location Definition and can be overridden on individual Web PartsSeparate XSL definition for both of the Federated Results Web Parts“More Results” Link Template available for click through to HTML results

Authentication

Three modes of authenticating to a locationAnonymous – No credentials requiredCorporate – Shared credentials for every userUser – Per user credentials

Per user credentials requires custom workCode to capture credentials in the UICode to store credentials if requiredCode to retrieve stored credentials if requiredSample code will be available on MSDN by RTMException is Kerberos which does work OOB

Per user credentials are sent to the location in the correct format automatically

An Example: www.live.com

How to connect Search Server to Live Search Query Template

http://search.live.com/results.aspx?q={searchTerms}&format=rss

“More Results” Link Templatehttp://search.live.com/results.aspx?q={searchTerms}

Works without any extra effort because live.com canProcess a query in a URLReturn results formatted as XML (RSS)Be accessed anonymously

There are many other search engines and applications that provide the same connectivity

Federated Search Connectors

Develop or purchase if the location does not support one or both of the following

Process a query in a URL parameterReturn results as XML

Federated Search Connectors do one or both of the following

Accept a query as a URL parameter and convert it to a format compatible with the locationConvert and render results in XML

Developed as a web page (aspx)Packaged and deployed as a Solution Deployment Package

An Example: SQL Server

Connect Search Server to the full text search engine in SQL Server

Cannot send a query to SQL in a URL

Build an ASPX page thatProvides a URL for the Query TemplateExtracts the {searchTerms}Contains code to run the query against SQLFormats and renders the results as XML (RSS)

If required include code to leverage either corporate or per user authenticationPackaged up and installed as a Solution Deployment Package

Connector Gallery

Federated Location Definition GalleryLinked from within the admin UISelection of FLD’s to download and import from Microsoft and Community submissionsQuick, easy and free

Federated Search Connector GalleryPartner built Federated Search ConnectorsPrimarily for LOB applications and servicesAvailable for purchase

Summary

The user interface is easily customized to achieve a different look and feelIt’s easy to extend search to develop custom applications, local or remoteYou can plug in any data source and search it securely

Resources1. Creating custom XSL -

http://msdn2.microsoft.com/en-us/library/bb428855.aspx

2. Adding managed properties to Advanced Search - http://www.codeproject.com/spoint/MOSS_Advanced_Search.asp

3. Querying Enterprise Search and Custom Security Trimming - http://msdn2.microsoft.com/en-us/library/aa981100.aspx

4. MOSS Query Toolhttp://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=89b3cda7-aad9-4919-8faf-34ef9b28c57b

5. BDC Tool – In the August SDK!

6. Ontolica Wildcard Search - http://www.ontolica.com

7. PDF IFilter32 bit from Adobe - http://www.adobe.com/support/downloads/detail.jsp?ftpID=261164 bit from Foxit - http://www.foxitsoftware.com/pdf/ifilter/

8. OneNote IFilterhttp://technet2.microsoft.com/Office/en-us/library/2e715e42-c09b-4b4f-a082-b19e1cad96031033.mspx?mfr=true

9. Add MOSS to Windows Desktop Search or Vista Instant Search Providerhttp://blogs.msdn.com/joelo/archive/2006/12/08/tips-to-enable-sharepoint-intranet-search-using-windows-desktop-search-3-0-rtw.aspx

© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after

the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.