Jeff Fried & Keenan Newton Senior Technical Product Managers Microsoft Corporation SESSION CODE:...
-
Upload
darren-short -
Category
Documents
-
view
215 -
download
0
Transcript of Jeff Fried & Keenan Newton Senior Technical Product Managers Microsoft Corporation SESSION CODE:...
Getting to know FAST Search for SharePointJeff Fried & Keenan NewtonSenior Technical Product ManagersMicrosoft Corporation
SESSION CODE: OSP206
Best of SharePointBest of High-end
Best of Microsoft
• Dynamic user experiences• Advanced content processing• Broad repository connectivity• Extreme scale and performance• Open and Extensible Platform
• Simplicity and low TCO• Familiar, integrated
developer tools• Windows 7 Integration• Robust partner and platform
eco-system
Enterprise Search from
Microsoft• Out-of-the-box intranet search• People Search• SharePoint platform integration• No code customization
Our Vision: A New Choice in Search
Microsoft Enterprise Search- ProductivityProducts for Every Customer Need
OOB General Productivity Search with light customization
Heavily customizable OOB General Productivity Search and easy to build Search-Driven Applications
• Common UI Framework• Social search features and integration• SharePoint platform integration • End user and site administrator enablement
Common across the product line
• Common Connector Framework (BDC)• APIs and developer Experience• Admin & deployment capabilities• Operations advantages (SCOM, scripting)
OSP217 What’s New in Enterprise Search in Microsoft SharePoint Server 2010
FAST Search Extends SharePoint Superset of capabilities; common base functionality and platform
End Users Content
Content search
People search
Common query and federation APIs
Common connectors and content APIs
Best High-End Content Search based on FAST ESP
Unique Social Search from SharePoint Server 2010
Common IT Pro and developer experience
FAST Search for SharePoint
SharePoint Server2010
Advanced Content Processing
+ Profiles
Introducing FAST Search for SharePointA new choice for enterprise search that eliminates compromise
Productivity Search Experience
Search Driven Applications
A Single, Cost-effectiveInfrastructure
Customized productivity search and search driven applications
Social, Conversational++, Visual, Contextual
Connectivity+, Security, Relevance+, Performance, Administration+, Scale+
+
SharePoint Enterprise CAL + FAST Search for SharePoint server license
SharePoint Standard CAL + SharePoint server license
General Productivity searchLight customization
Social, Conversational
Connectivity, Security, Relevance, Performance, Administration, Scale
SharePoint 2010 search and FAST search comparison
Topics
OOB Search ExperienceCustomizing SearchSearch-Driven ApplicationsPlatform Architecture Deployment and Manageability
ContextualTailored to your
role, task, and perspective, making you more productive
ConversationalChange how you
interact with information, giving you better answers
VisualHelp you identify patterns and
discover new insights
Turn information into business outcomes through engaging experiences.
SocialConnect and collaborate to apply your
talent more effectively
OOB Search ExperienceFAST Search for SharePoint
Demo
OOB User Interface SummaryVisual and conversational interaction with precise control
Built on SharePoint Search CenterLeverages all of innovations in SharePointOpen Web Parts, Federation, query suggestions, related queries, Did you mean?
Visual results connects users with contentThumbnails for Word and PowerPointVisual Best Bets highlight premium content Preview in browser without leaving the results
Deep Refinement
Thumbnails
Previews
Sort on any field
Similar Results
Discovery: Find and Explore
Penecillin
LSD
Uranus
Viagra
Safety glass
Infrared radiation
Microwave ovens
Inkjet printers
Corn Flakes
Chocolate chip cookies
Deep RefinementEnables precise control of results
Contains exact countsLeads to discovering non-obvious relationships, key data trends, and deep analysis of your content
Enables conversational experience You will never miss any content; enabling better findability and exploration across the entire result set
Exact Counts
Sorted by frequency
Provides a sorted viewEach refiner is sorted by frequency, from highest to lowest, indicating the importance of each term
Topics
OOB Search ExperienceCustomizing SearchSearch-Driven ApplicationsPlatform Architecture Deployment and Manageability
Customize search to meet your business needsKey ingredients to a great customized search experience
Search in the language of your business
Deliver results that are contextually relevant
Tune relevancy to meet diverse needs
Process content with advanced linguistics
Customize the user experience to build engaging applications
Custom Query Suggestions
Query Suggestions & the Term Store
# retreive values from termstore$ts = Get-SPTaxonomySession -Site http://esgismoss14$groups = $ts.DefaultKeywordsTermStore.Groups$groups | select Name, Description, TermSets
# Get a reference to the query application$searchapp = Get-SPEnterpriseSearchServiceApplication -Identity "FAST Query SSA" # Add new query suggestionsnew-spenterprisesearchlanguageresourcephrase -SearchApplication $searchapp -Language En-Us -Type QuerySuggestionAlwaysSuggest -Name "faster“
# Start the job that deploys the query suggestions (runs once every 24 hours)Start-SPTimerJob -Identity "prepare query suggestions" # List query suggestionsGet-SPEnterpriseSearchQuerySuggestionCandidates -SearchApplication $searchapp
# Remove query suggestionsRemove-spenterprisesearchlanguageresourcephrase -SearchApplication $searchapp -Language En-Us -Type QuerySuggestionAlwaysSuggest -Identity "faster"
Maintaining a custom query suggestion list using the SharePoint Term Store repository, instead of putting the information in a list on a site.
Visual Best BetsVisual cues to highlight essential content
Built on SharePoint KeywordsMatches keywords and synonyms that are contextually relevant to users. Include banners, videos, external websites.
Easy and quick to setupPoint and click setup for site admins. Set and forget with content expiration dates . Web Parts allow for easy page customization
Visual Notification
Web Part Flexibility
What’s for Lunch?Visual Best Bets at Hafslund
Audience-specific search experiencesUse User Context to meet the needs of diverse groups
User context
Inform-ation conte
xt
Application context
Social conte
xt
Renee LoEngineeringContoso Consulting”What should I know about implementing ERP?”
Alan BrewerSales ManagerContoso Consulting”What should I know about selling ERP consulting?”
Username & Group Memberships
LocationLanguages
Business UnitDepartment
TeamTime of Day
Preferred SitesSharePoint Audiences
Interests & Current ProjectsContext of Current Task
Quickly build a contextual experienceUser based tools for creating results that are relevant to your users
Pick the right ingredients Match the proper terms and contexts to boost relevancy for targeted users to ensure your users are always finding the right content
One-way synonymsKeywords map to other termsTwo-way synonymsKeywords become equivalent to other termsBest BetsHighlights key resources that are always relevant to a keywordVisual Best BetsExtend Best Bets with pictures, video, Silverlight controlsDocument Promotion / DemotionTailor specific document relevancy
Create new user contextsSite administrators create contexts based on user profiles to deliver relevant results to the right audiences
Create new keywordsSite Administrators have powerful and simple tools to configure the search experience for groups of users
User ContextFAST Search for SharePoint
Demo
Tune relevancy to meet diverse needsA flexible solution for your organizations, groups and individuals
Optimize Relevancy for broad intranet use
Query results with the default relevancy
”I want to know about my customer Woodgrove Bank and customers in Financial
Services"- Alan Brewer, Sales Lead
New Default Sorting
Promotes relevantResults
Quickly tailor relevancy models Deliver the right results to the right people by creating new Rank Profiles
”I want to get right to the technical documents"
- Renee Lo, Consultant
Documentation, RFPs and SOWs are now promoted Same Results
different order Users can select rank profiles in the sort by box or create their own default views by modifying the web part
Rank ProfilesTune relevancy without impacting the default algorithm
Quality Also known as static rank, consists of multiple managed properties including site, URL depth (preference for shorter URLs), and relative importance of links to this document.
Authority Applies when the query word falls in the link or anchor text.
Query Authority Maps the popularity of a document, or the click-through rate when documents are clicked as a result of a query
Freshness Increases the relevancy if a document was recently created or modified, based on the last modified property.
Proximity Applies to where query terms fall and how close they are to each other within a document
Context Increases the rank of a document if the query term is a managed property associated with that document
Managed Property Effects relevancy when a managed property contains a specific value, such as Woodgrove Bank or Financial Services
Out of the box relevancyTuned for great general productivity experience, relevancy improves with click-throughs and link text analysis.
Extend the default algorithmsCreate new default relevancy models. Blend static and dynamic ranking parameters to instantly improve search results.
Custom Rank ProfileFAST Search for SharePoint
Demo
How to create a Rank ProfileIT Pros are empowered to create new profiles quickly
Rank Profiles created in PowerShell by extending the default relevancy algorithm…
… and are exposed in the user interface by modifying the sorting web part.
Advanced Content ProcessingA new level of search quality
End UsersContent
Fed
era
tion
OpenSearch Source
Content Processor
CrawlerIndexerQuery
Processor
Search Center
MetadataRelevanceControl
UserContext
IndexingConnectivity
Index Partition
…
Form
at
Con
vers
ion
Lan
gu
ag
eD
ete
cti
on
En
tity
Extr
acti
on
Lem
mati
za
tion
Map
per
…
Content Processing Pipeline
Breaks down content to the smallest addressable chunks to build meaningUnderstands file encoding, data formats, and written languages Supports 400+ file formats, 80+ languages
Process content with advanced linguisticsAutomatic and detailed analysis creates a great search experience
Map Crawled Properties Maps all of the metadata that was discovered by the various pipeline stages Web Link Analysis Analyzes documents for hyperlinks extracting anchor text which reinforces the authority ranking of a document.
Document Vector Creates a unique representation of a document that reflects important terms and frequency of occurrence. Used to find similar documents.
Date and Time Normalization
Converts dates and times to a standard representation, to handle locale specific representations. For example, knows that 14-Mar-10 is equivalent March 14, 2010.
Entity Extraction Finds terms in the content and maps them to predefined categories. Out of the box support for People, Companies and Locations, but can be extended to any category.
Lemmatization Finds the root of a word for a given language. For English it maps run, runs, running and ran back to a single lemma. Understands language specific grammar and context.
Tokenization Apply the language specific rules for identifying words, concepts, idioms and phrases. Also applies custom word breakers found in part numbers or telephone numbers.
Language Encoding and Detection
Identifies the native written language and locale specific encoding so that the proper dictionaries can be used by the tokenization and lemmatization stages
Format Conversion Extracts plain text from multiple file formats, encodings, and applications
FAST Content Processing Pipeline
Configure Optional Processing StepsXML Properties mapperOffensive Content FilterPipeline Extensibility
Calls external applications for custom item processingField CollapsingAdd Custom Processing
Content classificationGeo-taggingMachine translationSentiment Analysis
Extending Pipeline capabilitiesSafely add additional analysis and processing
What is Custom Processing?Pipeline Extensibility is a specially defined stage that takes a set of crawled properties, as flat text as input and maps output to another crawled property
Custom Processing is Safe Executable arguments and temporary files automatically handled in sandbox with timeouts. Runs before Crawled Property mapping stage, making new metadata accessible in SharePoint
Type Doc ID Title Author Date Size Keywords Companies Expertise Industry ... Body Text
123 Sales Forec… John Doe 2010-04-15 386 KB sales; pipe… Contoso; Tail.. Strategic Con.. Financial Ser… … IT Market Gr…
456 … … … … … … … … … …
963 … … … … … … … … … …
Map metadata to Managed Properties Automatic association of metadata to content
Expertise
Strategic Consulting
Market Analysis
IT Implementation
…
Industry
Financial Services
Manufacturing
Technology
...
Crawled Properties
Managed PropertiesIndex Schema:
Companies
Contoso
Tailspin Toys
Woodgrove Bank
…
Mapped in Central Admin Or PowerShell
Crawled Properties Standard document metadata discovered by the crawler or extracted from the full text by the FAST Content Processing Pipeline
Managed Properties Map one or more Crawled Properties to a single field. Enables sorting, refinement, relevance tuning and fielded searching
Title
Sales Forecast
Any data can befound and mapped
Managed Properties + RefinersFAST Search for SharePoint
Demo
Topics
OOB Search ExperienceCustomizing SearchSearch-Driven ApplicationsPlatform Architecture Deployment and Manageability
Search Driven ApplicationsMeet all the search application needs you have across your business
“How do I support theunique search needs of
teams and work that impact our business?”
To do so, you need a search platform that has• A deep understanding of your
information• Flexible relevance to meet
diverse needs• A customizable UX to increase
user efficiency
Sales: 360o Customer Insight
Services: Knowledge Browser
Marketing: Competitive Intelligence
Research & Development:Innovation Portal
Support:Call Center Advisor
Operations:Systems/Logistics Portal
Legal, HR, IT, Finance, ……
Search-Driven ApplicationFAST Search for SharePoint
Demo
OSP218 Customizing FAST Search Server 2010 for SharePoint
6/10/2010 9:45-11:00 AM Room 261
News and external opinion to monitor and send to
clients
Top information from Woodgrove…new market view report to send to clients
Drilldown to single view with all clues about a customer: portfolio, holdings, communications, annual and quarterly customer plans, etc…
Set of Customers to explore
Experts to help, with availability and rating
Immediate actions on selected items
View of information across different pivots, with drilldown
Finds customers beyond direct terms (REIT ~ real estate)
Start with a simple query, quickly find and explore
Content Crawling: bring in data from lots of placesOOB connectors to SharePoint (reports, account documents), exchange public folders, shared files; BDC with customization in SPD (no code) for customer portfolio/holdings
Content processing: creating metadataNames of holdings, offerings, key concepts, companies, peopleSynonyms for key concepts (real estate ~ REIT)
OOB web parts configured for style Federation, People Search, Search actions
Custom web parts for visual navigationRoll-up configured via results collapsingCustom relevance profileSharePoint workflows for act-on-selected-items
How would you create this?
Back End Processing Tasks:Load content from many different places
Out of the box connectors for SharePoint, exchange public folders, and shared filesSharePoint Designer to configure connection to customer portfolio/holdings database
Create custom metadata with content processing pipelineNames of holdings, offerings, key concepts, companies, peopleSynonyms for key concepts (real estate ~ REIT)Roll-ups configured with optional results collapsing stage
Create custom relevance profileDesigners can stylize the User Interface
Apply styles to web partsFederation, People Search, Search actions
Build custom web parts for visual navigationUse SharePoint workflows to perform business specific actions
Leveraging the platform to build applicationsPutting together all of the pieces to build search-driven applications
Search 2010 ArchitectureThe platform for Search Customization
SearchWeb Parts
SharePoint Search Index
OpenSearch /Custom Source
SharePoint Indexer
Federation OM Web Service, RSS
FASTSearch Index
FAST Indexer
What’s New in 2010?Primary Search Web Parts now UnsealedFederation now a key Public OM layer
All Web Parts built on federationQuery alteration, custom Runtimes, blending results from multiple sources
Web Service / RSS Enhancements
FAST Search / SharePoint Search:Shared Web Parts, RSS, Web ServiceShared Federation OMIndex and Crawling SeparateFAST unique Content Processing Pipeline
SearchWeb Parts
Federation / Query OM Web Service, RSS
Content Processing
Topics
OOB Search ExperienceCustomizing SearchSearch-Driven ApplicationsPlatform Architecture Deployment and Manageability
FAST Search Service Architecture“Sandwich” with two SSAs
End Users Content
Content search
People search
Common query and federation APIs
Common connectors and content APIs
Best High-End Content Search based on FAST ESP
Unique Social Search from SharePoint Server 2010
Common IT Pro and developer experience
FAST Search for SharePoint
SharePoint Server2010
Advanced Content Processing
+ Profiles Query SSA Content
SSA
FAST Search Server 2010Summary of architectural components
Custom Front-End
OpenSearch or Other Sources
SharePoint Front-end
People Search
Qu
ery
Obje
ct M
od
el
Query and Result
Processing
Security AccessModule
SearchCore
Indexing
Federation Object Model
Query Web Service
AdvancedContent
Processing
Linguistics
WebLink
Analysis
Connectors• Web Crawler• JDBC
Connectors• SharePoint• File Traverser• Web • BDC• Exchange• Notes• Documentum
Microsoft System Center Operations Manager
Monitoring Services
Administration and Schema Object Model
Site Collection Level Admin UI
• Keyword Management• User Context Management• Site Promotion/Demotion
PowerShell
• Schema configuration• Admin configuration• Deployment configuration
Central Administration UI
• Property mapping• Property extraction• Spell-checking
FAST Server(s)
SharePoint Server(s)
Other Server(s)
Content
FAST Search for SharePoint Scaleout
Content Volume
Query Volume
Scale-out multiple “dimensions”Query VolumeContent VolumeIndexing freshness
Redundancy optionsSearchIndexing
*Depends on content and hardware specifics
Search and Indexing
Crawling and Content Processing
Query and Result Processing
Back-end with extreme and flexible scale out options
No theoretical upper bounds!
FS4SP – Main Components to Scale
SharePoint CrawlerCapacity: ~30 mill items per crawler node, SQL server needs to be scaled for increased IO
Web AnalyzerCapacity: ~30 mill items per web analyzer node
Indexer/search nodeCapacity: ~15 mill items per indexer/search nodeNote: Will also be used to scale the Item Processing component
SharePoint Crawler
Crawl DBSP CrawlPeople Crawl
Indexer/Search
FAST-FSTIDX-11Index/Search
Web Analyzer
FAST-WA-1Web Analyzer
FS4SP – Minimum Deployment
Single-tier setup Two-tier setup
SP2010 Farm FAST Search for SP 2010 Farm
SQL 2008 Cluster
Crawl DBSearch Admin DB
All roles
SP2010 Farm
All roles
FAST Search for SP 2010 Farm
All roles All roles
Note:None of these environments are redundant. For a production environment, you should at a minimum consider running the SharePoint and FAST servers in a virtual environment and with a mirrored or clustered SQL server.
FS4SP – Small Deployment
Scope:Query rate: < 10 QPSContent: < 10 M itemsStandard configuration
SP2010 Farm FAST Search for SharePoint 2010 Farm
SQL 2008 Cluster
WFEQuery SSA
WFEQuery SSA
Crawl DBSearch Admin DB
FAST-ADM-1Admin
Content Distributor 112 Docprocs+Web Analyzer
FAST-FSTIDX-11Index (Search)
Indexing Dispatcher 212 Docprocs+
QR Server
FAST-FSTIDX-21(Index) Search
QR Server
SP CrawlPeople Crawl
SP CrawlPeople Crawl
*
Note:Servers marked with a * are only
needed for redundancy.
*
*
FS4SP – Medium Deployment
Scope:Query rate: < 10 QPSContent: < 40 M itemsStandard configuration
FAST Search for SharePoint 2010 Farm
FAST-ADM-1Admin
Content Distributor 1Web Analyzer12 Docprocs+
FAST-FSTIDX-11Index (Search)12 Docprocs+
FAST-FSTIDX-12Index (Search)12 Docprocs+
FAST-FSTIDX-21(Index) Search
QR Server
FAST-FSTIDX-22(Index) Search
QR Server
FAST-ADM-2Content Distributor 2
Web Analyzer12 Docprocs+
(Enterprise Crawler)
FAST-FSTIDX-13Index (Search)12 Docprocs+
FAST-FSTIDX-23(Index) Search
QR Server
SP2010 Farm
SQL 2008 Cluster
WFEQuery SSA
WFEQuery SSA
SP CrawlPeople Crawl
SP CrawlPeople Crawl
Crawl DBSearch Admin DB
FS4SP – Large DeploymentScope:
Query rate: < 10 QPSContent: < 100 M itemsStandard configuration
SP2010 Farm
FAST Search for SharePoint 2010 Farm
SQL 2008 Cluster
WFEQuery SSA
WFEQuery SSA SP Crawl
People CrawlSP Crawl
People Crawl
Crawl DBSearch Admin DB
SP Crawl
FAST-ADM-1Admin
ConfigServerSpelltunerSamAdmin
Content Distributor 1Web Analyzer12 Docprocs+
FAST-FSTIDX-11Index (Search)12 Docprocs+
FAST-FSTIDX-12Index (Search)12 Docprocs+
FAST-FSTIDX-21(Index) Search
QR Server
FAST-FSTIDX-22(Index) Search
QR Server
FAST-ADM-2Content Distributor 2
Web Analyzer12 Docprocs+
FAST-FSTIDX-13Index (Search)12 Docprocs+
FAST-FSTIDX-23(Index) Search
QR Server
FAST-FSTIDX-14Index (Search)12 Docprocs+
FAST-FSTIDX-15Index (Search)12 Docprocs+
FAST-FSTIDX-24(Index) Search
QR Server
FAST-FSTIDX-25(Index) Search
QR Server
FAST-FSTIDX-16Index (Search)12 Docprocs+
FAST-FSTIDX-26(Index) Search
QR Server
FAST-ADM-3Web Analyzer12 Docprocs+
Secure, unified access to informationIndex or federate with content, applications, and services
Access content securelywith a full-featured content crawler and both index- and query-time trimming of results
Federate queriesusing OpenSearch to quickly access existing search indexes and online information services
Extend your reachusing Business Connectivity Services to access your content sources and business applications
OpenSearch Federation
Indexing Connectors
EnterpriseContent
Business Applications
Information Services
User Experience
Search Index
Websites and Intranet
File Shares
IBM Lotus Notes
EMC Documentum
SharePoint Server
Exchange Public Folders
AD & LDAP Profiles
Etc …
Simplified, powerful administrationA high-end enterprise search solution that’s easy to deploy and manage
Deploy easilyusing wizard-driven installation, a topology designer, and native support for 64-bit virtualization
Manage efficiently with full support for Microsoft System Center and PowerShell scripting to automate tasks
Streamline administrationwith a simplified admin console that helps you manage search services across your enterprise
Enterprise Search from Microsoft
UX ITDXGo beyond the
search boxEliminate
compromiseDo more
with search
Introducing FAST Search for SharePointA new choice for enterprise search that eliminates compromise
Productivity Search Experience
Search Driven Applications
A Single, Cost-effectiveInfrastructure
Customized productivity search and search driven applications
Social, Conversational++, Visual, Contextual
Connectivity+, Security, Relevance+, Performance, Administration+, Scale+
+
SharePoint Enterprise CAL + FAST Search for SharePoint server license
SharePoint Standard CAL + SharePoint server license
General Productivity searchLight customization
Social, Conversational
Connectivity, Security, Relevance, Performance, Administration, Scale
SharePoint 2010 search and FAST search comparison
Summary
FAST Search for SharePoint is NEW
It combines the best of high-end search, SharePoint and Microsoft
It extends SharePoint Server 2010Superset of CapabilitiesHighly capable and easy to manage
BIO206 Improving Business Intelligence with Search 6/10/2010 1:30-2:45 PM Room 238
Track Resources
For More Information: http://sharepoint.microsoft.com
SharePoint Developer Center: http://msdn.microsoft.com/sharepoint
SharePoint Tech Center: http://technet.microsoft.com/sharepoint
Official SharePoint Team Blog: http://blogs.msdn.com/sharepoint
Related Content
Breakout Sessions – See Conference Guide for full list of OSP Track Sessions
Interactive Sessions – OSP Track has 10 Interactive Sessions – OSP01-INT – OSP10-INT
Hands-on Labs – OSP01-HOL – OSP20-HOL
Product Demo Stations: Yellow Section, OSPOffice 2010, SharePoint 2010, Project Server 2010, Visio 2010 have kiosks & demos
Complete an evaluation on CommNet and enter to win!
Play the Microsoft Office & SharePoint Track Tag Contest
Download the Microsoft Tag ReaderOpen the internet browser on your mobile phone and visit http://gettag.mobi
Come to the Expo Hall – Yellow Section OSP Info Desk for Official Rules & Collect Additional Tags from all OSP Track Sessions, Speakers and Expo Hall!
Grand Prize (1)Xbox 360 Prize Package
and Microsoft® Office 2010
Daily Prizes40 copies of
Microsoft® Office 2010
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to
be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
JUNE 7-10, 2010 | NEW ORLEANS, LA