Real-life experience building search-driven applications for product-centric sites

Marius ConstantinescuSwitzerland, GMT+1

April 16th /17th, 2014

MVP SharePoint, acting as Solution architect focusing on solutions around enterprise content management & collaboration, search, customizations, frequently involved in information architecture and governance planning or integration solutions. MCT for more than 5 years, holding a number of professional certifications around SharePoint, Office 365 and Web solutions, one of the leaders of the Swiss SharePoint Club, frequent speaker at events.

Marius ConstantinescuManaging Consultantblue-infinity SAGenevaSwitzerland

Contact

@c_mariusc_marius@msn.comhttp://nettitude.wordpress.com

1.Introducing organizational scenario1. Looking for new innovative ways of accessing content

2.Understanding Cross-site content publishing

1. Why another way of content publishing?2. What are content Catalogs and why do we need them?3. Cross-site publishing vs. In-Place publishing

3.Does it really work ? > our experience, trade-offs, solutions

1. Authoring vs. Publishing sites > Content types, Catalog connections, Crawled/Managed properties, refiners

2. Managed Metadata driven navigation > SEO, friendly-URLS

4.Search-driven applications1. Content sources, Query results, Query rules, Display templates2. Adaptive Search and user segmentation3. Usage analytics, recommendations

Agenda

Organizational content aggregation Source > Collaboration

Workspaces Multiple collaboration site

collections Each to its own content

publishing processes & security management

Target > Published Intranet Flexible content publishing

with respect to security rules already in place

Support content aggregation across multiple sources

Multilingual support Relevant content targeting Usage tracking Performance considerations

Process – planning & preparationsI. Planning

Identified authoring sites, interviewed owners, identify current content publishing processes

Mapped-out existing content across various site collections, including security

Identified fields meant for sharing, filtering, sorting or querying across

II. Preparations1) At source

Deployed new structures (sub-sites, lists, etc.) to better serve our content rollup needs > new IA promoting consolidated content types, site columns, taxonomy

Enable lists as Catalogs Use PowerShell (and migration tools) to migrate

existing content into new structures Re-classify content where needed Establish common Assets location, update links

2) At Destination Prepare target publishing sites topology

3) At Farm level Configure dedicated Content Source & Crawling

schedule in Search SA for the cross-site publishing (enlist Catalogs into Content Source)

Run a Full-Crawl

Business value of Search-driven publishingIndependent Authoring and Publishing

Promotes Content reusability Clear separation of responsibilities Support for Multilingual content publishing scenarios

Build next-gen user interfaces Advanced query building capabilities empowering Information Workers Freedom of choice in applying different layouts to category/items pages Display templates (for refiners, search results, etc.) for unprecedented

UI flexibility

Power of search Analytics and recommendations Scaling and performance Across site collection boundaries Automatically updated UX

On-premises vs. Online features availabilityGenerally available• Query rules, display templates, rank models, managed navigation, search

schema, analytics & recomandations, entity extraction, CSOM, KQL, query builder, result sources

Available in Office 365

• Content Search web part (CSWP), Content Iterm Reuse Web Part (CIRWP)• Cross site publishing (XSP) & Category/Item page layouts

On-premises only• Product catalog site collection template• User segmentation and adaptative interfaces• Taxonomy Refinement Panel Web Part  • Faceted navigation – supports taxonomy refiners• No Suport for Anonymous access for XSP (e.g. Public web-sites)

Knowledge Center – our solution

Central Publishing site acting as knowledge hub

Tabbed user interface aggregating various type of content (News, Tips & Tricks, Articles)

Customized master & unified page layouts for multiple “products”

Faceted navigation using tabbed widgets showing composite refiners using article managed metadata

Benefits of using cross-site content publishing

Separate presentation from storage• Different teams working on content

& design• Choice of design on authoring vs.

published sites

Flexible and dynamic• Can be used virtually for any type of

list (not only Pages)• Multiple authoring sites > multiple

publishing sites

Breaking down site collection boundaries• Write once > reuse multiple times• Easily publish content across site

collections – not using Content Deployment

Eliminate large list thresholds• Take advantage of search high

performance• Ability to distribute content across

many lists/libraries, with adequate security

Allows flexible & dynamic publishing• Whenever you want, wherever you

want

XSP Building blocks

Content Catalogs

Faceted navigation, Refiners•Managed Navigation•Managed properties

Content Search, Catalog Item

Reuse Web parts

Query Builder

XSP Search driven architecture

CRAWLER

Index

Documents

Pages

Catalogs

Digital Assets

SharePoint Content

Searc

h

Query

Rule

s

Reco

mm

endati

on

s

Share

Poin

t A

PI

Pag

e F

ram

ew

ork

Web

Part

Fr

am

ew

ork

Publishing

WWW

LOB Data store

External

SEO

Managed Navigation

Content

Query

Content

Search

Publishing Page

Term Context

Cross-site content publishing process >simplified view

Content Source• Authoring Site Collection(-s)• List(-s) to share as Catalogs• Configure anonymous access• Primary key fields (used in

building the final unique URL to the “product”)

• A Managed metadata column used in the Category-based navigation

Search & Faceted Navigation• Share navigational Term-Sets• Prepare for Faceted navigation

> used for Category browsing (generic or specific)

• Define Refiners (term-specific or same for all)

Content Destination• Target Publishing Site

Collection (-s)• Connect to Catalogs• Select “product” ID &

[Group]• Establish URL building

User Interface• Select/Customize Master Pages,

Category & Item pages• Customize Search Display

templates• Search/Catalogs web parts

(Relevant or Recommended results)

• User Segmentation

Not using “Product Catalog” site template for Authoring site requires activation of the “Cross-Site Content Publishing” site collection feature + creation of appropriate site columns used later as managed properties, category navigation, “product” unique Id, etc.

The “Product Catalog” site template does not exist in Office 365

…a little more insight into the XSP process

Deploy content types at site

collection level [recommended]

Deploy site columns vs. list columns[If any] Create

custom lists, attach content types

Populate (& publish) all fields needed as Managed Properties

Enable lists as Catalogs

Run Full-Crawl

Deploy

Activate Cross-Site Collection

Publishing“Publishing”

site NOT required (but

recommended when

variations will be used)

Prepare

Identify Authoring site(-s), consider

multilingual, content security strategies

Identify data to publish across >

content types hierarchies & shared

site columnsConsider faceted

navigation hierarchy & refiners

Consider the one column to hold the

Unique Item Id

Plan

A. Either start by using the “Product Catalog” site collection

It run as a Publishing site, and activated “Cross-site Collection Publishing” site collection feature – enables the “Use as Catalog”,

Added the Product Catalog list with Content types (Product & Product with Image)

Site Columns (Item Number, Group Number, Language Tag, Item Category & Rollup Image)

B. Any other site template (including existing sites) would require redoing the steps & structures manually

1. Preparing authoring site(-s)

[Office 365] - The “Product Catalog” template does not exist (choose option “B”)

Master data planning and deploymentMultiple Content Types scenario:

• Define base Content type - not needed to inherit from default ”Product with Image” (do consider adding a sort of unique “product Id” – for friendly-URL building)

• Consider specificities of each product & plan adequate columns for each “product-specific refiner”

• Faceted navigation – one or multiple managed metadata columns can be used to build navigational hierarchies.

• [Recommendation] Do not think a too deeper hierarchy (< 3) – allow for additional refiners in specific cases.

Televisions Laptops Cameras

Title Title Title

Brand Brand Brand

Description Description Description

Price Price Price

 Item Number Item Number Item Number

Technology (LED; LCD, Plasma)

CPU Type(DSLR, Bridge, etc.)

Max- Resolution(720P, 1080i, 1080P)

Screen Size Optical Zoom

Screen-Size (40,42, 50, 60) Hard Drive Lens Type

Tip – columns added by default “Product with Image” content type are automatically added as Managed Properties – for your custom ones you need to consider doing it manually.

Master data planning and deployment

News Announcements Classifieds Events

Title Title Title Title

Start Date Start Date Start Date Start Date

Description Description Description Description

Category (Science, Technology, Sports, )

Category Category (Automotive, Real-Estate, Carrier, Pets, )

Category

  Expiry Date Expiry Date End Date

Scope(Local, Regional, Global)

Scope Scope Scope

Accessibility (Private, Public, Team, Project, Department, )

Accessibility Accessibility Accessibility

 TypeHire, Retirement, Policy

TypeReal-Estate > Buy, Sell, Rent, Professional Services

Type (Meeting, Holidays, )

Define the shape of the data to be used

• Map the data, identify Entity specific attributes

• Define Content types - a ”Product with Image” provided with default “Product Catalog” list

• Reuse shared site columns

• At least one column must be Managed Metadata (Item Category - by default already mapped to “Product Hierarchy”) – used to define managed Navigation

• When creating columns use SingleWordsWithNoSpaces ( or special Characters – this will become Internal name (fixed) > rename will only update Display Name

• By default Approval is ON – either approve items or disable it prior to Full Crawl – items is ignored otherwise.

Tips• Site Columns get promoted as managed properties in search (but

only as Text columns)• Create your own base CT to have others inherit from - idea of

sharing same common base• Content Type Hub (not available in Office 365) or Solution-based

deployment could be an option

Documents

Catalogs

PagesDigital Assets

SharePoint Content

www

Ext. CMSLOB + more

External ContentUser Behavior (+custom events)

Content & Authoring FAST Search Publishing User Experience

REST/OData

Managed Navigation

Page Framework

Content Search WP

Display Templates

Devic

e C

hann

els

Vari

ati

ons

Dis

pla

y T

em

pla

tes

Query

Reports

FAST Index

Reco

mm

end

ati

on

s

Query

Rule

Engin

e

Cra

wle

r

Analysis Services

Steps & considerations impacting your deployment…

Faceted navigation & Refiners

Configuring

Configuring Metadata-driven navigation

Metadata driven navigation, cont.

• Linked termsReused terms between term sets

• Pinned termsBlocks changes to linked terms (read-only)

3

1

2

Faceted navigation & RefinersAsk yourself

Multiple Content types? Which columns should be searchable,

used in filters? What information will be used in the

Navigation hierarchy > terms should be pinned or not?

Define content types Shared (Reusable) Columns Preferable to use Site columns vs.

List columns

Identify data facets > future refiners (filters) Could be different per “product”

(catalog item) must be configured

Product

CodeTitle

Television

Display Type (LCD, LED, Plasma)HD-Ready (720P, 1080P, UHD)

DescriptionPrice

Computer

CPU

Screen Size (13, 14, 15, 17)

Camera

Type (DSLR, Point'n'Shoot)Resolution (in MegaPixels)

All from Product +

Screen Size (40, 55, 60, 77)

Hard drive

RAM Optical Zoom (24x)Digital Zoom (12x)Lens TypeStorage Format (SDHC, SD)

CodeTitle

DescriptionPrice

TitleDescriptionPrice

TitleDescriptionPrice

Code Code

What are they and why we need them?Refiners are filters narrowing search

results

They come in different types and shapes – tightly related to underlying data type

[New] Range-based (sliders) and multi-valued refiners are available

Only managed properties can be configured

[New] Via the use of «Yes latent» Farm Administrators can delegate to a Site Collection administrator the possibility to decide.

[New] Refinement web parts can now be configured individually in-page (horizontal or vertical)

Refiners – why use them?

Make sure data exists for each column before a Full Crawl

[New] Site Collection Admins can trigger re-index

Journey from field > crawled / managed property > refiner

Site columns, Managed properties, Refiners

Televisions[Catalog]

Cameras[Catalog]

Computers[Catalog]

Content Search Web Part

Managed Properties

Content Search Web Part

Content Search Web Part

Title

ows_q_TEXT_ ProductCatalogItemNumber

ows_q_TEXT_ ProductCatalogGroupNumber

ows_taxId_ ProductCatalogItemCategory

ows_r_IMGE_ PublishingRollupImage

ows_q_CHCS_ DeviceScreenSizeSite Column

Title

Brand

DeviceScreenSize

Site Column Type

Single line of text

Managed Metadata

Choice

1

Crawled Properties

2 3

Title

ProductCatalogItemNumberOWSTEXT

ProductCatalogGroupNumberOWSTEXT

owstaxIdProductCatalogItemCategory

PublishingImage

DeviceScreenSizeOWSCHCS

4

Refiners

Refiners

Automatically created managed properties are created with TEXT data type

Tips• The Catalog Item Reuse WP auto detects rendering mechanism based on the naming of managed

properties

Farm Admins can use the NEW “Yes-Latent” to allow promotion by Site Collection administrator & avoid the need to Full Crawl

Map properties starting with ows_PropertyName

• Indexed property bag used to flag lists enabled as Catalogs [IndexedRootFolderPropertyKeys: {PublishingCatalogSettings, IsPublishingCatalog}],

further mapped to IsPublishingCatalog managed property

• Automatically created managed properties are created with TEXT data type (except for Person and Managed Metadata) All site columns require manual mapping to proper types (*)

Either loose capability to use proper comparison (e.g. Price, Size, Width) Either require use of JavaScript to re-format data in the refiner display template

Require marking them as Queryable & Refinable

• Use automatically mapped Refiner groups (especially in Office 365) RefinableDate00/19, RefinableDecimal00/09, RefinableDouble00/09

RefinableInt00/49, RefinableString00/99

• Columns in standard Catalog lists are automatically mapped

…putting the pieces together

(*) Require either Farm Administrator or Search SA administration delegation – unless provided groups are usedRead more @ http://nettitude.wordpress.com/2013/10/17/introducing-indexed-property-bag-in-sharepoint-2013-a-searchable-collection-of-properties/

Faceted navigation & refiners Content Item reuse web part Content Search web part Display templates for custom refiners, catalog, search results

User experienceConsiderations

Cornerstones of a product-centric site

Category & item pages

Term-driven pages

Category & product item pagesCategory page Catalog-Item Page

Configuration of Term-Driven pages

…anatomy of the term-driven Category page

• The actual Catalog(contentclass:sts_listitem OR IsDocument:True) SPSiteUrl:http://contoso/sites/catalog ListId:3a3f66cd-9741-4f15-b53a-b4b23c3187ea

• The Category column owstaxidProductCatalogItemCategory• The term under selection (its GUID) #c771504f-6a2f-423f-98de-0e12fcfa08c9

(:) is the actual separator specifying the “contains”

…anatomy of the term-driven Item page Content Item Reuse default Web

part is automatically provisioned with the default “Product Item”

...configuring URL buildingClick icon to add picture

Giant step towards modern web

Why the need for

new concept

s?

Editing XSLT is a cumbersome - even for the veteransModern Web requires Standardization – empower Power-UsersClient-side "templating" techniques are current trends (e.g. Knockout, Kendo UI, Razor)What

are design templat

es?

Re-usable Styles Files for your Content based & Search Results Web-Parts

Where does one

use "Design

templates"?

Search Results & Web parts, Catalog reuse web parts

What does one need

to create a new

template ?

Access to Master Page libraryYour favorite web development tool (even Notepad) to create them

To eliminate the for designers and power users to work with XSLT (and the use of ddwrt JavaScript functions) each time they a new look & feel is needed, SharePoint 2013 introduces Design Templates.

Each display template consists of two files (in Master Page library under Display templates):

An HTML file (.html) - the actual template

JavaScript File (.js) - the script that makes the magic happen – automatically generated

jQuery supported, or custom JavaScript and CSS

Custom managed properties can be displayed (from search results)

Design Templates

Applying display templates to a result source

Control Display Template

Which Control Display Template

Ranking & relevance Recommendations

Monitor & improve

Fine tune your deployment

How usage analytics, recommendations can improve relevance

Usage AnalyticsThe importance of

Analytics processing component

Search Analytics

• Analyze content being added into the index (such as links, anchor text) and together with clicks on search results gets added into the Links database, leading further to improved relevance & reporting.

Usage Analytics

• User’s actions (such as Clicking on links, viewing search results) generate usage events (standard or custom) which can further contribute to recommendations, popularity reports, most viewed links, etc.

Search Anal

ytics

•Search Clicksboost/demote ranking of items in the search index based on user clicks•Deep Linksimprove relevance of sub-pages in a site based on what user clicks in results

Usage A

nalytics

•Usage Countshow many times items (in lists/library & search results) are Viewed, Clicked•Recommendationsidentify patterns in usage, relying on Usage Counts to build relationships graphs•Activity rankinganalyses trends & adapts ranking of items in search results based on the Usage Events

Usage analytics – how does it work ?1) Views – a usage event is registered when

a visitor views an item on your site (event type ID 1)

2) Recommendation1) Displayed (ID 2) – a usage

event is registered when an item is displayed as a recommendation on your site.

2) Clicked (ID 3) – a usage event is registered when a visitor clicks an item that is displayed as a recommendation on your site.

• Map a single crawled property to UsageAnalyticsID managed property to support recommendations

• Property must be part of the URL for the Usage Event to be recorded properly

• At least 3 different users must be performing the activity

• Log events pointing to item URL in the Authoring site

#Get Usage events available$ss = Get-SPEnterpriseSearchServiceApplicationProxy$ssp.GetAnalyticsEventTypeDefinitions([Guid]::Empty, 3) | ft

More at http://blogs.technet.com/b/tothesharepoint/archive/2014/01/21/modify-the-content-search-web-part-display-template-and-use-windows-powershell-to-start-usage-analytics-in-sharepoint-server-2013.aspx

Ranking

• Ranking models compute search results relevance through rank evaluation generating a rank score classified in 3 main groups (General, People & Special).

• Ranking can be influenced by• Query rules (prioritize promoted

results, add additional result blocks, directly influence ranking – dynamic ordering, sorting, query-based use of a different ranking-model, etc.)

• Search Schema (adapt contribution weight by changing context of targeted managed properties) – requires Farm /Search Service Administrator

• Custom ranking model

• Default Search model• Search Ranking Model with

Two Linear Stages

General

• Catalog Ranking Model• Popularity Ranking Model• Recommender Ranking

Model

Special

People

SP24 Search SessionsSession

Time (GMT)

Time (EST) Title Level Speaker Twitter

#SP24S090 5:00 1:00Using the Content Enrichment Web Service with SharePoint Server 2013 Search

Technical - AdvancedSezai Komur - NEC Australia

@sezai

#SP24S021 6:00 2:002013 Search, Display Templates, Query Rules, Result Types

Technical - BeginnerOmer Zubair - CSC

@Omer_Zubair

#SP24S047 8:00 4:00Real-life experience building search-driven applications for product-centric sites

Technical - AdvancedMarius Constantinescu - blue-infinity SA

@c_marius

#SP24S032 10:00 6:00Custom Indexing Connectors - How to integrate external system into your SharePoint Enterprise Search

Technical - AdvancedHeinrich Ulbricht - Communardo Software GmbH

@h_ulbricht

#SP24S083 11:00 7:00Search First Migration with SharePoint 2013 – Benefits / Disadvantages

Technical - IntermediateMax Melcher - Alegri International Service GmbH

@maxmelcher

#SP24S034 14:00 10:00The Search Immaturity Cycle, and How to Create a Search Strategy

Business - IntermediateJeff Fried - BA Insight

@jefffried

#SP24S009 16:00 12:0010 Things to Know about Search in SP2013 and Office 365

Business - BeginnerAgnes Molnar - Search Explained

@molnaragnes

#SP24S019 18:00 14:00 Search Topology and Optimization Technical - AdvancedMike Maadarani - MCM Consulting

@mikemaadarani

#SP24S050 22:00 18:00Search Driven Application Development in SharePoint 2013

Technical - IntermediateMatt Youngstrom - Magenic

@spguru

...the anchor will pick some of them up!

Questions?Post your questions in the chat...

Thank you!