Intelligent Search: Connected, Search-Driven & Mobile · Don Miller –2 Rules and an Observation...

More Chocolate Chip Muffins vs. More Chihuahuas and how AI and Machine Learning are impacting the “Search User Experience”

Don Miller

BA Insight

Intelligent Search: Connected, Search-Driven & Mobile

Don Miller – 2 Rules and an Observation

• Information - Consistency equals transparency, transparency allows you to surface and find the right information

• Relevance is subjective, but consistent metadata makes everything equal even if it is over tagged.

• You need to immediately start looking at how Machine Learning can impact your business. This is not a typical Gartner Hype Cycle

3

Observation - Machine Learning will not go into trough of disillusionment

▪

–

▪

▪

▪

▪

About BA Insight

4

Over 3.5 Million Users

AutoClassifier

70+ Enterprise Systems

Searc

h I

nte

rface

Query

Engin

e

SearchIndex(es)

Machine Learning

AI PlatformsGoogle Cloud AI

SmartHubConnectivityHub

Microsoft Cognitive Services

BA Insight Product Portfolio

▪

▪

–

–

▪

–

Our Technology Strategy

8

▪ Natural Language Query

▪ Bots

▪ Intelligent Query Engine

▪ Search UI (HTML)

▪ Analytics: Reporting and Recommendations

▪ API: Custom UI Development

SmartHub: AI-Driven Platform

9

▪ Standalone with Elastic and Azure Search and Solr

▪ Integrated with SharePoint/O365

▪ Being Integrated With Existing Applications:

– SiteCore, Outlook, Teams, Browsers, Netdocs, ServiceNow

SmartHub Configurations

10

▪ Automated Metadata Generation and Tagging

▪ Multiple Tagging Methods:

– Rule-based engine for Auto-tagging, Pattern recognition (PII,

GDPR)

– AI / Machine Learning

o Natural Language Processing: Entity extraction, Document

Summary Generation

o Image Analysis & OCR, Video Analysis, Sentiment Analysis

– Extensible Enrichment

AutoClassifier

11

▪ Tight Integration with:

– ConnectivityHub: Tagging of content from all sources

– Microsoft / Google AI suites

▪ Plug-Ins for In-Place Tagging of Other Systems

▪ Supports SharePoint, Elastic, Azure Search, ClarityNow

AutoClassifier: Architecture

Connectors to Many Enterprise Systems

• Aderant

• Adobe Experience Manager

• Amazon Aurora

• Amazon RDS

• Amazon S3

• Alfresco

• Azure SQL

• Bentley - Assetwise

• Biomax

• Box

• Confluence

• CuadraSTAR

• Deltek Vision

• Elite / 3E

• File Shares

• Google Cloud

• Google Drive

• HP Consolidated Archive

• (EAS, aka Zantaz)

• HPE Records Manager/

HP TRIM

• IBM Connections

• IBM Content Manager

• IBM DB2

• IBM FileNet P8

• IBM Lotus Notes

• IBM WebSphere

• iManage Work

• Jive

• Kaltura

• LegalKEY

• LexisNexis Interaction

• Lotus Notes Databases

• MediaPlatform

• Microsoft Dynamics CRM

• Microsoft Exchange

• Exchange Public Folders

• Microsoft SQL Server

• MySQL

• NetDocuments

• Neudesic The Firm Directory

• Objective

• OneDrive

• Oracle Database

• OpenText Documentum

• OpenText eDOCS DM

• OpenText eRoom

• OpenText LiveLink/RM

• OpenText Media Manager

• Oracle WebCenter

• PLC/Practical Law

• PostgreSQL

• ProLaw

• Salesforce.com

• SAP ERP

• SAP HANA

• ServiceNow

• SharePoint Online

• SharePoint 2016

• SharePoint 2013

• SharePoint 2010

• SharePoint 2007

• Sitecore

• Slack

• Any SQL-based CRM system

• Veeva Vault

• Veritas Enterprise Vault

(Symantec eVault)

• Website Crawler

• West KM

• Workplace by Facebook

• Xerox DocuShare

• DocuShare

• Yammer

Plus a proven process for creating new connectors to complex systems

15

Agenda – Today’s topic is how to impact search

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

▪

2019 Microsoft Search - Ignite

16

▪

▪

▪ SharePoint Multi-Geo

▪

▪

Hub Sites – Go big provide meaning in title

17

https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/multigeo-introduction

18

Graph and Delve – Cloud based

▪

–

▪

–

–

–

o

o

•

▪

–

▪

▪

Siri or audio conversion

19

Why does search matter

▪

▪

▪

▪

▪

▪

Customer Use Cases/Applications

21

▪

–

–

▪

Metadata is context and structure about information

22

Consistency equals

transparency,

transparency allows

you to surface and

find the right

information

Relevance is

subjective, but

consistent metadata

makes everything

equal even if it is

over tagged.

Internet Sets the Bar for Enterprise Search

WWW leading the way: Search & Refine

Categorisation

Map

Related Content

Refiners & Sorting

Results

with

Metadata

&

Imagery

Search

Navigation

Find

▪ Large Fortune 500 Pharma

– Auto tag metadata but spent manual time going through content sources to figure out what

could be leveraged as a rule to assign consistent metadata, then wrote scripts in alignment with

content sources to automate the rule building process

o Manpower plus cash and platform investment

▪ Large Fortune 500 Oil and Gas

– Leverage picture tagging to assign metadata in alignment with rig maintenance

o Primarily cash plus platform investment

▪ Large Top 50 Legal

– Auto tag via entity extraction

o Primarily cash and platform investment

Examples of automating tagging process (Manual is to PAINFUL!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!)

25

She no worka….

Language and Entity Extraction

28

Video Image and Audio with Timestamp

29

Facial Recognition

30

Signature Recognition

31

Sentiment Analysis

32

Language Detection

33

34

Where is the Intranet heading?

35

Ewe ☺

The 4 Must Dos

37

The missing concierge – Where do your searches/information requests fail?

High Value

– Manual

Tag

High Value

– Auto Tag

Low Value

– Manual

Tag

Low Value

– Auto Tag

High Value

– Multi

System

Search

High Value

– Single

Search

Low Value –

Multi

System

Search

Low Value –

Single

Search

Nearly every Search will need to add

1. Connectivity

Connectors to content & systems outside of O365

2. Structure

Information Architecture & Tags for navigation & findability

3. Context

Personalization through behavioural information

Connectors to Many Enterprise Systems

• Aderant

• Adobe Experience Manager

• Amazon Aurora

• Amazon RDS

• Amazon S3

• Alfresco

• Azure SQL

• Bentley - Assetwise

• Biomax

• Box

• Confluence

• CuadraSTAR

• Deltek Vision

• Elite / 3E

• File Shares

• Google Cloud

• Google Drive

• HP Consolidated Archive

• (EAS, aka Zantaz)

• HPE Records Manager/

HP TRIM

• IBM Connections

• IBM Content Manager

• IBM DB2

• IBM FileNet P8

• IBM Lotus Notes

• IBM WebSphere

• iManage Work

• Jive

• Kaltura

• LegalKEY

• LexisNexis Interaction

• Lotus Notes Databases

• MediaPlatform

• Microsoft Dynamics CRM

• Microsoft Exchange

• Exchange Public Folders

• Microsoft SQL Server

• MySQL

• NetDocuments

• Neudesic The Firm Directory

• Objective

• OneDrive

• Oracle Database

• OpenText Documentum

• OpenText eDOCS DM

• OpenText eRoom

• OpenText LiveLink/RM

• OpenText Media Manager

• Oracle WebCenter

• PLC/Practical Law

• PostgreSQL

• ProLaw

• Salesforce.com

• SAP ERP

• SAP HANA

• ServiceNow

• SharePoint Online

• SharePoint 2016

• SharePoint 2013

• SharePoint 2010

• SharePoint 2007

• Sitecore

• Slack

• Any SQL-based CRM system

• Veeva Vault

• Veritas Enterprise Vault

(Symantec eVault)

• Website Crawler

• West KM

• Workplace by Facebook

• Xerox DocuShare

• DocuShare

• Yammer

Plus a proven process for creating new connectors to complex systems

Only SharePoint and OneDrive Content Surfaces in Delve,

InfoPedia, etc. via the Office Graph

SAP / Oracle

OpenText/Filenet/

Documentum

Content from Other Systems Surfaces in Delve, InfoPedia, etc. via the Office Graph


1. Connectivity


2. Structure


3. Context

Personalization through behavioural information

AutoClassifierrules-driven core, combined with learning-based modules

Enhanced Content

Enriched with

Metadata and

Content Types

Search Visualization Workflow


1. Connectivity


2. Structure


3. Context

Personalization through behavioral information

45

SmartHub - Context equal personalization

The fourth must do for search success

1. Connectivity


2. Structure


3. Context


4. Measure


Agent Bots Cognitive Machine Learning Augmented RealityMicrosoft Cortana Bot Framework Cognitive Services

- Vision- Speech- Language- Knowledge- Search- Labs

Azure MLCognitive Toolkit

Hololens

Google Google Assistant API.AI Cloud Vision APICloud Video IntelligenceCloud Speech APICloud Natural Language APIGoogle Knowledge GraphGoogle Custom SearchML Advanced Solutions Lab

Cloud ML EngineTensorFlow

Glass at WorkDaydream/Tango

IBM Watson Virtual Agent Watson Conversation Watson APIs- Vision- Speech- Language- Data Insights

Watson ML ServiceApache SystemML

Amazon Alexa Amazon Lex RekognitionPolly

Amazon ML

Apple Siri SiriKitSpeech

Core ML ARKit

Facebook Facebook M ParlAI FastTextCommAI

Torch Oculus Rift

The Cognitive Arms Race

Today’s Session – Recap

▪

More examples

53

AutoClassifier


Searc

h I

nte

rface

Query

Engin

e

SearchIndex(es)

Machine Learning





Content

ContentProcessing

SearchIndex

/ Graph

Search UI

Enhancing Content with Cognitive Services

QueryProcessing

Content

Enrichment

AutoClassifierrules-driven core, combined with learning-based modules

Enhanced Content

Enriched with

Metadata and

Content Types


Adding Text Analytics to ContentUser

Uploads Content Event

Fires

Content Text

Analysis

Sentiment - Is text positive or negative?

Key Phrases - What are people discussing in a single article?

Topics - What are people discussing across many articles?

Language - What language is text written in?

Add metadata to item

Shows in O365 search

Adding Translation to ContentUser


Fires

Content Text

Analysis

Translation – Copy existing content and translate to specified language

save in SharePoint


Create new file/content for specific language

Image SearchUser


Fires

Computer Vision

Image Processing – recognize objects and text inside images


Add metadata to item

Extract image(s)

from documents

Image Classifier and Search

project libraries

and

issue detection


61

Where is the Intranet heading?

Search Provides A Unified View

Making O365 search intelligent using Azure Cognitive Services

more

ˇ

Start with understanding how it works

Matching IntentContent

Context

ContentEnrichment

UX

Connections

QueryAugmentation

Internet Sets the Bar for Enterprise Search, How do we make search happen on the intranet?

Insight Data Sets

Trending Documents

People I Work With

UsedDocuments

Email

Files

Contacts

Groups

Meetings

Org

Tasks

File Enrichments

ItemMetadata

# ActivityHistory

myGraph

Sites I Use

Suggested Sites

Usage Statistics

How does the search box work?

Used

Documents

Trending

Documents

Sites

I Use

Suggested

Sites

Beautiful

Bergen

Contoso

Norway

Kathrine

Hammervold

Metadata Drives Great User ExperiencesDocuments from many sourcesAll client or matter-relevant documents are integrated. Rich MetaData

Content annotated automatically – concepts,

categories, citations, matters, clients, etc

Navigation ControlsExplore, Discover, Drill-down

PersonalizationIt knows what to present to whom

Where is the Metadata in Delve?

So….I don’t need metadata, right?

Delve doesn’t have refinement,

so I don’t need metadata, right?

Chart courtesy of xkcd

AutoClassifier


Searc

h I

nte

rface

Query

Engin

e

SearchIndex(es)

Machine Learning





72

The End

▪

▪

▪

▪

Addendum 1- Chihuahuas for Precision and Recall

How quickly can we improve from 30%

▪

Chihuahuas for Precision and Recall

▪

Search for Chihuahua

▪

▪

▪

Provides these 4 as Results

How quickly can we improve to 90%

For More Information

▪ Check out resources at:

www.BAinsight.com

▪ Reach me via email at:

[email protected]

▪ Connect with me on LinkedIn at:

https://www.linkedin.com/in/donaldtmiller/

https://www.linkedin.com/in/donaldtmiller/

Intelligent Search: Connected, Search-Driven & Mobile · Don Miller –2 Rules and an Observation...

Documents

Transcript of Intelligent Search: Connected, Search-Driven & Mobile · Don Miller –2 Rules and an Observation...