1 MsSQLGirl.com | @MsSQLGirl

DirectQuery vs In-Memory in BISM Julie Koesmarno www.MsSQLGirl.com

Sydney SQL User Group @MsSQLGirl

2 MsSQLGirl.com | @MsSQLGirl

About Me

8+ years experience with SQL Server (since

SQL Server 2000)

MCITP 2008

Past projects: large scale 24/7 Data Integration,

Data Management and Data Cleansing system

Current project: Data Warehouse and Analysis

Services Tabular Model project

2

3 MsSQLGirl.com | @MsSQLGirl

Agenda

Terminology

DirectQuery vs In-Memory

Hybrid Mode

Out of Scope:

Sharepoint Integration

3

4 MsSQLGirl.com | @MsSQLGirl

Abstract

The new and flashy Tabular Model for Analysis Services has been highly pitched with the

in-memory (VertiPaq) capability allowing for complex queries to run very fast

What are the advantages and disadvantages of using in-memory cache to store and

query data?

Is DirectQuery the answer to address in-memory cache mode limitations?

What are the restrictions on DirectQuery tabular model?

When can a hybrid mode be used?

How would the selected mode can affect the Processing option during Deployment?

In this session, all the above questions will be answered with live demos, comparing the

two modes and the scenarios that are applicable to them. It also discusses how partition

plays an important role in DirectQuery mode and choosing the correct processing mode

when deploying a Tabular Model solution.

4 MsSQLGirl.com | @mssqlgirl

5 MsSQLGirl.com | @MsSQLGirl

v

Terminology

Tabular Model Architecture and Solution

6 MsSQLGirl.com | @MsSQLGirl 6

DirectQuery

Vertipaq

In-Memory

xVelocity Cached

7 MsSQLGirl.com | @MsSQLGirl

Tabular Model Architecture

7

Tabular Model

7

DirectQuery In-Memory

Third Party Application Excel Power View Reporting Services

SQL Server

Databases Non SQL Server

Databases

OData Files Cloud Services

8 MsSQLGirl.com | @MsSQLGirl

Tabular Model Architecture

8

Tabular Model

8

In-Memory

Third Party Application Excel Power View Reporting Services

SQL Server

Databases Non SQL Server

Databases

OData Files Cloud Services

9 MsSQLGirl.com | @MsSQLGirl

Tabular Model Architecture

9

Tabular Model

9

Third Party Application Power View Reporting Services

SQL Server

Databases

DirectQuery

10 MsSQLGirl.com | @MsSQLGirl

Tabular Model Solutions

10

Authoring

(SSDT) Deployment Option

Connectivity

Database

Properties

demo

11 MsSQLGirl.com | @MsSQLGirl

v

Execution

DirectQuery vs In-Memory

12 MsSQLGirl.com | @MsSQLGirl

Querying

12

Data Access

Query

Data Sources

12

DirectQuery

Process

In-Memory

DAX / MDX

VertiPaq SE

SQL

DAX

13 MsSQLGirl.com | @MsSQLGirl

DirectQuery Security

13 13

Role

DirectQuery Impersonation

Impersonate Current User

Default (Data Source login)

Data Source Impersonation

Windows User

Service Account

DirectQuery

SQL

Query

14 MsSQLGirl.com | @MsSQLGirl

In-Memory Security

14

Role

Row Level Security

Data Source Impersonation

14

VertiPaq SE

In-Memory

Process

Query

15 MsSQLGirl.com | @MsSQLGirl

Other Security Consideration?

Identity flow in Sharepoint 2010

http://www.sqlpass.org/summit/2011/Speakers/CallForSpeakers/SessionDetail.

aspx?sid=2027

15

BISM file

RSDS file

Simple Tabular Model Connection String

16 MsSQLGirl.com | @MsSQLGirl

v

Design

DirectQuery vs In-Memory

17 MsSQLGirl.com | @MsSQLGirl

Formula Compatibility

Semantic differences

xVelocity in-memory analytics engine (VertiPaq)

SQL Server

No Calculated Columns in DirectQuery

17

18 MsSQLGirl.com | @MsSQLGirl

DAX Functions

Some DAX Functions are not supported in

DirectQuery:

No equivalent calculations in relational engine

No equivalent SQL expressions

Performance of converted expressions

E.g. TOTALYTD, SAMEPERIODLASTYEAR

More information on BOL:

http://msdn.microsoft.com/en-us/library/hh213006.aspx

18

19 MsSQLGirl.com | @MsSQLGirl

v

Administration

DirectQuery vs In-Memory

20 MsSQLGirl.com | @MsSQLGirl

Processing Options (In-Memory)

Mode Database Table Partition

Process Default × × ×

Process Full × × ×

Process Data × ×

Process Clear × × ×

Process Defrag ×

Process Add ×

Process Recalc ×

20

Further reading:

http://msdn.microsoft.com/en-us/library/hh758414.aspx

21 MsSQLGirl.com | @MsSQLGirl

Partitioning in DirectQuery

One partition in Tabular Model

Can take advantage of Partitioning and ColumnStore

Index technology on SQL Server 2012

Partitioning / ColumnStore management done at the

source database level

21

22 MsSQLGirl.com | @MsSQLGirl

Partitioning in In-Memory

In-Memory can have multiple mutually exclusive

partitions

To eliminate unnecessary processing and

processor load on the AS servers

22

C u r r e n t

Mar 2012

Jan 2012

Feb 2012

…

In-Memory

Partitions Process

23 MsSQLGirl.com | @MsSQLGirl

v

Pros and Cons

DirectQuery vs In-Memory

24 MsSQLGirl.com | @MsSQLGirl

In-Memory - Pros

ALL functionality of Tabular Model

Calculated Columns, ALL DAX functions

Row Level security

xVelocity in-memory analytics engine

Many choices of client tool

Excel, Power View, SSRS, MDX client*

Many supported data sources

24

25 MsSQLGirl.com | @MsSQLGirl

In-Memory - Cons

AS requires Memory & CPU resources

Caching

Processing

Querying

Diligent Partitioning and Processing

Out of date data

25

26 MsSQLGirl.com | @MsSQLGirl

DirectQuery - Pros

Real time access

Scalable

SQL Server permission

SQL Server optimisation

Consistency

One Partition to manage

No processing required

26

27 MsSQLGirl.com | @MsSQLGirl

DirectQuery - Cons

Restricted DAX functions

Restricted client tools: PowerView, SSRS

No Row Level Security

Unless implemented at the Source Database level

No Calculated Columns

27

28 MsSQLGirl.com | @MsSQLGirl

v

Hybrid Mode

The best of both worlds

29 MsSQLGirl.com | @MsSQLGirl

Tabular Model Solutions

29

DirectQuery Mode

On

Off

Connection String:

DirectQueryMode=DirectQuery

DirectQueryMode=In-Memory

Query Mode

DirectQuery

DirectQuery with In-Memory

In-Memory with DirectQuery

In-Memory

30 MsSQLGirl.com | @MsSQLGirl

Which Hybrid Mode?

Query Mode Excel Connection String

DirectQuery with In-Memory DirectQueryMode = InMemory

In-Memory with DirectQuery No need to specify DirectQueryMode

30

PowerView always uses the default Query Mode

(i.e. the first bit of the Query Mode) – can’t

change this!

SSRS works with both Query Mode and

Connection String combinations

31 MsSQLGirl.com | @MsSQLGirl

Partitioning for Hybrid Mode

One Partition for DirectQuery

Processing Option: Never process this partition

Multiple Partitions for In-Memory (No Overlap)

31

…

C u r r e n t

Aug 2012

Jun 2012

Jul 2012 In-Memory

Partitions

DirectQuery

Partitions

32 MsSQLGirl.com | @MsSQLGirl

Implications

Inconsistency

Data – stale data in Cache

Calculations – semantic differences xVelocity vs SQL

Security

DirectQuery Design

Flexibility at run time / client tools

In-Memory (Excel) or DirectQuery (Real time)

http://www.mssqlgirl.com/hybrid-mode-in-tabular-bi-semantic-model-part-1.html

http://www.mssqlgirl.com/hybrid-mode-in-tabular-bi-semantic-model-part-2.html

32

33 MsSQLGirl.com | @MsSQLGirl

v

Wrap Up

34 MsSQLGirl.com | @MsSQLGirl

Tabular Model Modes

In-Memory Mode (default)

DirectQuery Mode

DirectQuery only

DirectQuery with In-Memory

In-Memory with DirectQuery

34

35 MsSQLGirl.com | @MsSQLGirl

Do I DirectQuery?

SQL Server database is optimised

Columnstore Index

Partitioning

Contains all required business rules

High Volume

Can import non-SQL data into one SQL Server

database which will be the source

35

36 MsSQLGirl.com | @MsSQLGirl

Further Reading

Cathy Dumas’ blog

http://cathydumas.com/

SQLBI – (Marco Russo & Alberto Ferrari)

http://www.sqlbi.com/

DAX resources (with links to Tabular Model)

http://social.technet.microsoft.com/wiki/contents/articles/1088.dax-resource-center.aspx

Julie Koesmarno’s blog

http://www.MsSQLGirl.com

36

37 MsSQLGirl.com | @MsSQLGirl

Q & A

37

signal@mssqlgirl.com

http://www.mssqlgirl.com

http://au.linkedin.com/in/juliekoesmarno

@mssqlgirl

38 MsSQLGirl.com | @MsSQLGirl

v

Thank you!

39 MsSQLGirl.com | @MsSQLGirl 39 MsSQLGirl.com |

@mssqlgirl