SSAS_HardwareSizingTabularSolutions

7/26/2019 SSAS_HardwareSizingTabularSolutions

1/33

Hardware Sizing a TabularSolution (SQL Server AnalysisServices)

SQL Server Technical Article

Authors: John Sirmon, Heidi Steen

Contributors and Technical Reviewers:Karen Aleksanyan, Greg Galloway, Darren

Gosbell, Karan Gulati, Chris Kurt, Teo Lachev, Greg Low, Akshai Mirchandani, BoyanPenev, he! he!!ard, Chris Testa"#$%eill

Published:January 2013

Applies to:SQL Server 2012 Analysis Services, Tabular Solutions

Summary:Provides guidance &or esti'ating the hardware re(uire'ents needed to

su!!ort !rocessing and (uery workloads &or an Analysis ervices tabular solution)


2/33

Co!yright

This docu'ent is !rovided *as"is+) n&or'ation and views e-!ressed in this

docu'ent, including ./L and other nternet 0eb site re&erences, 'ay change

without notice) 1ou bear the risk o& using it)

o'e e-a'!les de!icted herein are !rovided &or illustration only and are 2ctitious)

%o real association or connection is intended or should be in&erred)

This docu'ent does not !rovide you with any legal rights to any intellectual

!ro!erty in any Microso&t !roduct) 1ou 'ay co!y and use this docu'ent &or your

internal, re&erence !ur!oses)

3 4567 Microso&t) All rights reserved)

4


3/33

Contentsntroduction)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) 8

9ardware Considerations &or a Develo!'ent :nviron'ent))))))))))))))))))))))))))))))))))))))));

Me'ory))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

CP.))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) 6>

Cores))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) 45

#ther considerations))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))45

#nboard Cache L6, L4)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))46

%.MA)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) 44

Monitor CP. .sage))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))44

CP. usage during (uery e-ecution))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))47

Key Points))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) 48Disk #))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) 4;

9ardware Con2guration :-a'!les)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))4

Conclusion)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) 4


10/33

Maximum database size

&here are no theoretical limits on the physical si!e of the

dataase, e'cept those imposed y the system, which must e

ale to store all data dictionaries, column segments (this is the

storage unit for taular dataases), caches, and the formula

engine in memory. &he system will page uncompressed data

to disk during processing, ut you should not count on it for

normal server operations.

Calculate 'e%ory "or #atabases at &estAs a 2rst ste!, you will need to calculate the a'ount o& /AM you will need to si'!ly

store the database, irres!ective o& the !rocessing and (uery o!erations that will

co'e later)

or databases that are already built, you can calculate database siEe in 'e'ory

using any o& the &ollowing techni(uesH

/un a DMI (uery that re!orts on database siEe

/ead the database 2le siEe on disk

Iiew esti'ated database siEe in Manage'ent tudio as 'entioned in the

!revious section

or !ro?ects at a 'ore !reli'inary stage, including !ro?ects with data siEes that

e-ceed the ca!acity o& develo!'ent 'achines, you will need to base your esti'ateon the unco'!ressed dataset that you are 'odeling, or build a !rototy!e that uses

a subset o& your data)

Many custo'ers routinely use PowerPivot &or :-cel as a tool &or testing how 'uch

co'!ression they$ll get &or their raw data) Although PowerPivot uses a slightly

diFerent co'!ression algorith', the co'!ression engine is the sa'e, allowing you

to arrive at reasonable esti'ate when !rototy!ing with the PowerPivot add"in)

PowerPivot add"in is available in 74"bit and 8"bit versions) The 8"bit version

su!!orts 'uch bigger 'odels) Try to use that version i& you can)

#ther custo'ers who work with large datasets usually a!!ly 2lters during i'!ort toselect a subset o& data &or e-a'!le, 2ltering on one day$s worth o& transactions)

This a!!roach gives you a s'aller and 'ore 'anageable dataset, while retaining

the ability to e-tra!olate a realistic esti'ate o& the 2nal dataset all things being

e(ual, i& one day$s dataset is 45 MB, a 'onth is roughly 55 MB) 0hen choosing a

2lter, however, it is i'!ortant to 'ake sure that the subset o& the data is still

re!resentative o& the overall data) or e-a'!le, it 'ight 'ake 'ore sense to 2lter

by date than by city or state)

65
http://office.microsoft.com/en-us/excel-help/data-model-specification-and-limits-HA102837464.aspxhttp://office.microsoft.com/en-us/excel-help/data-model-specification-and-limits-HA102837464.aspx


11/33

& you are contending with tabular 'odel de!loy'ent on a syste' that has little

/AM to s!are, you can o!ti'iEe your 'odel to reduce its 'e'ory &oot!rint)

Co''on techni(ues include o'itting high cardinality colu'ns that are not

necessary in the 'odel) Another tradeoF that a solution architect will consider is

using a 'easure in lieu o& a calculated colu'n) A calculated colu'n is evaluated

during !rocessing and !ersisted to 'e'ory) As a result, it !er&or's well during(uery e-ecution) Contrast that with a 'easure that !rovides e(uivalent data, but is

generated during (uery e-ecution and e-ists only until evicted &ro' cache) The

(uery runs slower due to e-tra co'!utations, but the bene2t is a reduction in

!ersistent storage used on an ongoing basis)

'e%ory &e.uire%ents "or #isaster &ecovery0hen esti'ating 'e'ory usage, re'e'ber to budget &or database restore to allow

&or situations where the database beco'es corru!ted or unusable) 9aving

insucient 'e'ory to !er&or' this o!eration will li'it your recovery o!tions

considerably, !erha!s re(uiring you to detachother databases that you$d rather

kee! online) deally, you should have enough /AM to restore and test a backu!

be&ore re'oving the one that is being re!laced) & the backu! is corru!t, unusable,

or too stale, you 'ight want to salvage what you can &ro' the original database

be&ore deleting it co'!letely)

9aving enough 'e'ory &or restore e(uates to about 4- the siEe o& the database,

about the sa'e a'ount you would need &or &ull !rocessing) :sti'ating hardware

resources &or disaster recovery does not 'ean you need an additional 4- /AM on

to! o& that already budgeted &or !rocessingJ it ?ust gives you one 'ore reason to

have /AM that is 4"7- siEe o& your solution)

An alternative a!!roach that others have &ound eFective is to use IMs) 1ou can

(uickly reload a !revious virtual 'achine sna!shot to restore data access to a

tabular solution)

'e%ory &e.uire%ents "or rogra% /ecutionAnalysis ervices reserves a!!ro-i'ately 6;5 'egabytes MB o& 'e'ory &or

!rogra' e-ecution, or'ula :ngine, torage :ngine, shared ob?ects, and shared

cache) An Analysis ervices instance that has no databases will use 6;5 MB o& /AM

on the host co'!uter)

n addition, data dictionaries are always locked in 'e'ory, never !aged to disk

only seg'ent data can be !aged to disk, and then only when the syste' is under

'e'ory !ressure) The data dictionaries o& all databases running on a tabular 'ode

server will load into 'e'ory on service start u!)

66
http://msdn.microsoft.com/en-us/library/cc280583.aspxhttp://msdn.microsoft.com/en-us/library/cc280583.aspx


12/33

sti%ating 'e%ory "or rocessingProcessing, which includes reading, encoding, and loading co'!ressed data into

tabular 'odel ob?ects, is the 2rst resource"intensive workload you$ll need to

consider) Processing is a 'e'ory"intensive o!eration, re(uiring at least two ti'es

the a'ount o& 'e'ory re(uired to store the database when it$s 2nally !rocessed

and de!loyed) Processing uses 'e'ory &or storing both co'!ressed andunco'!ressed data while it$s being read and encoded)

or e-ce!tionally large databases, you can oOoad !rocessing to another 'achine,

and then synchroniEethe &ully !rocessed database to a !roduction server) & you

re&er back to the server 'a! illustration, this a!!roach saves you the 'e'ory

re(uired &or hosting the current database used &or servicing (ueries while the new

database is built in the background) Although this a!!roach to !rocessing can lower

your resource re(uire'ents considerably, &or the !ur!oses o& this guide, we$ll

assu'e you are !rocessing the database on the !roduction server)

The 'ost co''on a!!roach &or esti'ating 'e'ory usage &or !rocessing is basedon a &actor o& database siEe at Eero load) Me'ory re(uire'ents will vary de!ending

on how well data is co'!ressed) %otably, co'!ression is greater &or colu'ns that

store re!eating values, whereas colu'ns containing distinct values do not co'!ress

well)

The techni(ue we reco''end &or getting !reli'inary esti'ates is to use a si'!le

&or'ula &or calculating co'!ression and !rocessing re(uire'ents) n s!ite o& its

li'itations, using a &or'ula to is a reasonable !lace to start i& you re(uire a ball!ark

2gure) To i'!rove u!on your esti'ate, you can !rototy!e the 'odel you !lan to

de!loy using a re!resentative sa'!le o& the larger datasetH

.se a &or'ula to get an initial esti'ate)

Build a !rototy!e to re2ne the esti'ate)

Alternatively, i'!ort a subset o& rows &ro' each table and then 'ulti!ly to get an

esti'ated siEe)

0se a "or%ula to get an initial esti%ate

The &ollowing &or'ula !rovides an initial esti'ate o& the 'e'ory you$ll need to

!rocess and store all o& your data in local 'e'ory)

iEe o& unco'!ressed dataQ 65 R 4

.nco'!ressed data is the siEe o& the database 2le, assu'ing you$re i'!orting all or

'ost o& the database into a tabular solution)

64
http://msdn.microsoft.com/en-us/library/ms174928(v=sql.110).aspxhttp://msdn.microsoft.com/en-us/library/ms174928(v=sql.110).aspx


13/33

or a @L erver database, you can run s!Ss!aceusedto return siEe in&or'ation &or

each table) The &ollowing screenshot !rovides an illustration using the

act#nlineales table, which 'easures around 74 'egabytes)

Alternatively, to get overall database siEe, o'it the table na'e and e-ecute

s$1s$aceusedas &ollowsH

Co%$osition o" t*e "or%ula

Dividing unco'!ressed data by ten gives you an esti'ate o& 'odel siEe a&ter the

data is co'!ressed) This is how 'uch 'e'ory the 'odel will consu'e once it is

de!loyed to the server)

The deno'inator 65 was arrived at by averaging the actual co'!ression rates

over a wide range o& data sets) or your solution, actual co'!ression 'ight be

'uch lower 4 or 7 or 'uch higher 655-, de!ending on data ty!e, length, and

cardinality) Tables or colu'ns containing 'ostly uni(ue data values such as &act

tables will e-!erience 'ini'al co'!ression, while other tables 'ight co'!ress by

a lot) or esti'ation !ur!oses, ten is o&ten cited as an acce!table, 'iddle o& the

road esti'ate o& overall co'!ression &or a database that includes a balance o&

uni(ue and non"uni(ue values)

The second !art o& the &or'ula asks you to double the a'ount o& 'e'ory used by

the database) Doubling 'e'ory accounts &or !rocessing o!erations, as the server

kee!s a second co!y o& the database in 'e'ory to service (uery re(uests while

!rocessing runs in the background)

67

Use ContosoRetailDW;

Go

Exec sp_spaceused;

Go
http://msdn.microsoft.com/en-us/library/ms188776.aspxhttp://msdn.microsoft.com/en-us/library/ms188776.aspx


14/33

Me'ory used to store the database co!y is i''ediately released a&ter !rocessing

concludes, but that doesn$t 'ean 'e'ory usage di'inishes to ?ust the /AM needed

to store the database) @ueries and torage :ngine caches also consu'e 'e'ory)

#n the server, a (uery devolves into table scans that select and aggregate data,

calculations, and other o!erations)

$s you investigate memory usage y taular solutions, you

might come across other formulas that are more roust

mathematically, yet harder to use if you are not deeply familiar

with the data. In the end, you might *nd it more productive to

use a simplistic formula for an initial estimate, and then move

on to prototype a solution using a suset of your own data.

'easure %e%ory used by individual ob2ectsAnalysis ervices !rovides Dyna'ic Manage'ent Iiews DMI that show

'etadata and syste' in&or'ation, !articularly 'e'ory usage, &or 'odel ob?ects)

0hile not strictly a hardware siEing e-ercise, knowing which ob?ects take u! the

'ost 'e'ory gives you the o!!ortunity to identi&y and !ossibly re'ove any

resource"intensive ob?ects you don$t actually need)

6) #!en an MD (uery window in @L erver Manage'ent tudio, connecting to

the tabular instance running your database)4) :nter the &ollowing DMI co''and, elect R &ro'

Uyste')discoverSob?ectS'e'orySusage, and click /ecute) 1ou can orderby nonshrinkable 'e'ory to see which colu'ns in this case, the alesKey in

the actalesTable are using the 'ost 'e'ory)

0hile running a DMI is acce!table, we reco''end that you download and use a

workbook created and !ublished by Kas!er De Vonge, an Analysis ervices !rogra'

'anager) 9is workbook uses DMI (ueries to re!ort 'e'ory usage by ob?ect, but

i'!roves u!on the raw DMI by organiEing and !resenting the results in a hierarchy

that lets you drill down into the details)

68
http://www.powerpivotblog.nl/what-is-using-all-that-memory-on-my-analysis-server-instancehttp://www.powerpivotblog.nl/what-is-using-all-that-memory-on-my-analysis-server-instancehttp://www.powerpivotblog.nl/what-is-using-all-that-memory-on-my-analysis-server-instancehttp://www.powerpivotblog.nl/what-is-using-all-that-memory-on-my-analysis-server-instance


15/33

Best o& all, the workbook re!orts on database siEe)

&e3ne t*e esti%ate by building a $rototy$e t*at yields a better value "or

data co%$ression

As we work our way u! the continuu' o& esti'ation techni(ues, we arrive at one o&

the 'ore robust a!!roachesH !rototy!ing using your own data)

The best way to deter'ine how well your data co'!resses in a tabular solution is

to start with so'e initial data i'!orts) Because the ob?ective is to understand

co'!ression behavior, treat this as a !rototy!ing e-ercise) & you were building a

'odel you !lanned to kee!, you would s!end ti'e thinking about 'odel design) orour !ur!oses, you can set those !roble's aside and &ocus si'!ly on choosing which

tables and colu'ns to i'!ort &or the !ur!ose o& esti'ating database siEe)

The &ollowing ste!s a!!roach !rototy!ing &ro' the stand!oint o& large datasets) &

your dataset is not large, you can ?ust run rocess ull) #therwise, !rocess ?ust

one table at ti'e, and then run rocess &ecalcat the end to !rocess table

de!endencies)

6) Create a new tabular !ro?ect using @L erver Data Tools)4) '!ort the largest table &ro' your e-ternal data source into a tabular 'odel)

& it$s a &act table, e-clude any colu'ns that are not needed in the 'odel).sually, the !ri'ary key o& a &act table is a good candidate &or e-clusion, as

are colu'ns that are only re(uired &or :TL !rocessing)7) Process the table and de!loy it to a develo!'ent server)8) & de!loy'ent succeeds, 'easure the siEe o& the co'!ressed table in

'e'ory) & de!loy'ent &ails, a!!ly a 2lter to get a s'aller rowset, while

ensuring that the 2ltered rowset is still re!resentative o& the overall table)

6;


16/33

;) '!ort and !rocess a second table, de!loy the solution, 'easure 'e'ory

usage, and then re!eat with additional tables)) to! when you have sucient dataset re!resentation in your 'odel)


17/33

DA (uery o!ti'iEation is beyond the sco!e o& this guide, but other sources are

available that cover this 'aterial) ee the links at the end o& this docu'ent &or

'ore in&or'ation)

About Concurrency Testing0hen you have a large nu'ber o& users re(uesting data &ro' a 'odel, 'e'ory

usage will cli'b accordingly, but the a'ount re(uired will vary de!ending on the

(uery itsel&) & you have 655 users accessing a re!ort that issues an identical (uery

each ti'e, the (uery results are likely to be cached and the incre'ental 'e'ory

usage will be li'ited to ?ust that result set) #& course, i& each user a!!lies a 2lter to

that (uery, it$s the e(uivalent o& having 655 uni(ue (ueries, a vast diFerence in

'e'ory consu'!tion)

To truly understand the i'!act o& a large nu'ber o& re(uests, you will need to load

test your solution) Iisual tudio includes a load testing 'odule that lets yousi'ulate 'ulti!le user re(uests at varying intervals &or diFerent client a!!lications)

The con2guration o& load tests using this &acility is beyond the sco!e o& this guide,

but worth 'entioning due to its relevance to hardware !rovisioning) erious load

testing re(uires a !ro&essional toolset) t$s not si'!le to do, but Iisual tudio oFers

one o& the better a!!roaches &or testing a server under load) or 'ore in&or'ation,

see 0orking with Load Test Iisual tudio Tea' :dition, Can your B solution scale[,

and Load Testing Analysis ervices)

'onitor %e%ory usage during $rocessing and .ueryingPer&or'ance Monitor is an o&t"used tool &or understanding 'e'ory usage and

trends at the o!erating syste' level) 1ou$ll use it regularly to understand 'e'ory

allocation and release !atterns &or your tabular solutions)

The &ollowing ste!s will get you started with 'onitoring /AM usage in Per&or'ance

MonitorH

6) solate a database at the instance level by detaching other databases so that

an instance is running only the database we want to look at) 1ou can detach

databases using Manage'ent tudio)

4) n Per&or'ance Monitor, select the tabular instance on the syste' shown inthe screenshot, a tabular server is installed as na'ed instance, as

localhostXtabular)

6


18/33

7) :-!and 'SOLA4TA50LA&!''O&6and select 'e%ory 0sage -5,

7ertia. Non$aged -5, and 7ertia. aged -5)

8) :-ecute a rocess ullagainst the database in Manage'ent tudio to

understand how 'e'ory is used)

The &ollowing screen ca!ture shows a ty!ical !attern o& increasing 'e'ory

consu'!tion the counter shown is &or 'e%ory 0sage -5, leveling out at

'a-i'u' allowed =5N /AM by de&ault, with 'e'ory released as !rocessing

concludes)

6=


19/33

%e-t, issue (ueries against the database to understand the (uery !ro2le o& your

client a!!lication)

& the (uery can be e-ecuted against the co'!ressed data, you won$t see a

noticeable diFerence in 'e'ory usage on the server) #therwise, you$ll see a

transient surge in 'e'ory usage as te'!orary tables are created to store and scanunco'!ressed data)

Although 'e'ory s!ikes during (uery e-ecution are s!oradic, the sa'e cannot be

said &or client a!!lications) Client a!!lications will 'ost certainly consu'e 'e'ory

as data is retrieved &ro' the tabular 'odel) As !art o& your investigation, consider

adding client !rocesses to the trace to 'onitor 'e'ory usage)

6) #n the client co'!uter, start the client a!!lications used to (uery the 'odel)4) n Per&or'ance Monitor start a new trace)7) Add counters &or client !rocesses) Client !rocesses are listed under the

rocessob?ect in Per&or'ance Monitor)

a) or Manage'ent tudio, select ss%snot to be con&used with s%ss)b) or :-cel, select e/cel)c) or Power Iiew in harePoint, hosting is in one the harePoint ervice

A!!lication A!!Pool !rocesses usually w7w!)e-e)

-ey ointsA tabular solution uses 'e'ory during !rocessing, when loading 'etadata a&ter a

service restart, and when loading re'aining data on the 2rst (uery issued against

the 'odel) #nce a tabular database is loaded, 'e'ory usage re'ains relatively

stable unless A needs to build te'!orary tables during (uery e-ecution)

6>


20/33

/elative to !rocessing and basic storage re(uire'ents, 'e'ory used &or (ueries

tends to be 'ini'al) & you$re 'onitoring a (uery workload in Per&or'ance Monitor,

you$ll notice that 'e'ory usage is o&ten \at &or 'any (ueries)

A (uery can result in a te'!orary but radical u!tick in 'e'ory usage i& a colu'n or

table needs to be deco'!ressed during (uery e-ecution) Certain &unctions such as:A/L:T, .M, and LT:/ are known to have !er&or'ance i'!act) Always test

the (ueries and re!orts you !lan to run in a !roduction environ'ent to understand

their !er&or'ance !ro2le)

0hen calculating 'e'ory usage on a develo!'ent 'achine, any works!ace

databases loaded in 'e'ory will skew your 'easure'ents) Be sure to unload the

works!ace database when collecting 'etrics about 'e'ory usage)

inally, re'e'ber to 'onitor 'e'ory used by client a!!lications) A (uery 'ight be

trivial &or the torage :ngine, yet bring a client workstation to its knees i& a 'assive

a'ount o& data is read into its 'e'ory)

'e%ory 5andwidt* and S$eed

Moving &ro' a disk"bound architecture to 'e'ory"bound architecture shi&ts

!er&or'ance bottlenecks &ro' disk access to 'e'ory access) Vust like you needed

&ast disks &or 'ultidi'ensional 'odels, you$ll need &ast 'e'ory to get the

!er&or'ance you want &ro' o& a tabular solution)

As o& this writing, we$re still testing 'e'ory subsyste' &eatures to understand

where the tradeoFs are) @uestions under investigation include identi&ying changes

in (uery !er&or'ance when adding 'ore integrated 'e'ory controllers versus

adding 'ore /AM) n a &uture u!date to this !a!er, we !lan to share our 2ndings to

these and other (uestions related to 'e'ory !er&or'ance)

0hat we do know at this !oint is 'e'ory bandwidth beco'es increasingly

i'!ortant as you add cores) The 'ore cores you have, the greater the nu'ber o&

threads re(uesting read o!erations &ro' 'e'ory) Many server workloads involve

scanning large tables with little !rocessingJ having sucient 'e'ory bandwidth

lets 'ore o& those re(uests get through at one ti'e)

Bus s!eed is also crucial) n a &ew isolated cases, high"end workstations that have a

shorter &aster bus have been known to out!er&or' business class servers)

n the ne-t section, we$ll discuss the i'!ortance o& onboard cache) 9owever,

regardless o& how 'uch onboard cache you have, reading data &ro' /AM is going to

ha!!en regularly) yste's that have high !er&or'ance 'e'ory and e-tra

bandwidth are better e(ui!!ed to deal with the technical challenges o& hosting an

45


21/33

in"'e'ory solution) As you evaluate diFerent hardware syste's, look &or syste's

that oFer better than average 'e'ory !er&or'ance and integrated 'e'ory

controllers)

-ey oints

Tabular solution architectures are o!ti'iEed &or (uery !er&or'ance, !redicated on/AM storage being 'uch &aster to read &ro' than disk) 0hen evaluating /AM,

consider 'e'ory designs that balance through!ut and s!eed)

C0

electing a &ast CP. with a sucient nu'ber o& cores is also a to! consideration) n

a tabular solution, CP. utiliEation is greatest when (uery evaluation is !ushed to the

torage :ngine) n contrast, CP. bottlenecks are 'ore likely to occur when a (uery

or calculation is !ushed to the single"threaded or'ula :ngine) Because each (ueryis single"threaded in the or'ula :ngine, on a 'ulti"core syste', you 'ight see one

!rocessor at 'a-i'u' utiliEation while others re'ain idle)

Constructing (ueries that only run in the torage :ngine 'ight sound te'!ting, but

it$s unrealistic as a design goal) & the !oint o& your 'odel is to !rovide insights that

solve business !roble's, you$ll need to !rovide (ueries and e-!ressions that 'eet

business goals, irres!ective o& (uery e-ecution 'echanics) urther'ore, i& you$re

!rovisioning a server that hosts sel&"service B solutions built and !ublished by other

!eo!le, (uery synta- construction is !robably beyond your control)

A CP. with clock s!eeds o& at least 4)= to 7 G9E is your best insurance against(ueries that e-ecute single"threaded in the or'ula :ngine, slow (ueries that are

dicult o!ti'iEe, or subo!ti'al (uery synta- created by novice 'odel designers)

Query Load on C0

n so'e cases, (ueries against a tabular 'odel can o&ten be !ushed down to the

'ulti"threaded -Ielocity torage :ngine, where each (uery ?ob runs on a se!arate

core) De!ending on the siEe o& the dataset and the (uery itsel&, you 'ight see

greater CP. utiliEation, which usually e(uates to better !er&or'ance)

@ueries 'ost likely to run in the torage :ngine are based on si'!le 'easures that

can be calculated without having to deco'!ress the data 2rst) & a colu'n needs to

be deco'!ressed, such as when co'!uting a rank order o& all values, a te'!orary

table is created in 'e'ory to store and scan the values) This o!eration consu'es

'e'ory and CP., es!ecially i& the calculation needs to be handled by the or'ula

:ngine)

46


22/33

@ueries that run only in the or'ula :ngine, such as an evaluation o& a .M or

ilter o!eration, are single"threaded and a co''on (uery !er&or'ance bottleneck)

LT:/ iterates over the entire table to deter'ine which rows to return)

8ect o" Concurrent Queries on C0

The nu'ber o& clients re(uesting data &ro' the 'odel will also &actor heavily into

how 'uch CP. resource you$ll need) As !reviously noted, each (uery 'oves

through the or'ula :ngine as a single"threaded o!eration) & you have 655 uni(ue

(ueries running si'ultaneously, you$ll want signi2cantly 'ore cores to handle the

load)

rocessing Load on C0

Processing can take a long ti'e to co'!lete, but is ty!ically not considered to be

CP. intensive) :ach !rocessing ?ob uses one to two coresJ one core to read the

data, and another core &or encoding) Given that each !artition within one table 'ust

be !rocessed se(uentially, the !attern o& a !rocessing o!eration tends to be a s'all

nu'ber o& cores sustained over a longer !eriod o& ti'e) 9owever, i& !rocessing

'any tables in !arallel, CP. usage can rise)

Cores%ow that you have a basic understanding o& how CP. resources are used, let$s

'ove on to s!eci2c CP. designs 'ost o&ten used to su!!ort 'ediu' to large

solutions)

or tabular solutions, the 'ost &re(uently cited CP. designs range &ro' = to 6

cores) Per&or'ance a!!ears to be better on syste's that have &ewer sockets) or

e-a'!le, 4 sockets with = &ast cores, as o!!osed to 8 sockets with 8 cores) /ecall

&ro' the !revious section the i'!ortance o& 'e'ory bandwidth and s!eed in data

access) & each socket has its own 'e'ory controller, then in theory, we should

e-!ect that using 'ore cores !er socket oFers better !er&or'ance than 'ore

sockets with &ewer cores)

:(ually i'!ortant, !er&or'ance gains tend to level oF when you e-ceed 6 cores)

Per&or'ance doesn$t degrade as you add 'ore coresJ it ?ust &ails to !roduce the

sa'e !ercentage increase that you achieved !reviously) This behavior is not s!eci2cto tabular solutionsJ si'ilar outco'es will be encountered when de!loying any

'e'ory intensive a!!lication on a large 'ulti"core syste')

The !roble' is that 'e'ory allocations &all behind relative to the threads 'aking

'e'ory re(uests) Contention arises as all cores to read and write to the sa'e

shared resource) #!erations beco'e serialiEed, eFectively slowing down server

!er&or'ance) The end result is that a CP. 'ight be at 75"85N utiliEation yet unable

44
http://www.powerpivotblog.nl/tune-your-powerpivot-dax-query-dont-use-the-entire-table-in-a-filter-and-replace-sumx-if-possiblehttp://www.powerpivotblog.nl/tune-your-powerpivot-dax-query-dont-use-the-entire-table-in-a-filter-and-replace-sumx-if-possiblehttp://www.powerpivotblog.nl/tune-your-powerpivot-dax-query-dont-use-the-entire-table-in-a-filter-and-replace-sumx-if-possiblehttp://www.powerpivotblog.nl/tune-your-powerpivot-dax-query-dont-use-the-entire-table-in-a-filter-and-replace-sumx-if-possible


23/33

to !er&or' any additional !rocessing due to bottlenecks in the 'e'ory subsyste')

Cores are idled while waiting &or 'e'ory allocations to catch u!) or 'ore

in&or'ation about this behavior, see Detecting Me'ory Bandwidth aturation in

Threaded A!!lications)

Ot*er considerations0hen evaluating the nu'ber o& cores, consider associated so&tware costs and li'its

that increase with the core count) n 'ost cases, so&tware licensing &ees go u! with

the nu'ber o& cores)

Software licensing fees

o&tware licensing &ees vary based on the nu'ber o& cores used by @L erver) As

o& this writing, the use o& 'ore than 45 cores re(uires an :nter!rise edition and a

!er"core or volu'e licensing o!tion) ee Co'!ute ca!acity li'its by editionto

deter'ine the 'a-i'u' nu'ber o& cores su!!orted by the @L erver edition

you$re using)

Alternatively, you can use the Business ntelligence edition and erver CALs to

license by the nu'ber o& !eo!le using the server) Check the Microso&t licensing web

site&or an e-!lanation o& B edition licensing)

+e aware that running multiple S Server features on the

same hardware is known to slow system performance as the

operating system and various services compete for the same

resources. +efore installing multiple S Server features on the

same machine, take a look at a server memory calculator that

S CSS created. It can give you an idea of how well your

machine supports the relational engine and operating system.

&he calculator does not account for $nalysis Services, ut if

you are installing multiple S Server feature components,

youll at least understand how to con*gure the system to

provide ade"uate #$% for the relational engine and operating

system.

Virtual machine limits and licenses

Be aware that using virtual 'achines will i'!ose new li'its on the nu'ber o&logical cores you can use) n 9y!er"I on 0indows erver 4564, 'a-i'u' /AM is 4

TB and 'a-i'u' virtual cores is 8) :arlier versions o& 0indows bring that

'a-i'u' down to 8 cores) or 'ore in&or'ation, see 'a-i'u' virtual !rocessor

in&or'ation &or 0indows erver 4564and earlier versions) #ther virtualiEation

vendors o!erate under si'ilar constraintsJ i& you are using a diFerent IM

technology check the !roduct web site &or 'a-i'u' li'its)

47
http://software.intel.com/en-us/articles/detecting-memory-bandwidth-saturation-in-threaded-applicationshttp://software.intel.com/en-us/articles/detecting-memory-bandwidth-saturation-in-threaded-applicationshttp://msdn.microsoft.com/en-us/library/ms143760.aspxhttp://www.microsoft.com/licensing/about-licensing/sql2012.aspx#tab=3http://www.microsoft.com/licensing/about-licensing/sql2012.aspx#tab=3http://blogs.msdn.com/b/sqlsakthi/archive/2012/05/19/cool-now-we-have-a-calculator-for-finding-out-a-max-server-memory-value.aspxhttp://blogs.msdn.com/b/sqlsakthi/archive/2012/05/19/cool-now-we-have-a-calculator-for-finding-out-a-max-server-memory-value.aspxhttp://technet.microsoft.com/en-us/library/hh833682(d=lightweight,v=ws.11)http://technet.microsoft.com/en-us/library/cc794868(v=WS.10).aspxhttp://software.intel.com/en-us/articles/detecting-memory-bandwidth-saturation-in-threaded-applicationshttp://software.intel.com/en-us/articles/detecting-memory-bandwidth-saturation-in-threaded-applicationshttp://msdn.microsoft.com/en-us/library/ms143760.aspxhttp://www.microsoft.com/licensing/about-licensing/sql2012.aspx#tab=3http://www.microsoft.com/licensing/about-licensing/sql2012.aspx#tab=3http://blogs.msdn.com/b/sqlsakthi/archive/2012/05/19/cool-now-we-have-a-calculator-for-finding-out-a-max-server-memory-value.aspxhttp://blogs.msdn.com/b/sqlsakthi/archive/2012/05/19/cool-now-we-have-a-calculator-for-finding-out-a-max-server-memory-value.aspxhttp://technet.microsoft.com/en-us/library/hh833682(d=lightweight,v=ws.11)http://technet.microsoft.com/en-us/library/cc794868(v=WS.10).aspx


24/33

Onboard Cac*e (L9: L;)Because onboard cache is so 'uch &aster than /AM, syste's that oFer

!ro!ortionally 'ore L6 and L4 cache are better &or a tabular data access)

n @L erver 4564 P6, Analysis ervices added cache o!ti'iEations to s!eed u!

!er&or'ance) To use the onboard cache to its best advantage, data structures

!roviding (uery results were resiEed s'aller to 'ore easily 2t the siEe constraints o&

onboard cache) n !articular, having a large L4 cache, which is ty!ically shared by

all cores on the sa'e socket, has !roven to be es!ecially hel!&ul in boosting (uery

!er&or'ance &or a tabular solutions)

The L4 cache is valuable during table scans i)e), when caches are 'issed because

the iteration over 'e'ory co'!letes sooner when data is &ound in L4 cache) As the

o!erating syste' accesses the 2rst &ew bytes o& a block o& data, it can &etch e-tra

bytes into the L4 cache, and as the scan !roceeds to bytes &urther in the block, itwill hit the' in the L4 cache rather than /AM) Also, (ueries can bene2t &ro' the L4

cache when doing looku!s o& co''on dictionary values and relationshi!s)

If you are familiar with how $nalysis Services manages caching

for multidimensional solutions, caching ehavior for a taular

solutions is consideraly di/erent. &aular solutions use two

types of cache0 a Storage 1ngine cache, and a cell level cache

created y %23 "ueries. Cell-level caching is done y the

4ormula 1ngine. Its the same caching used for

multidimensional solutions.

Storage 1ngine caches are created during "uery e'ecution.

&he 4ormula 1ngine will sometimes *re one or more Storage

1ngine "ueries, which are then cached y the Storage 1ngine.

If a client e'ecutes the same "uery multiple times, "uery

results are returned from the cache, eliminating the Storage

1ngine portion of "uery e'ecution. +y default, server

con*guration properties specify 567 Storage 1ngine cache

slots. 8ou can change this setting in the msmdsrv.ini *le if you

get too many cache misses.

In terms of hardware caching, the operating system fully

controls which data structures are stored in onoard cache or

#$%, with no intervention from $nalysis Services. In terms of

solution architecture, what this means to you is that there is no

speci*c action or server recon*guration on your part that can

change hardware caching ehavior. 2ata structures will either

48


25/33

e placed in onoard cache or in #$%, depending on resource

availaility and competition from other applications using the

same resources.

Ti$!1ou can download CP."] utility &ro' CP.D)co'to deter'ine L6, L4, and L7cache on your co'!uter)

N0'A.nlike its 'ultidi'ensional M#LAP counter!art, a tabular solution is not %.MA

aware) %either the or'ula :ngine nor the torage :ngine will 'odi&y e-ecution

when running on %.MA 'achines)

This 'eans that i& you have a %.MA 'achine, you 'ight run into worse

!er&or'ance than i& you used a non"%.MA 'achine with the sa'e nu'ber o& cores)Ty!ically, this only ha!!ens on syste's having 'ore than 8 %.MA nodes)

Per&or'ance degradation occurs when 'e'ory access has to traverse %.MA nodes

i)e), a thread or instruction e-ecuting on one node needs so'ething that is

e-ecuting on another node) 0hen choosing between syste's that have the sa'e

nu'ber o& cores and /AM, !ick a non"%.MA syste' i& you can)

To oFset !er&or'ance degradation, consider setting !rocessor anity on 9y!er"I

IMs, and then installing Analysis ervices tabular instances on each IM) or 'ore

in&or'ation about this techni(ue, see .sing Tabular Models in a Large"scale

Co''ercial olutionand orcing %.MA %ode anity &or Analysis ervices Tabulardatabases)

'onitor C0 0sage.sing Task Manager, you can get an overall sense o& CP. utiliEation during

!rocessing and (uery o!erations) 0hile Task Manager 'ight not be the best tool to

use &or !er&or'ance tuning, it should be ade(uate &or assessing the CP.

re(uire'ents o& your tabular solution) 0ith Task Manager started, run a

re!resentative sa'!le o& !rocessing and (uery workloads to see how the syste'

!er&or's)

or e-a'!le, the &ollowing screenshot shows CP. usage while e-ecuting a rocess

ullon Contoso/etailD0) Tables are 'ostly !rocessed in se(uence, and as you can

see, overall CP. usage is relatively 'odest)

4;
http://www.cpuid.com/softwares/cpu-z.htmlhttp://msdn.microsoft.com/en-us/library/dn751533.aspxhttp://msdn.microsoft.com/en-us/library/dn751533.aspxhttp://blogs.msdn.com/b/sqlcat/archive/2013/11/05/forcing-numa-node-affinity-for-analysis-services-tabular-databases.aspxhttp://blogs.msdn.com/b/sqlcat/archive/2013/11/05/forcing-numa-node-affinity-for-analysis-services-tabular-databases.aspxhttp://www.cpuid.com/softwares/cpu-z.htmlhttp://msdn.microsoft.com/en-us/library/dn751533.aspxhttp://msdn.microsoft.com/en-us/library/dn751533.aspxhttp://blogs.msdn.com/b/sqlcat/archive/2013/11/05/forcing-numa-node-affinity-for-analysis-services-tabular-databases.aspxhttp://blogs.msdn.com/b/sqlcat/archive/2013/11/05/forcing-numa-node-affinity-for-analysis-services-tabular-databases.aspx


26/33

C0 usage during .uery e/ecution

@ueries that are !ushed to the torage :ngine co'!lete 'uch sooner than those

that s!end additional cycles in the or'ula :ngine) 1ou can identi&y which (ueries

are !ushed to the torage :ngine by using @L erver Pro2ler) 0hen setting u! a

new trace, click S*ow all events) :vents you will want to select a!!ear under

@uery ProcessingH 7ertia. S Query 5egin, 7ertia. S Query Cac*e 'atc*,

7ertia. S Query nd)

4


27/33

0hen you run the trace, you can 'onitor (uery e-ecution to deter'ine (uery

duration and e-ecution) @ueries !ushed to the torage :ngine are indicated through

the event na'e) A line has IertiPa( in the na'e tells you that !art o& the e-!ression

has been !ushed down to -Ielocity torage :ngine)

As you 'onitor (uery e-ecution to understand hardware re(uire'ents, take the

ti'e to identi&y any (ueries that could bene2t &ro' o!ti'iEation) @ueries that take

longer than e-!ected will not only &rustrate users, it will arti2cially and

unnecessarily raise the level o& syste' resources re(uired by your solution)

Although DA (uery o!ti'iEation and DA (uery !lan analysis is beyond the sco!e

o& this guide, there are links at the end o& this docu'ent that !rovide 'ore

in&or'ation)

-ey ointsCP. resources are heavily used &or (ueries and so'e calculations) i'!le

'athe'atical co'!utations based on a single colu'n &or e-a'!le, su''ing or

averaging a nu'eric colu'n are !ushed to the torage :ngine and e-ecuted as a

'ulti"threaded o!eration on 'ulti!le cores) n contrast, a calculation that ranks or

4


28/33

sorts values re(uires the single"threaded or'ula :ngine, using ?ust one core and

!ossibly lots o& 'e'ory de!ending on the siEe o& the te'!orary table)

ncre'ental !er&or'ance gains tend to level oF a&ter 6 logical cores) Although a

greater nu'ber o& cores 74 or 8 will de2nitely increase ca!acity, you won$t see

the sa'e gain in !er&or'ance increase when going beyond 6 cores)

As A Tabular is not %.MA aware, avoid %.MA unless you need it &or other

a!!lications that run on the sa'e hardware) There will be longer wait ti'es i& a

re(uest has to traverse %.MA nodes during a read o!eration)

inally, when you$ve narrowed your server selection to a &ew choices, take a look at

the onboard cache and choose the syste' that oFers the larger onboard cache)

@uery cache o!ti'iEations in the tabular engine target the L6 and L4 cache) 1ou

gain the 'ost bene2t &ro' those o!ti'iEations on a syste' that oFers 'ore

onboard cache)

#is+ I


29/33

Hardware Con3guration /a%$les

This section !rovides a &ew starter e-a'!les o& hardware con2gurations o& e-isting

de!loy'ents, along with additional notes about the !roduction environ'ent) #ver

ti'e, we !lan to u!date this table with 'ore e-a'!les that de'onstrate thebreadth o& de!loy'ent that include in"'e'ory databases)

De!loy'ents described in the &ollowing table range &ro' dedicated servers to 'ulti"

tenant servers using IM technology running on newer and older hardware) or each

de!loy'ent, solution architects re!orted above average !er&or'ance on the

hardware used to run the 'odel)

Model Size System Information RAM ther !etails

85 GB Dell Power:dge

/=65, dual ="core

CP.

4; GB erver runs other @L erver

&eatures as well, including the

relational engine and Analysiservices in 'ultidi'ensional

'ode)

Processing &or the tabular solution

runs on the sa'e server)

rocessullon a weekly basis,

and rocess0$datenightly)85 GB 9ewlett"Packard

ProLiant DL;=5 4

6 TB Multi"tenant architecture

su!!orting at least 8 virtual

'achines will run on the two

syste's, hosting Analysis

ervices in tabular 'ode, Analysis

ervices in 'ultidi'ensional

'ode, harePoint with /e!orting

ervices Power Iiew, and a @L

erver relational database engine)

Decisions about how to allocate

'e'ory across all IMs are still

!ending)

olution design consists o& several

s'aller tabular 'odels, about 65

total, consu'ing around 85 GB o&

'e'ory all together)

8 GB 9ewlett"Packard > GB yste' is !ur!osely oversiEed to

4>


30/33

ProLiant BL85 Gcool"now"we"have"a"

calculator"&or"2nding"out"a"'a-"server"'e'ory"value)as!-

htt!Hblogs)'sdn)co'bs(lsakthi!'a-"server"'e'ory"calculator)as!-

Load Testing

video #!ti'iEing 1our B e'antic Model &or Per&or'ance and cale

video Load Testing Analysis ervices

Can your B solution scale[

htt!Hwww)to'shardware)co'reviewsra'"s!eed"tests,6=558o!ti'iEe"su''ariEe"with"

addcolu'ns"in"da-"ssas"tabular"da-"!ower!ivot)as!-

Did this aer hel you! "lease #ive us your $eedbac%& Tell us on a scale o$ 1 'oor( to )

'e*cellent(, ho+ +ould you rate this aer and +hy have you #iven it this ratin#! or e*amle-

Are you ratin# it hi#h due to havin# #ood e*amles, e*cellent screen shots, clear +ritin#,

or another reason!

Are you ratin# it lo+ due to oor e*amles, $u..y screen shots, or unclear +ritin#!

This $eedbac% +ill hel us imrove the /uality o$ +hite aers +e release&

74
http://office.microsoft.com/en-us/excel-help/create-a-memory-efficient-data-model-using-excel-2013-and-the-powerpivot-add-in-HA103981538.aspxhttp://blogs.msdn.com/b/sqlsakthi/archive/2012/05/19/cool-now-we-have-a-calculator-for-finding-out-a-max-server-memory-value.aspxhttp://blogs.msdn.com/b/sqlsakthi/archive/2012/05/19/cool-now-we-have-a-calculator-for-finding-out-a-max-server-memory-value.aspxhttp://blogs.msdn.com/b/sqlsakthi/p/max-server-memory-calculator.aspxhttp://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/DBI414http://sqlbits.com/Sessions/Event10/Load_Testing_Analysis_Serviceshttp://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/DBI330http://www.tomshardware.com/reviews/ram-speed-tests,1807-2.htmlhttp://mdxdax.blogspot.com/2011/12/dax-query-plan-part-1-introduction.htmlhttp://mdxdax.blogspot.com/2012/01/dax-query-plan-part-2-operator.htmlhttp://mdxdax.blogspot.com/2012/03/dax-query-plan-part-3-vertipaq.htmlhttp://www.powerpivotblog.nl/tune-your-powerpivot-dax-query-dont-use-the-entire-table-in-a-filter-and-replace-sumx-if-possiblehttp://www.powerpivotblog.nl/tune-your-powerpivot-dax-query-dont-use-the-entire-table-in-a-filter-and-replace-sumx-if-possiblehttp://sqlblog.com/blogs/marco_russo/archive/2011/02/07/powerpivot-filter-condition-optimizations.aspxhttp://sqlblog.com/blogs/marco_russo/archive/2011/02/07/powerpivot-filter-condition-optimizations.aspxhttp://www.sqlbi.com/articles/optimize-many-to-many-calculation-in-dax-with-summarize-and-cross-table-filtering/http://www.sqlbi.com/articles/optimize-many-to-many-calculation-in-dax-with-summarize-and-cross-table-filtering/http://sqlblog.com/blogs/marco_russo/archive/2012/09/04/optimize-summarize-with-addcolumns-in-dax-ssas-tabular-dax-powerpivot.aspxhttp://sqlblog.com/blogs/marco_russo/archive/2012/09/04/optimize-summarize-with-addcolumns-in-dax-ssas-tabular-dax-powerpivot.aspxhttp://office.microsoft.com/en-us/excel-help/create-a-memory-efficient-data-model-using-excel-2013-and-the-powerpivot-add-in-HA103981538.aspxhttp://blogs.msdn.com/b/sqlsakthi/archive/2012/05/19/cool-now-we-have-a-calculator-for-finding-out-a-max-server-memory-value.aspxhttp://blogs.msdn.com/b/sqlsakthi/archive/2012/05/19/cool-now-we-have-a-calculator-for-finding-out-a-max-server-memory-value.aspxhttp://blogs.msdn.com/b/sqlsakthi/p/max-server-memory-calculator.aspxhttp://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/DBI414http://sqlbits.com/Sessions/Event10/Load_Testing_Analysis_Serviceshttp://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/DBI330http://www.tomshardware.com/reviews/ram-speed-tests,1807-2.htmlhttp://mdxdax.blogspot.com/2011/12/dax-query-plan-part-1-introduction.htmlhttp://mdxdax.blogspot.com/2012/01/dax-query-plan-part-2-operator.htmlhttp://mdxdax.blogspot.com/2012/03/dax-query-plan-part-3-vertipaq.htmlhttp://www.powerpivotblog.nl/tune-your-powerpivot-dax-query-dont-use-the-entire-table-in-a-filter-and-replace-sumx-if-possiblehttp://www.powerpivotblog.nl/tune-your-powerpivot-dax-query-dont-use-the-entire-table-in-a-filter-and-replace-sumx-if-possiblehttp://sqlblog.com/blogs/marco_russo/archive/2011/02/07/powerpivot-filter-condition-optimizations.aspxhttp://sqlblog.com/blogs/marco_russo/archive/2011/02/07/powerpivot-filter-condition-optimizations.aspxhttp://www.sqlbi.com/articles/optimize-many-to-many-calculation-in-dax-with-summarize-and-cross-table-filtering/http://www.sqlbi.com/articles/optimize-many-to-many-calculation-in-dax-with-summarize-and-cross-table-filtering/http://sqlblog.com/blogs/marco_russo/archive/2012/09/04/optimize-summarize-with-addcolumns-in-dax-ssas-tabular-dax-powerpivot.aspxhttp://sqlblog.com/blogs/marco_russo/archive/2012/09/04/optimize-summarize-with-addcolumns-in-dax-ssas-tabular-dax-powerpivot.aspx


33/33

Send $eedbac%&
mailto:[email protected]?subject=White%20Paper%20Feedback:%20Hardware%20Sizing%20of%20Tabular%20Solutionsmailto:[email protected]?subject=White%20Paper%20Feedback:%20Hardware%20Sizing%20of%20Tabular%20Solutions

SSAS_HardwareSizingTabularSolutions

Documents

Transcript of SSAS_HardwareSizingTabularSolutions