Sql Server & PowerShell

Aaron Shilo www.dbconsultant.co.il [email protected]

http://www.dbconsultant.co.il/

Who am I ?

• Married + 2.5 children

• A DBA for 10 years.

• Oracle And Sql-Server Certified Professional.

• Used to be CTO@johnbryce training.

• Lead my own consulting business.

• Advisor to Tapuz.co.il / BezeqInt / Lavie Time-tec and more

More about me @ Www.dbconsultant.co.il

Introduction to PowerShell

Using PowerShell to automate administrative tasks

Governing Your Enterprise with PolicyBased Management

Extend Policy Based Management to the Enterprise Scaling PowerShell

Evaluation

Enhance your usage of DMVs

Developing maintenance tasks using SSIS Multi Server Administration

in SQL Server 2008 R2

Deploying apps using the Data Tier

Topics

Introduction to PowerShell

New Command-line shell & Scripting Language

Improves productivity & controlAccelerates automation of system adminEasy-to-use Works with existing scripts

Powershell

Hundreds of Scripts

Books & Training Materials

Community Support

MS MVPsPowerShell Team BlogActive NewsgroupChannel 9: DFO ShowIIS.net

Manning PublicationsO’Reilly MediaSapien Press & others…

TechNet ScriptCenterExchange Server 2007Terminal ServerWMI, Registry, Hardware, etc.Community-Submitted scripts

MyITForum.com

More Control

What is Powershell?

Revolutionary new interactive shell and scripting language

•Based on .NET

•New set of built-in tools (+120)

•New language to take advantage of .NET

•A new “object-pipeline” system view

•Can continue to use current tools

•Can continue to use current automation (COM)

Benefits

•Automate administration of multiple servers through a task-oriented scriptinglanguage

•Accelerate script authoring, testing and debugging and write customer tools in anew command shell environment

•Utilize new scripts and Cmdlets•Manage command-line services, processes, registry, and WMI data•Manage and/or automate administration tasks for server roles such as IIS andActive Directory

•Automate Terminal Server configuration changes by means of PowerShell scripts,and examine configuration similarities and differences across a Terminal Serverfarm.

•Manage an Internet Information Services 7.0 environment.•Remotely manage servers.

• Do I need to learn .NET before I can use PowerShell?

– No - you can continue to use existing tools.

• Do I need to rewrite all my existing tools?

– No - existing tools will run just fine.

• Do I need to learn the new language?

– No - You can easily run existing commands without modification.

• Learning the new

– Online help is full of examples that are ready to use

– The new language elements make interacting with .NET a snap.

– Using .NET can help you where new tools don’t exist and opens a vast space of productivity

– Learn at your own pace, PowerShell will be ready for you

No Fear, Uncertainty or Doubt

10 reasons why you should learn to use PowerShell

1: It’s not going away any time soon

Microsoft has made it clear that PowerShell is here to stay. In fact, PowerShell version 2 is not only included in Windows Server 2008 R2 and in Windows 7, it is enabled by default. Part of the reason why Microsoft has done this is that going forward, many add-on products

will be based on PowerShell.

2: Most Microsoft products will eventually use it

Virtually all of the server products Microsoft is producing right now can be managed through PowerShell. From an administrative standpoint, this means that if you become proficient in PowerShell, you will have the skill set necessary for managing most of Microsoft‘s newer products. The basic built-in PowerShell commands are used in every product that supports PowerShell. However, some server products extend PowerShell to include additional cmdlets.

3: You can’t do everything from the GUI any more

When Microsoft created Exchange 2007, it designed the GUI so that it

could be used only for the most common administrative functions.

Any obscure functions or anything potentially destructive has to be

performed using PowerShell. I expect this design philosophy to carry

over to other Microsoft products.

4: It can make your life easier

Believe it or not, using the command line can make your life easier.

Suppose for a moment that you need to update an Active Directory

attribute for a thousand users. Performing the task manually would

likely take hours to complete. Using PowerShell, though, you can

complete the task using a single line of code.

5: Many GUIs are PowerShell front ends

Many of the GUI interfaces that Microsoft has been designing for its various products are actually front end interfaces to PowerShell. Probably the best known example of this is the Exchange Management Console. Although this utility looks like a standard management tool, it is built entirely on top of PowerShell. Any function you perform through the GUI actually generates PowerShell code that completes the requested task. In many cases, the console even shows you the PowerShell command that was used at the completion of the task.

6: Microsoft certification exams contain PowerShell questions

Microsoft has been adding PowerShell-specific questions to many of its new certification exams. My experience with these exams has been that you don‘t necessarily have to know the full command syntax, but you do need to know which command you should be using in a given situation.

7: You can use PowerShell commands to manage your domains

If you have domain controllers running Windows Server 2003 with Service Pack 2 or higher, you can install the Active Directory Web Services on at least one domain controller. After doing so, you will be able to use the Windows 7 RSAT Suite to manage Windows 2003 and Windows 2008 domains.

8: It enables interactivity between products

PowerShell is the common thread between all the new server products Microsoft is creating, so I expect to start seeing PowerShell used as a mechanism for providing interactivity between server products. I have yet to see a real world example of this interactivity, but eventually I would expect to be able to use a PowerShell script to work seamlessly between products such as IIS, SQL Server, and Exchange.

9: Microsoft says it’s important

Just because someone at Microsoft says that something is important, that doesn‘t mean I take it as gospel. However, In the October 2009 issue of TechNet Magazine, Microsoft says, ―It‘s safe to say that the single most important skill a Windows administrator will need in the coming years is proficiency with Windows PowerShell.‖

Such a bold statement is hard to ignore. This is especially true given the fact that this statement mirrors what I‘ve been hearing from various people at Microsoft every time I have made a trip to Redmond lately.

10: If you don’t learn it, someone else will

As we all know, the economy is in a slump, and many companies are downsizing. Needless to say, there is a lot of competition for the few IT jobs that are available. Therefore, if you suddenly find yourself looking for another job, your odds of finding one may be better if you can list PowerShell among your skill set.

CMDlets

•A cmdlet (pronounced "command-let") •is a single-feature command that manipulates objects in WindowsPowerShell. •You can recognize CMDlets by their name format –

•a verb and noun separated by a dash (-), •Get-Help•Get-Process•Start-Service.

Don‘t throw away any existing scripts or batch files – they can still be used!

Start using Windows PowerShell immediately!

Don‘t forget the power of the wildcard, such as ―*‖

Don‘t deploy Windows PowerShell on any machine where it is not actually needed

Centrally-Control Windows PowerShell security settings through GPOs – do it now!

Powershell - Recommendations

Powershell : lets do it

PowerShell: variable syntax

Basic syntax similar to all scripting languages

$a = ―value‖

$a = 1 + 2

$a += 137

Built-in support for types

$a = [xml]―<test><a>avalue</a></test>‖

[int]$a = 42

or cast to arbitrary .NET object

PowerShell: parsing

Two contexts:

command context

starts with regular character

tries to execute command

expression context

starts with num, var, or quoted string

executes as expression

Can reset mode with ―( )‖

(Get-Date).day + 2

PowerShell: array and hash

$a = 1,2,3,4

$a += 5..10

+ adds elements to array

.. is range operator. What is $a[1..3]?

$a[1], $a[2], $a[3]

$a = @{ one=1; val = get-childitem }

$a[‗val‘]

$a.one

PowerShell: if/switch

if { <test> } { <true> } else { <false> }

elseif { <test> } { <else> }

switch (<var>) { <val> { <case>; break } … }

default case

switch –regex (<var>) { word.* { <case> } … }

dropping break gives multiple matches

$_: variable referring to current <var>

where { <test> }

PowerShell: looping

while($a –lt 137) { <block> }

foreach($var in get-process) { <block> }

IEnumerable support

Shorten foreach to %

ls | %{ $_.Length }

receive piped objects

$_ as before

for { $i = 0; $i –lt 10; $i += 2} { $i }

regular for loop

PowerShell: useful operators

&, .

call operator

$a = ―Get-childitem‖; &$a # calls get-childitem

. used for executing scripts in current context

-as, -is

is/as in C#

$a –as [int]

if { $a –is [System.DateTime] } { … }

PowerShell: functions

function [(args)] { <body> }

if Param is first statement, then gives parameters

arguments passed in $args

input passed in $input

parameter passing on cmd line

add –x 2 –y 3

add 2 3

Useful commands

get-member

return the members of an object

eg. get-member –MemberType property

or, method

Authenticode signing

can use certificates to verify scripts

checks the hash and returns

PowerShell: errors

throw

throw ―error‖

same as ThrowTerminatingError in Cmdlets

trap

catch exceptions

trap [DivideByZeroException] { <do something> }

break/continue semantics

PowerShell: surprises

Drives

C, D

Env, Alias, HKLM, variable, function

mount lets you create others

variable scoping

private, local, script, global


import/export –CSV

get-unique

$(foreach ($line in get-content C:\Test1\File1.txt)

{$line.tolower().split(" ")}) | sort | get-unique

get-item

group

$x = new-object –COM <ProgID>

eg. Outlook.Application


calc

$calc = get-process calc

$calc.add_exited({write-host $this.Processname has exited})

$calc.HasExited

$calc.kill()

$calc.HasExited

Scriptblocks

a chunk of script

a type in PowerShell

$x = [scriptblock]{$y = 137}

&$x;

Can be used as an EventHandler delegate

this is what‘s happening in the last example

object parameter -> $this

EventArgs parameter -> $_

Cmdlet creation

Subclass of System.Management. Automation.Cmdlet

must have an attribute

gives ―verb‖ and ―noun‖ components of name

overrides at least one of

BeginProcessing, ProcessRecord, EndProcessing

use make-shell command to include it

adds dll to set of cmdlets

requires some registry manipulation

Cmdlet creation

Cmdlet parameters

add [Parameter] attribute

Mandatory=true|false

Position=<index>

… others for in-pipeline action

Return value: any object

native object facilities allow inspection

can use arbitrary code

Example in script

function MyWhere {

param ( [scriptblock]$expression )

begin { $matches = 0 }

process {

if ( &$expression ) { $_; $matches++ }

}

end { "Found $matches matches" }

}

Execution Policy

AllSigned

require a digital signature and prompt

user must agree to run script

RemoteSigned

only files from internet need to be signed

default setting

Unrestricted

no signing required

Attacks on AllSigned?

Using PowerShell to Manage And automate

administrative tasks

SQLPS.exe – The SQL Server Mini-Shell

SQL Server Management Objects – SMO

Using ADO.NET for Queries

To make things easier for administrators using PowerShell for SQL

Server, Microsoft created an executable - a mini-shell - which

automatically loads the SQL Server PowerShell snap-ins. The

program is called sqlps.exe and is included when you install SQL

Server 2008. Besides automatically loading the snap-ins the

sqlps.exe environment enables script execution automatically by

setting the execution policy to RemoteSigned.

This mini-shell can be invoked in a number of ways. You can use the

Windows Start, Run menu item and type in sqlps and start it that

way. SQL Server Agent jobs can use the SQL Server Agent

PowerShell subsystem in a step to run a script by selecting the step

type of PowerShell. Each time a step runs the PowerShell step it

loads a separate copy of sqlps.exe into memory, each taking about

40MB, so caution is advised when using this feature.

The most interesting method is within SQL Server Management

Studio. In Object Explorer you can right-click on any object in the

―tree‖ under an instance and select ―Start PowerShell‖, and sqlps.exe

will start up, and the Set-Location cmdlet is used to set the current

location in the shell window to the object you pointed to. This

provides a very nice ability to do some ad-hoc browsing of SQL

Server objects as necessary.

SQL Server Management Objects - SMO

SQL Server Management Objects (SMO) is the object library provided by Microsoft to perform administrative actions against SQL Server. The library is built on top of the .NET Framework (2.0) so applications written using SMO are using managed code and benefit from the features available within the Framework, including the security and cleanup features. SMO was introduced with SQL Server 2005 and allows you to manage servers running SQL Server 2000, SQL Server 2005, and SQL Server 2008.

Using ADO.NET for Queries

ADO.NET has two sets of objects, a connected set that allows you to

connect with a data source, run queries and return result sets, and a

disconnected set of objects that allow you to work with the data

after it‘s been collected. Here‘s a list of the objects.

DescriptionADO.NET Object

A connection to the data source

Connection

Object

Can represent a query against a database, a call to a stored procedure, or a direct

request to return the contents of a specific tableCommand Object

Designed to return query results as quickly as possible

DataReader

Object

Groups a number of changes to a database and treats them as a single unit of work

Transaction

Object

The Connection object has a BeginTransaction method that can be used to create

Transaction objects

Allows the specification of parameters for stored procedures or parameterized queriesParameter Object

Acts as a bridge between the database and the disconnected objects in the ADO.NET

object model

DataAdapter

Object

DescriptionADO.NET Object

Allows the examination of data through collections of rows and

columnsDataTable Object

Corresponds to a column in a table

DataColumn

Object

Defines and enforces column constraintsConstraint Object

Provides access to the DataTable's Rows collectionDataRow Object

The container for a number of DataTable objectsDataSet Object

Defines the relations between DataTables in the DataSet object

DataRelation

Object

Allows the examination of DataTable data in different waysDataView Object

Using the Set-ExecutionPolicy Cmdlet

The Set-ExecutionPolicy cmdlet enables you to determine which Windows PowerShell scripts (if any) will be allowed to run on your computer. Windows PowerShell has four different execution policies:

Restricted - No scripts can be run. Windows PowerShell can be used only in interactive mode.

AllSigned - Only scripts signed by a trusted publisher can be run.

RemoteSigned - Downloaded scripts must be signed by a trusted publisher before they can be run.

Unrestricted - No restrictions; all Windows PowerShell scripts can be run.

Using PowerShell to Manage And automate administrative tasks

Conclusion

Scripting has been a powerful tool for Unix administrators for a long time. Windows administrators have had fewer and less capable options for automating administrative processes until the introduction of PowerShell.

There are a great many sites providing quality information on using PowerShell to automate administrative tasks. The addition of PowerShell support to SQL Server 2008 tool set adds an extra dimension of manageability to the Windows Server environment. You should review the available books and online material to develop your skills in using this powerful tool, and by doing so automate and streamline the processes in your own environment.

Governing Your Enterprise with Policy-Based

Management

Enterprise Policy Management

• Provide auditors with assurance that SQL Server

complies with all security and business guidelines

• Complement All Actions AuditedCompliance

• Ensure peak performance

• High levels of security & reliabilityConsistency

• Drive strategic management initiative to control

costs

• More efficient and proactive managementCosts

Policy-Based Management Framework

Introduction to the Framework

Evaluation Modes

Extending Policy-Based Management to the Enterprise

Solution Architecture

Considerations for Large-Scale Environments

Centralized Reporting

Policy-Based Management

Set of related logical properties

Specifies a set of allowed states of a managed target with regard to a facet

Defines the evaluation mode, target filters, and schedule of the conditions.

Facet

Condition

Policy

Policy

Server

Restriction

Target

Evaluation

Mode

Category

Policy Evaluation Modes

On Demand

• Evaluate a policy when specified by user

• Available through SSMS or Windows

PowerShell™

• Option to force certain conditions to

comply with policy

• Supports down-level evaluation

(depends on properties exposed)

On Schedule

• SQL Server 2008 only

• SQL Server Agent job periodically evaluates

a policy

On Change: Prevent


• DDL triggers prevent policy violations

On Change: Log Only


• Event notification evaluates a policy when a

relevant change is made

Evaluation modes

On Demand Evaluation

Windows PowerShell™ Integration

Windows PowerShell™ is a framework and runtime for executing

management commands

Cmdlets are instances of .NET classes that process input objects from the

pipeline

SQL Server Provider for Windows PowerShell™ encompasses SMO

Invoke-PolicyEvaluation –Policy

DatabaseStatus.xml,

Trustworthy.xml -TargetServerName inst1

Invoke-SQLCMD –Query ”SELECT name FROM

sys.Databases;” –ServerInstance

“MyServer\Instance”

http://en.wikipedia.org/wiki/Image:Windows_PowerShell_icon.png

policy results

policy results

policy results policy

results

policy results

policy results

Extend you enterprise- Bringing it all together


Enterprise Policy Management

Extend to the Enterprise

Bringing It All Together

policy results

policy results

policy results

policy results

policy results

policy results



Extend to the Enterprise- Central Management

Server

Logically group instances based on

business function(s)

Centrally publish policies to groups of

SQL Server 2008 instances

Evaluate policies on-demand against

a group of servers

Filter by logical groups in Windows

PowerShell™ scripts

Extend to the Enterprise- Scaling PowerShell

Evaluation

Add Intelligence to Policies

Place each policy in a category

Define server restrictions for

versions and editions where

appropriate

Extend to the Enterprise- Scaling PowerShell

Evaluation

Create Custom Server Groups in the

CMS

Run specific policies against a list

of servers

Examples: Production,

Development, PCI

Define Concurrent Jobs

Define multiple concurrent

executions based on Policy

Category and/or logical Central

Management Server group

Extend to the Enterprise

Benefits of Upgrading to SQL Server 2008

Real-Time Enforcement and

Reporting

Monitor the event log through

Alerting integration

Advanced functionality and

integration with SSMS

Dependency, health states,

subscriptions, history

Scale

Security

Access to other rich features in SQL

Server 2008

Centralizing Policy History

SQL Server 2008

policy results

policy results

policy results

syspolicy_policy_execution_history

syspolicy_policy_execution_history_details

policy results




Additional Resources

Policy Based Management

Enterprise Policy Management Framework

http://www.codeplex.com/EPMFramework

Policy Based Management Blog

http://blogs.msdn.com/sqlpbm/default.aspx

Lara Rubbelke‘s Blog

http://sqlblog.com/blogs/lara_rubbelke/default.aspx

Dan Jones‘ Blog

http://blogs.msdn.com/dtjones/default.aspx

Buck Woody‘s Blog

http://blogs.msdn.com/buckwoody/default.aspx

http://www.codeplex.com/EPMFramework

http://blogs.msdn.com/sqlpbm/default.aspx

http://sqlblog.com/blogs/lara_rubbelke/default.aspx

http://blogs.msdn.com/dtjones/default.aspx

http://blogs.msdn.com/buckwoody/default.aspx

Additional Resources

Windows PowerShell™

To learn more about the Windows PowerShell™ scripting Language

http://www.microsoft.com/downloads/details.aspx?FamilyID=b4720b

00-9a66-430f-bd56-ec48bfca154f&DisplayLang=en

Windows PowerShell™ Blog

http://blogs.msdn.com/powershell/

Allen White‘s Blog

http://sqlblog.com/blogs/allen_white/

SQL Server PowerShell Overview

http://msdn.microsoft.com/en-us/library/cc281954.aspx

http://www.microsoft.com/downloads/details.aspx?FamilyID=b4720b00-9a66-430f-bd56-ec48bfca154f&DisplayLang=en




















http://blogs.msdn.com/powershell/

http://sqlblog.com/blogs/allen_white/






Extend Policy Based Management to the Enterprise Scaling PowerShell Evaluation

The Enterprise Policy Management Framework (EPM) is a solution to

extend SQL Server 2008 Policy-Based Management to all versions of

SQL Server in an enterprise, including SQL Server 2000 and SQL

Server 2005. The EPM Framework will report the state of specified

SQL Server instances against policies that define intent, desired

configuration, and deployment standards.

When the Enterprise Policy Management Framework (EPM) is

implemented, policies will be evaluated against specified instances of

SQL Server through PowerShell. This solution will require at least one

instance of SQL Server 2008. The PowerShell script will run from this

instance through a SQL Server Agent job or manually through the

PowerShell interface. The PowerShell script will capture the policy

evaluation output and insert the output to a SQL Server table. SQL

Server 2008 Reporting Services reports will deliver information from

the centralized table.

This solution requires the following components to be configured in

your environment. All SQL Server 2008 requirements listed below

may be executed from and managed on the same instance:

§ SQL Server 2008 instance to store policies

§ SQL Server 2008 instance to act as the Central Management Server

§ SQL Server 2008 instance to execute the PowerShell script

§ SQL Server management database to archive policy evaluation results

§ SQL Server 2008 Reporting Services to render and deliver policy

history reports

Extend Policy Based Management to the Enterprise Scaling PowerShell Evaluation

67

Overview

SQL Server 2000 provided

~11 PerfMon objects and counters

Access to some system tables

A few table-valued functions

SQL Server 2005 introduced numerous long-awaited performance measurement metrics

Compared to SQL 2000, SQL 2005

Approximately tripled SQL-specific PerfMon objects and counters

Created numerous views and functions (~80 Dynamic Management Views and Functions) that provide very useful and previously unavailable performance information

SQL 2008

Added four SQL-specific PerfMon objects and counters

Added ~40 Dynamic Management Views and Functions

68

Overview

Sheer number of performance views and new metrics can be dizzying

New metrics documented in

Books Online

Numerous articles and books

However, much of the literature simply defines the DMVs and metrics, so knowing where to begin is extremely difficult

http://www.microsoft.com/downloads/details.aspx?FamilyID=531c53e7-8a2a-4375-8f2f-5d799aa67b5c&displaylang=en




























69

Overview

Performance analyst must understand

Which PerfMon objects and DMVs are pertinent to a particular

problem

How to interpret numerous metrics provided by new objects and

views

Most valuable views and metrics

70

Emphasis of Presentation

Highlight most useful of the newly available

SQL Server-specific PerfMon objects and counters

DMVs

How to convert the numeric DMV identifiers into understandable text

Properly use and interpret most important

SQL Server-specific PerfMon objects and counters

DMV metrics

71

SQL Server PerfMon Objects

Many more useful metrics, e.g.,

Wait statistics

Tempdb (General Statistics)

Cursors

Transactions

.Net Data Provider for SQL Server

Deprecated Features (2008)

72

Wait Statistics Object

Subset of this information was previously available only via complex

queries from several disparate sources

Small overlap with Locks object

Instances where Counters would normally be

Most useful instances

Cumulative wait time (ms)

Average wait time (ms)

73


Cumulative wait time (ms) instance extremely useful

Unfortunately, still only counts values within the last second

Sampling every 15-30 seconds usually provides an adequate

representation

Use sys.dm_os_wait_stats to determine

Total wait times by specific type of wait without sampling problems

Discussed later in presentation

74


Summarizes very useful counters including

Lock waits (total for all lock resources)

Log buffer waits

Log write waits

Memory grant queue waits

Network IO waits

Non-Page latch waits

Page IO latch waits

Page latch waits

Thread-safe memory object waits

Transaction ownership waits

75

Lockable PerfMon Resources

Resource Description

AllocUnit (2005/2008) Allocation unit

Application

(2005/2008)

Application-specified resource

Database Entire database

Extent Contiguous group of 8 data or index pages

File (2005/2008) Database file

HoBT (2005/2008) Heap or B-Tree index

Key Row lock within an index that protects range of keys in

serializable transactions

Metadata (2005/2008) Meta data or catalog information

Object (2005/2008) Job flow synchronization object, table, stored procedure, or

view

Page 8-kilobyte (KB) data or index page

RID Row ID. Used to lock a single row within a table

Table (replaced by

Object on 2005/2008)

Entire table, including all data and indices

76

Deprecated Features Object

Simplifies detection of deprecated feature usage since last SQL Server instance restart

Examples of deprecated features/counters

'::' function calling syntax

Data types: text ntext or image

DATABASEPROPERTY

DATABASEPROPERTYEX('IsFullTextEnabled')

DBCC DBREINDEX

DBCC INDEXDEFRAG

DBCC SHOWCONTIG

NOLOCK or READUNCOMMITTED in UPDATE or DELETE

sp_adduser

sysindexes

sysobjects

Use Deprecation Announcement & Deprecation Final Support trace events to determine exact causes

77

Critical General Views

Required for converting numeric DMV identifiers into understandable text

Many were available under 2000 with slightly different names

2000 sysobjects -> 2005 sys.objects

See ―Mapping System Tables to System Views (Transact-SQL)‖ topic in BOL for further info

sys.databases

Lists all databases and their IDs so proper associations can be made

sys.partitions

Only way to decode HOBT IDs returned by lock-specific information, e.g. blocked process records

sys.configurations

Provides information regarding OS and SQL Server configurations

sys.dm_os_sys_info

Provides information regarding system (can be sampled once or repeatedly)

Useful for determining whether hyperthreading is active and accessible memory

78

sys.dm_os_sys_info Useful Columns

No parameters required

Number of logical CPUs on the system

Ratio of number of logical or physical cores exposed by one physical

processor package (hyperthread_ratio)

Amount of physical memory available

Amount of virtual memory available to the process in user mode.

This can be used to determine whether SQL Server was started by

using a 3-GB switch

Date and time SQL Server last started

79

Database-Specific Views

Query for each database separately

sys.objects

Lists all database objects such as tables, views, stored procedures,

etc.

sys.indexes

Lists all indices and their associated table IDs

Does not provide row counts as sysindexes does

sys.filegroups

Lists all file groups and their IDs

sys.database_files

Lists all physical database files and their IDs

80

Dynamic Management Views and Functions

“Designed to give you a window into what's going on inside SQL Server”

Two types

DMV - Pure view, i.e., no parameters required

DMF - Table-valued function, i.e., parameters required

Parameters usually specify database, table, index, partition, etc.

Provide significant amount of information regarding

System

Databases

Internal workings of SQL Server

Performance

81


Some were possible with complex queries on 2000

When possible

Very clumsy to use

Required temp tables

Required significant understanding of the underlying tables

Most DMVs are very simple to use

Some still require joins with other DMVs and DMFs

Some may lessen need for using SQL Trace

82


Contain values since last SQL Server instance restart, unless manually reset

Perfect for periodic or intermittent sampling

Therefore, must compute differences between adjacent or distant samples for same database, file, table, index, etc.

Many DMVs can be reset manually using command similar to

DBCC SQLPERF ('sys.dm_os_wait_stats', CLEAR)

83

Dynamic Management Views and Functions –

General Guidelines

Most performance DMVs begin with sys.dm_

Must reconcile numeric IDs with static views that contain textual

names

Values accumulated from last SQL Server instance restart

Sample rates can range from once per minute to a few times per

day

Must calculate differences between individual sample records

Be sure to diff records with same database ID, object ID, and index

ID

84


General Usage Scenario

Method 1

Capture before monitored activities begin and store results into a SQL table or a spreadsheet

Execute workload (whether natural production or test)

Capture again and compare value differences

Load into spreadsheets for further analysis, if necessary

Method 2

Capture every <n> minutes and store results in a flat file

After workload and capture process complete, load data into SQL tables

Use SQL to compare incremental value differences and store results

Extract interval data to spreadsheets for further analysis

85


Usage Warnings

Microsoft assures us these views DO NOT BLOCK (single exception

cited below) as usage of older system tables could

However, be careful of execution frequency when cross apply or

several joins involved because resource consumption could be

high

Beware of the following two

sys.dm_db_index_physical_stats, which examines the data within

the physical files and provides extremely valuable data, can block

depending upon the option used, but not nearly as badly as

DBCC SHOWCONTIG

sys.dm_tran_locks can consume large amounts of CPU and can

generate huge output when # of locks large

Use only if absolutely necessary

86


Category ListView Category View Category

Change Data Capture Related Dynamic

Management Views

Query Notifications Related Dynamic

Management Views

Common Language Runtime Related

Dynamic Management Views

Replication Related Dynamic Management

Views

Database Mirroring Related Dynamic

Management Views

Resource Governor Dynamic Management

Views

Database Related Dynamic Management

Views

Service Broker Related Dynamic

Management Views

Execution Related Dynamic Management

Views and Functions

SQL Server Extended Events Dynamic

Management Views

Full-Text Search Related Dynamic

Management Views

SQL Server Operating System Related

Dynamic Management Views

Index Related Dynamic Management

Views and Functions

Transaction Related Dynamic Management

Views and Functions

I/O Related Dynamic Management Views

and Functions

Security Related Dynamic Management

Views

Object Related Dynamic Management

Views and Functions

87

Execution-Related Dynamic Management Views

and Functions

Most commonly used ones

sys.dm_exec_query_stats

sys.dm_exec_requests

sys.dm_exec_sessions

sys.dm_exec_sql_text

Other less commonly used ones

sys.dm_exec_cached_plans

sys.dm_exec_connections

sys.dm_exec_cursors

88

sys.dm_exec_query_stats DMV

Returns aggregate performance statistics for cached and completedquery plans

Top <n> can be returned for any category

Execution count, CPU time, CLR time, and elapsed time

Physical and logical reads, Logical writes

Activity Monitor on 2008 appears to use this DMV primarily

Currently active long-running queries NOT shown via this view

View contains one row per query plan

Lifetime of a row is tied to the plan itself

Can reduce dependence upon SQL statement tracing, but creation_time, last_execution_time, and accumulated values MUST be accounted for

89

sys.dm_exec_query_stats Columns

―Starting and ending positions of the query‖… ―within the text of its

batch or persisted object‖

Pointer to plan, which can be passed to the sys.dm_exec_sql_text

DMF

Times at which plan was compiled and last executed

# of times plan executed since last compilation

Total, minimum, maximum, and last amounts consumed by executions

since compilation

CPU, CLR, and elapsed times, all in microseconds


90

sys.dm_exec_query_stats Example

Returns top 20 query statements that performed most logical readsselect top 20 [sql_handle], creation_time, execution_count,

last_execution_time, total_worker_time, total_physical_reads, last_physical_reads, max_physical_reads, total_logical_reads, last_logical_reads, max_logical_reads, total_logical_writes, last_logical_writes, max_logical_writes, total_clr_time, last_clr_time, max_clr_time, total_elapsed_time, last_elapsed_time, max_elapsed_time,SUBSTRING(st.text, (qs.statement_start_offset/2) + 1,

((CASE statement_end_offsetWHEN -1 THEN DATALENGTH(st.text)ELSE qs.statement_end_offset END- qs.statement_start_offset)/2) + 1) as statement_text

FROM sys.dm_exec_query_stats as qsCROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as stORDER BY total_physical_reads DESC

Note how other DMVs and DMFs are used

91

sys.dm_exec_requests Columns

Timestamp when request arrived

Status of the request

Identifies current type of command being processed, e.g., select,

insert, update, etc.

Plan handles and offsets similar to sys.dm_exec_query_stats

If blocked, ID of session blocking request

If request currently (or previously) blocked, returns wait type and wait

time (ms)

# of transactions and result sets open for request

CPU, total elapsed times in (ms)


# of rows that have been returned to the client by this request

92

sys.dm_exec_requests Example

Returns batches that contain top 20 currently active query statements that consumed the most CPU

select top 20 *

from sys.dm_exec_requests ExecReq

OUTER APPLY sys.dm_exec_sql_text (ExecReq.sql_handle) ExecSQLText

where (session_id > 50 and session_id <> @@spid) and

command not like 'WAITFOR%'

order by cpu_time desc

93

Index Related Dynamic Management Views and

Functions

sys.dm_db_index_usage_stats

Returns information regarding query code usage of tables and indices, e.g., inserts, updates, deletes, random accesses, and sequential accesses

sys.dm_db_index_operational_stats

―Returns current low-level I/O, locking, latching, and access method activity for each partition of a table or index in the database‖

Most commonly useful metric tracks actual forwarded record fetches

sys.dm_db_index_physical_stats

Execute infrequently because physical files are interrogated and causes some blocking to occur

Replaces DBCC SHOWCONTIG

94

sys.dm_db_index_usage_stats DMV

Returns

One row per index

Number of seeks, scans, lookups, and updates for user and system queries

Seeks random

Scans sequential

Time of last seek, scan, lookup, and update for user and system queries

System category

Maintenance, e.g., statistics updates

User category

Insert, update, delete, and select operations

Reports user statements, not record counts

One insert statement can result in millions of rows being inserted

View will report one, not millions

Helps determine index and table usage patterns

Particularly useful for identifying indices that are never used by queries

95

sys.dm_db_index_usage_stats Example

List Rarely-Used Indices For Specific DB With Resolved Names

declare @dbid int = db_id()

select objectname=object_name(inxusage.object_id),

inxusage.object_id,

indexname=sysinx.name, sysinx.index_id, user_seeks, user_scans,

user_lookups, user_updates

from sys.dm_db_index_usage_stats inxusage,

sys.indexes sysinx

where database_id = @dbid and

objectproperty(inxusage.object_id,'IsUserTable') = 1 and

sysinx.object_id = inxusage.object_id and

sysinx.index_id = inxusage.index_id

order by (user_seeks + user_scans + user_lookups +

user_updates) asc

96

sys.dm_db_index_operational_ stats DMF

One row per index

Helpful for

Determining how indices are used

Identifying contention areas

Four parameters required

{ database_id | NULL | 0 | DEFAULT } (use db_id() for current db)

{ object_id | NULL | 0 | DEFAULT }

{ index_id | NULL | -1 | DEFAULT }

{ partition_number | NULL | 0 | DEFAULT }

Use NULL parameters to obtain information for all databases and files

select * from sys.dm_db_index_operational_stats (NULL, NULL,

NULL, NULL)

97

sys.dm_db_index_operational_ stats Columns Pt 1

Cumulative counts

Range and table scans started on index or heap

Single row retrievals from index or heap

Rows that were fetched through forwarding record

Row locks requested

Page locks requested

Cumulative counts and elapsed times database engine

Waited on row lock

Waited on page lock

Waited because of latch contention

Waited on I/O page latch

98

sys.dm_db_index_operational_ stats Columns Pt 2

Cumulative counts

Leaf-level insert, delete, update, delayed delete operations

Above leaf-level insert, delete, update, delayed delete operations

Zeroes heap operations

Leaf-level page allocations in the index or heap

For an index, page allocation corresponds to a page split

Page allocations caused by page splits above leaf level

Page compression attempt and success counts (2008)

99

sys.dm_db_index_operational_ stats Usage Pt 1

Reports actual operation counts, not user statements

One insert statement can result in millions of rows being inserted

View will report millions, not one

To analyze a common access pattern to the table or index partition

leaf_insert_count

leaf_delete_count

leaf_update_count

leaf_ghost_count

range_scan_count

singleton_lookup_count

100

sys.dm_db_index_operational_ stats Usage Pt 2

To identify latching and locking contention

page_latch_wait_count and page_latch_wait_in_ms

Indicate whether latch contention exists on index or heap, and significance of contention

row_lock_count and page_lock_count

Indicate how many times the Database Engine tried to acquire row and page locks

row_lock_wait_in_ms and page_lock_wait_in_ms

Indicate whether lock contention exists on index or heap, and significance of contention

To analyze statistics of physical I/Os on an index or heap partition

page_io_latch_wait_count and page_io_latch_wait_in_ms

Indicate how many physical I/Os were issued to bring index or heap pages into memory and how much waiting was involved

101

sys.dm_db_index_physical_stats DMF

Five parameters required

{ database_id| NULL | 0 | DEFAULT } (use db_id() for current db)

{ object_id| NULL | 0 | DEFAULT }

{ index_id| NULL | -1 | DEFAULT }

{ partition_number| NULL | 0 | DEFAULT }

{ LIMITED | SAMPLED | DETAILED | NULL | DEFAULT }

LIMITED mainly provides fragmentation information

Strict locking scheme not used for LIMITED or SAMPLED, so DETAILED

mode primary cause of any blocking

DETAILED provides much useful information

Record count, forwarded record count, min-max-avg record

lengths

102

sys.dm_db_index_physical_stats


select * from sys.dm_db_index_physical_stats (NULL, NULL, NULL,

NULL, 'DETAILED')

DETAILED mode

Used instead of SAMPLED when index or heap has < 10,000 pages

Provides information about non-leaf levels

103

sys.dm_db_index_physical_stats Columns

Index_type_desc

Index type

Index_id, index_level

0 heap

Index_depth

# of index levels

Avg_fragmentation_in_percent

Logical fragmentation for indexes or extent fragmentation for heaps

Indices with values > 30 are candidates for rebuild/reorganization

Fragment_count

Number of fragments in the leaf level

Avg_fragment_size_in_pages

Average number of pages per fragment in the leaf level

Larger is better

Forwarded_record_count

# of heap records having forward pointers to another data location

104

I/O Related Dynamic Management Views and

Functions

sys.dm_io_pending_io_requests

sys.dm_io_virtual_file_stats

105

sys.dm_io_virtual_file_stats DMF

Returns I/O statistics for data and log files, one row per file


select * from sys.dm_io_virtual_file_stats (NULL, NULL)

Equivalent to select * from ::fn_virtualfilestats(-1,-1) on SQL 2000

106

sys.dm_io_virtual_file_stats Columns

ID of database

ID of file

Number of milliseconds since SQL Server instance started (useful for

detecting restarts)

Number of reads and writes issued against this file

Total number of bytes read from and written to this file

Total time, in milliseconds, users waited for I/O completions overall, as

well as reads and writes specifically

Number of disk bytes used by this file

107

SQL Server Operating System-Related Dynamic

Management Views

Abundant views

Most frequently used views

sys.dm_os_wait_stats

sys.dm_os_waiting_tasks

sys.dm_os_latch_stats

108

sys.dm_os_wait_stats DMV

Useful for diagnosing performance issues with

SQL Server

Specific queries and batches

Expands upon PerfMon Wait Statistics object

Returns information about waits encountered by threads in execution

226 possible wait types

Several of these are innocuous because they are accumulated as

part of normal SQL Server operation

Very important to eliminate normal wait types from

consideration

109

sys.dm_os_wait_stats

Very useful to categorize and group individual wait types to simplify

understanding and analysis, e.g.,

Combine LOGMGR, IMPPROV_IOWAIT, IO_AUDIT_MUTEX,

IO_COMPLETION, DISKIO_SUSPEND, IO_AUDIT_MUTEX,

IO_COMPLETION, ASYNC_DISKPOOL_LOCK,

ASYNC_IO_COMPLETION, REQUEST_DISPENSER_PAUSE,

PAGEIOLATCH_DT, PAGEIOLATCH_EX, PAGEIOLATCH_KP,

PAGEIOLATCH_NL, PAGEIOLATCH_SH, PAGEIOLATCH_UP into I/O

Combine PAGELATCH_DT, PAGELATCH_EX, PAGELATCH_KP,

PAGELATCH_NL, PAGELATCH_SH, PAGELATCH_UP into Latches

110

sys.dm_os_waiting_tasks DMV

Reports information about currently waiting tasks

Can be used to construct blocker/waiter lists

Important Fields

Total wait time (ms) for wait type

Name of wait type

Task currently holding this resource

Blocking task session ID

Description of resource being consumed

111

sys.dm_os_latch_stats DMV

SQL Server uses numerous kinds of latches (118 documented as of 2008 SP1)

Latch times can be helpful in verifying I/O subsystem performance

Large latch wait times often indicate poor I/O performance

View returns information regarding all latch types

Very useful to categorize and group individual wait types to simplify understanding, e.g.,

BUFFER_POOL_GROW and BUFFER into Buffer

ALLOC_CACHE_MANAGER, ALLOC_CREATE_FREESPACE_CACHE, ALLOC_CREATE_RINGBUF, ALLOC_EXTENT_CACHE, and ALLOC_FREESPACE_CACHE into Allocation

112

Transaction Related Dynamic Management Views

and Functions

Numerous views

sys.dm_tran_locks is useful, but can distort system depending upon #

of locks being held

113

sys.dm_tran_locks DMV

Each row represents currently active request to lock manager for lock

that has been granted or is waiting to be granted

Result set columns are divided into two main groups — resource and

request

Resource group describes resource on which the lock request is

being made

Resource type same as lockable resource

Request group describes the lock request

114

Conclusions

SQL Server 2008 provides a wealth of performance information

Numerous Dynamic Views are essential for uncovering and resolving

performance bottlenecks

Dynamic Views and Functions are

Easy to use

Well documented

Low overhead with very few exceptions

115

New sys.dm_ DMVs and DMFs in SQL 2008View Name View Name View Name

dm_audit_actions dm_fts_fdhosts dm_server_audit_status

dm_audit_class_type_map dm_fts_outstanding_batches dm_tran_commit_table

dm_cdc_errors dm_os_dispatcher_pools dm_xe_map_values

dm_cdc_log_scan_sessions dm_os_dispatchers dm_xe_object_columns

dm_cryptographic_provider_pro

perties dm_os_memory_brokers dm_xe_objects

dm_database_encryption_keys dm_os_memory_node_access_stats dm_xe_packages

dm_db_mirroring_auto_page_rep

air dm_os_memory_nodes

dm_xe_session_event_actio

ns

dm_db_mirroring_past_actions dm_os_nodes dm_xe_session_events

dm_db_persisted_sku_features dm_os_process_memory

dm_xe_session_object_colu

mns

dm_db_script_level dm_os_spinlock_stats dm_xe_session_targets

dm_exec_procedure_stats dm_os_sys_memory dm_xe_sessions

dm_exec_trigger_stats

dm_resource_governor_configurati

on

dm_filestream_file_io_handles

dm_resource_governor_resource_p

ools

dm_filestream_file_io_requests

dm_resource_governor_workload_g

roups

Developing maintenance tasks using SSIS Multi

Server Administration in SQL Server 2008 R2

Developing maintenance tasks using SSIS Multi Server Administration in SQL Server 2008 R2


Data-Tier Applications, abbreviated DAC

R2 makes it easier for DBAs to move databases around from server

to server in much the same way virtualization admins move guest

OS‘s around between physical hosts.

No more need for backup and restore.

Easily deploy databases.

The following steps illustrate a simple deployment process:

A database developer completes the development of a Finance version

1.0 DAC, and then builds a FinanceVersion1.dacpac DAC package.

A database administrator deploys the Finance 1.0 DAC to production.

The production instance of the database engine now has a deployed

DAC that has an application name of Finance, a version of 1.0, and an

associated database named Finance.

The database developer then starts work on the next version, and when

development completes builds a FinanceVersion2.dacpac.

The database administrator then runs the Upgrade Data-tier Application

Wizard specifying both the current deployed version 1.0 DAC and the

version 2.0 DAC package. The wizard performs the following steps:

Verifies that the deployed DAC and the DAC package both have

their application names set to Finance. It also evaluates whether

the instance of the Database Engine meets the requirements

specified in the server selection policy (if defined) in the version

2.0 DAC.

Deploys the Finance version 2.0 DAC from the DAC package. This

creates a new database with a temporary name.

Sets the original database to read-only if it is not already in read-

only mode, and copies the data to the new database.

If the original database was in read-only mode, the new database

is set to read-only.

The original database is renamed by appending a string to the

end of the database name.

The new database is assigned the original database name:

Finance.

After the database administrator has confirmed that the new database is

operating correctly, the original database can be archived.

The DAC does not include the data inside your database. For deployment best practices, you should have any necessary data (configuration tables, basic lookup tables) already scripted out as part of your deployment strategy. With the DAC approach, it makes sense to put these scripts inside the database as objects.

For example, you might have a stored procedure called usp_deploy that populates all of the necessary configuration tables via insert statements.

So We Got Some Time Left ….

Filtered indices.

Resource Governor.

CDC – change data capture.

T-sql New Features.

[email protected]

All PPT & samples @

Www.dbconsultant.co.il

mailto:[email protected]

http://www.dbconsultant.co.il/

Sql Server & PowerShell

Documents

Transcript of Sql Server & PowerShell