Understanding System Performance
-
Upload
teradata -
Category
Technology
-
view
3.126 -
download
2
description
Transcript of Understanding System Performance
Understanding System Performance – When is it Time for a System Tune-up?
Jim BlairData Warehouse Consultant / Marketing Specialist – Teradata
Agenda
• Overall System Performance> Overview of DBQL and
Viewpoint> Baselines and Benchmarks
– Metrics and reports
> Real-time alerts> Growth Patterns
• Time for a tune up> Best Practices in
Benchmarks> Query Tuning> Load considerations> Compression> Other Considerations
High Level Performance
• Inefficient or bad queries?• Updates and data loading?
> Locking tables and rows for too long
• Too many concurrent tasks?> Big consumption
workloads> Jobs that should
run later
Heatmap from DBQL
Total CPU usage – Is Your System Running HOT?HOT
Heartbeat - June 2009
0.1
1
10
100
6/1 6/2 6/5 6/6 6/7 6/8 6/9 6/12 6/13 6/14 6/15 6/16 6/19 6/20 6/21 6/22 6/23 6/26 6/27 6/28 6/29 6/30
Hour and Day (Business Days only 6 am - 6 pm)
!
Sec
on
ds
(Ela
pse
d)
Heartbeat Graph
What is Teradata Viewpoint?
• System administration portal> Portlets for status displays> Targets business and
technical users> Rewind for system analysis> Manage multiple systems
• Appliance > Server + OS + software> Completely supported by
Teradata• Future platform for Teradata
management products> All management Teradata
Tools and Utilities• NOT an Enterprise Portal
> Does not compete with WebSphere, BEA, SAP, etc.
Viewpoint Portlets in Action
Filtered Queries• List View of all sessions
on a Teradata system• View sessions by different
categories> Active, parsing, delayed, etc.
• Set thresholds to spot problem queries
• Drill down into a session, and access explain plan and SQL text
• Allows for DBAs to easily manage and track sessions and queries filtered by state
Benchmarks and Baselines
• Create a ‘real life’ benchmark> Body of 4 hrs real workload> Do not use static tables> Predictable results > Include all types of queries
– capture 25-40 different queries and extrapolate in 4 hrs of work
• Run the benchmark with consistency> Same # of concurrent queries each time> Same queries each time> Access production data> Capture metrics on each query
• Establish a Baseline> Run benchmark in lowest period of system usage> Quiese the system if possible> Capture metrics on all queries
• When to run the Benchmark process > Quarterly> Before/After every
Upgrade– HW, SW, – major implementations
> On Demand
• Analyze results > What’s changed since last run?
Standard Benchmark
0
50
100
150
200
250
300
350
Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10
Min
ute
s
Run time in minutes
Baselines and Benchmarks
Major Upgrade
Heartbeat - June 2009
0.1
1
10
100
6/1 6/2 6/5 6/6 6/7 6/8 6/9 6/12 6/13 6/14 6/15 6/16 6/19 6/20 6/21 6/22 6/23 6/26 6/27 6/28 6/29 6/30
Hour and Day (Business Days only 6 am - 6 pm)
!
Sec
on
ds
(Ela
pse
d)
Real Time Alerts
• Canary Queries> One in each work load> Set threshold for alerts
• DBA Alerts> High level of spool> Skewing> Number of users
• All production Raw data
• Benchmark tables
• Don’t forget Overhead!> Spool, DBC, DBA workspace
Track Growth Patterns
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10
Teradata Production data
Free
Data
Overhead
0
2
4
6
8
10
12
14
16
18
20
Jun-09
Jul-09
Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Jan-10
Feb-10
Mar-10
Apr-10
May-10
Jun-10
Jul-10
Aug-10
Sep-10
Oct-10
Nov-10
Ter
abyt
es
Overhead
Data
Free
Linear (Data)
Linear (Free)
Linear (Overhead)
Data Growth
Trends Over Time
Performance Tuning
Time to Get Under the Hood
Your
Sr. DBA
When to Run a Benchmark
• General performance trending
• To compare when migrating to or from Teradata
• Regressions
• Before and after Patches and Major or Minor Hardware or Software upgrades
• Test before and after certain project implementations
• May want to test before and after TASM implementations or modifications
Makings of a Successful Benchmark
• Consistent data being accessed
• Consistent indexing, views, security, priorities
• Queries should represent a true mixed workload
• Maintains consistent concurrency levels
• Correct results captured every time
• Queries should be run in same order every time
• Same number of queries completed every time
• Meaningful reports
Building the Benchmark Process
• Process should be 100% automated
• Capture Explains before and after
• Data and queries represent a true workload
• Process should ensure that all indices, statistics, join indices, etc. are current as expected
• Use DBQL to capture queries to represent workload
Building the Benchmark Process
• Capture short, medium and long running queries
• Process should allow for consistent number of concurrent queries
• Design queries to return a count and not return huge sets of rows – Network could skew timings
• Process should report on results when finished
Things to Check For
• Make sure response times for each query and overall process are not abnormally high
• Checking overall or individual response times is NOT enough!
• Make sure results are also accurate
• Examine explain plans to see if dramatically changed> Note: You probably will not care about this if query
is running much better.
• One query can throw off the entire benchmark
Result Report Example
Investigate!• Different results normally indicate a problem, but it could
be that the benchmark spanned two dates
QUERY NUM AVG TIME MIN TIME MAX TIME TIMES RUN
12 1:41:09 1:25:17 1:48:49 426 1:24:57 1:05:16 1:37:53 2017 1:11:02 1:04:50 1:16:54 414 0:53:57 0:36:20 1:01:54 422 0:29:16 0:23:36 0:35:28 824 0:18:13 0:17:45 0:18:38 4
6 0:18:33 0:14:43 0:24:43 24
Response Time Report Example
Look for Consistency and Compare to Past
Final Thoughts on Benchmarks
• Take special note when changes are made to views, indices, TASM, etc.
• These changes may unexpectedly improve or even impair your benchmark
• Keep the benchmark current
Number of Incoming Queries
RejectFilter
Delay Throttle
Reject Throttle
ExceptionReject
OutsideOf SLA
QueriesAfter
Filters and Throttles
Exception Reclass
Workload Management
Query Tuning
• Can save a tremendous load on your system
• Need process to tune query, but then inform and train users as well
• Identify offending queries and report
• Viewpoint-Query Replay
• Document all findings, strategies, time saved, CPU and IO saved, etc.
Query Tuning
• Look for common mistakes such as missing aliases, product joins, poor primary index selection on “Create Tables”
• Make sure necessary statistics are collected
diagnostic helpstats on for session;
• Try tricks like GROUP BY instead of DISTINCT if it applies
Query Tuning
• Look for ways to impact multiple queries with one tuning effort or enhancement
Load Techniques
• Goal – Avoid contention between loads, users, and backups
• Establish DDL naming conventions and document
• Automate process to validate all DDL
• Get Developers to collaborate with DBAs
• Educate Developers on database technology and features (i.e. Locking, Backups, Utilities)
• Educate DBAs on ETL complexities
Load Techniques
• Start using TPT
• Establish Load standards or conventions and enforce them at the ETL level
• Examples > Trickle Batch> Mini Batch> TPUMP> Dynamic stage table creation> Dual Database design
Compression
• Facts> Compress 255 values + Nulls> Cannot compress PI columns> Can improve performance
• May even pay to compress small tables so they can remain memory resident
• Changes coming soon> Compression on variable
length data types> Algorithmic compression
Other Ideas – Don’t reinvent the wheel!
• Materialize view definitions through Join Indexes or automation
• Utilize new features like Multilevel Partitioning
• Archive data
• Horizontal partitioning
• Upgrade – Query Rewrite and Optimizer improvements
• Investigate Hot/Cold Data
Thank You!
Questions?