Teradata Database Performance Management

232
Teradata Database Performance Management Release 13.10 B035-1097-109A February 2011

Transcript of Teradata Database Performance Management

Page 1: Teradata Database Performance Management

Teradata Database

Performance ManagementRelease 13.10

B035-1097-109AFebruary 2011

Page 2: Teradata Database Performance Management

The product or products described in this book are licensed products of Teradata Corporation or its affiliates.

Teradata, BYNET, DBC/1012, DecisionCast, DecisionFlow, DecisionPoint, Eye logo design, InfoWise, Meta Warehouse, MyCommerce, SeeChain, SeeCommerce, SeeRisk, Teradata Decision Experts, Teradata Source Experts, WebAnalyst, and You’ve Never Seen Your Business Like This Before are trademarks or registered trademarks of Teradata Corporation or its affiliates.

Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc.

AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc.

BakBone and NetVault are trademarks or registered trademarks of BakBone Software, Inc.

EMC, PowerPath, SRDF, and Symmetrix are registered trademarks of EMC Corporation.

GoldenGate is a trademark of GoldenGate Software, Inc.

Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company.

Intel, Pentium, and XEON are registered trademarks of Intel Corporation.

IBM, CICS, RACF, Tivoli, and z/OS are registered trademarks of International Business Machines Corporation.

Linux is a registered trademark of Linus Torvalds.

LSI and Engenio are registered trademarks of LSI Corporation.

Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United States and other countries.

Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries.

QLogic and SANbox are trademarks or registered trademarks of QLogic Corporation.

SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc.

SPARC is a registered trademark of SPARC International, Inc.

Sun Microsystems, Solaris, Sun, and Sun Java are trademarks or registered trademarks of Sun Microsystems, Inc., in the United States and other countries.

Symantec, NetBackup, and VERITAS are trademarks or registered trademarks of Symantec Corporation or its affiliates in the United States and other countries.

Unicode is a collective membership mark and a service mark of Unicode, Inc.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Other product and company names mentioned herein may be the trademarks of their respective owners.

THE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED ON AN “AS-IS” BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO THE ABOVE EXCLUSION MAY NOT APPLY TO YOU. IN NO EVENT WILL TERADATA CORPORATION BE LIABLE FOR ANY INDIRECT, DIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS OR LOST SAVINGS, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

The information contained in this document may contain references or cross-references to features, functions, products, or services that are not announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions, products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or services available in your country.

Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time without notice.

To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document. Please e-mail: [email protected]

Any comments or materials (collectively referred to as “Feedback”) sent to Teradata Corporation will be deemed non-confidential. Teradata Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform, create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including developing, manufacturing, or marketing products or services incorporating Feedback.

Copyright © 2000 – 2011 by Teradata Corporation. All Rights Reserved.

Page 3: Teradata Database Performance Management

Preface

Purpose

Performance Management provides information that helps users ensure that Teradata Database operates at peak performance based on user applications and processing needs. To that end, it recommends basic system management practices.

Audience

The primary audience includes database and system administers and application developers.

The secondary audience is Teradata support personnel, including field engineers and local as well as global support and sales personnel.

Supported Software Releases and Operating Systems

This book supports Teradata® Database 13.10.

Teradata Database 13.10 supports:

• Microsoft Windows Server 2003 64-bit

• SUSE Linux Enterprise Server 10

Teradata Database client applications can support other operating systems.

Prerequisites

You should be familiar with your Teradata hardware and operating system, your Teradata Database and associated client products, and the utilities you can use to tune Teradata Database to improve performance.

Performance Management 3

Page 4: Teradata Database Performance Management

PrefaceChanges to this Book

Changes to this Book

Release Description

Teradata Database 13.10February 2011

Clarified documentation of Cylinder Read settings.

Teradata Database 13.10August 2010

• Update book for Teradata Database 13.10 performance features.

• Added text and references to Teradata Viewpoint / deleted text and references to Teradata Manager, including Performance Monitor (PMON), Priority Scheduler Administrator (PSA), Teradata Dynamic Workload Manager (DWM), and Teradata SQL Assistant (Web edition).

• Removed references to Teradata Query Manager.

• Documentat algorithmic compression.

• Clarified information on multi-value compression.

• Added information on defining join indexes using UDT columns and expressions and on refreshing join indexes.

• Documented AutoCylPack.

• Documented Datablock Merge.

• Added reference to Cylinder Read Ageing Threshold.

• Clarified explanation of Cylinder Read defaults.

• Removed references to MP-RAS and 32-bit systems and references that specifically call out 64-bit systems.

• Removed Resource Usage (ResUsage) macros that Teradata does not directly support.

• Removed reference to xctl, xperfstate, RSS Monitor, puma, and information on extending DATE using CALENDAR.

• Revised chapter on Account String Expansion (ASE), on DBQL, on Resource Usage (ResUsage), on locks and performance, on data management, on Teradata Active System Management (ASM), and troubleshooting.

• Revised documentation of sar, TOP, awtmon, ampload, and dbschk.

• Reorganized chapter on managing space and on using, adjusting, and monitoring memory.

Teradata Database 13.0May 2009

Clarified documentation of TOP N optimizations.

4 Performance Management

Page 5: Teradata Database Performance Management

PrefaceAdditional Information

Additional Information

Teradata Database 13.0April 2009

Updated book for Teradata Database 13.0 performance features, including:

• Teradata Viewpoint.

• Query plans in Extensible Markup Language (XML).

• Enhancements to the use of join indexes.

• Partitioned Primary Index (PPI) enhancements.

• Support for No Primary Index (NoPI) tables.

• Enhancements to the ALTER TABLE statement.

• Enhancements to referential constraints.

• Performance enhancements to TOP N Row.

• Improvement to MiniCylPack.

• Clarification of FSG Cache calculations.

• Clarification of spool space accounting.

Removed:

• Documentation of performance enhancements that require no user action.

• References to Teradata MultiTool.

Release Description

URL Description

www.info.teradata.com/ Use the Teradata Information Products Publishing Library site to:

• View or download a manual:

1 Under Online Publications, select General Search.

2 Enter your search criteria and click Search.

• Download a documentation CD-ROM:

1 Under Online Publications, select General Search.

2 In the Title or Keyword field, enter CD-ROM, and click Search.

• Order printed manuals:

Under Print & CD Publications, select How to Order.

www.teradata.com The Teradata home page provides links to numerous sources of information about Teradata. Links include:

• Executive reports, case studies of customer experiences with Teradata, and thought leadership

• Technical information, solutions, and expert advice

• Press releases, mentions and media resources

Performance Management 5

Page 6: Teradata Database Performance Management

PrefaceAdditional Information

To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document. Please e-mail: [email protected].

www.teradata.com/t/TEN/ Teradata Customer Education designs, develops and delivers education that builds skills and capabilities for our customers, enabling them to maximize their Teradata investment.

www.teradataatyourservice.com Use Teradata @ Your Service to access Orange Books, technical alerts, and knowledge repositories, view and join forums, and download software patches.

developer.teradata.com/ Teradata Developer Exchange provides articles on using Teradata products, technical discussion forums, and code downloads.

URL Description

6 Performance Management

Page 7: Teradata Database Performance Management

Table of Contents

Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Supported Software Releases and Operating Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Changes to this Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

Chapter 1: Basic System Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Benefits of Managing Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Basic System Management Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Data Collection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

System Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Workload Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Capacity Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Application Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Operational Excellence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Activities Supporting BSMP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Data Monitoring and Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Using Teradata Viewpoint to Monitor and Collect Data . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Using Resource Usage Data to Evaluate Resource Utilization. . . . . . . . . . . . . . . . . . . . . . 25

Using AMPUsage Data to Evaluate Workload Utilization. . . . . . . . . . . . . . . . . . . . . . . . . 25

Using DBQL Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Kinds of Database Query Log Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

DBQL Collection Standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Collecting Useful DBQL Historical Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Establishing Account IDs for Workload and User Group Mapping to DBQL Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Having Clear Performance Expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Establishing Remote Accessibility to the Teradata Support Center . . . . . . . . . . . . . . . . . . . . . 28

System Performance Resources and Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Performance Management 7

Page 8: Teradata Database Performance Management

Table of Contents

Chapter 2: Account String Expansion and System Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

Account String and ASE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

ASE Impact on PE and AMP Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

AMPUsage Logging with ASE Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Account Strings and Performance Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Priority Scheduler and Performance Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Charge Back. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33

Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33

Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34

Charge Back and Overhead. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34

Cleaning Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34

Chapter 3: Database Query Log Data and System Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

DBQL Logging Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

User Type 1: Short Subsecond Known Work (Tactical Work) . . . . . . . . . . . . . . . . . . . . . .37

User Type 2: Mostly Long Running Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38

User Type 3: Mostly Short Subsecond Requests with Occasional Long Running or Unknown Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38

Multiple Logging Best Practices for a Single User ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

Chapter 4: Collecting Resource Usage Data. . . . . . . . . . . . . . . . . . . . . .41

Resource Usage and Priority Scheduler Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

ResUsage and Cache Hit Rates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42

Normalized View for Coexistence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42

ResUsage and DBC.AMPUsage View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

ResUsage and Capacity Planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Using the ResNode Macro Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Observing Trends. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

ResUsage and Host Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

ResUsage and CPU Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

CPU Busy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

CPU Tasks in a DSS Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

CPU Tasks during Transaction and Batch Maintenance Processing . . . . . . . . . . . . . . . . .45

Parallel Node Efficiency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45

8 Performance Management

Page 9: Teradata Database Performance Management

Table of Contents

Parallel Disk Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

CPU Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

ResUsage and Disk Utilization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

ResUsage and BYNET Data Macro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Chapter 5: Collecting Other System Performance Data . . . . . . 49

Using the DBC.AMPUsage View. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

DBC.AMPUsage View and ASE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

AMPUsage and Resource Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Using Canary Queries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

System Canary Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Production Canary Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Chapter 6: Query Analysis Resources and Tools . . . . . . . . . . . . . . . 53

Query Analysis Resources and Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Query Capture Facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

QCF and SQL EXPLAINs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Target Level Emulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Performance Benefits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Teradata Visual EXPLAIN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Teradata System Emulation Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Performance Benefits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Teradata Index Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Index Wizard Support for Partitioned Primary Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Performance Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Teradata Statistics Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Performance Benefits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Chapter 7: System Performance and SQL . . . . . . . . . . . . . . . . . . . . . . . 59

CREATE TABLE/ALTER TABLE and Data Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Adjusting Table Size. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Reducing Number of Columns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Reducing Row Size. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Altering Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Performance Management 9

Page 10: Teradata Database Performance Management

Table of Contents

Altering Tables and Column Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61

DATABLOCKSIZE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61

FREESPACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63

ALTER TABLE Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63

TOP N Row Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63

Performance Optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63

Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63

Recursive Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64

Ways to Specify a Recursive Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64

Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64

CASE Expression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65

Effects on Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65

Valued and Searched CASE Expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65

Analytical Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68

Analytical Functions and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68

Random Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .69

Random Stratified Sampling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70

Distincts and Multiple Aggregate Distincts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70

Data in Partition By Column and System Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70

Unique Secondary Index Maintenance and Rollback Performance. . . . . . . . . . . . . . . . . . . . . .71

Nonunique Secondary Index Rollback Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .71

Bulk SQL Error Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .72

Optimized INSERT . . . SELECT Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .72

Empty Table INSERT . . . SELECT Requests and Performance. . . . . . . . . . . . . . . . . . . . . .72

INSERT . . . SELECT Into an Empty SET Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73

INSERT . . . SELECT with FastLoad. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74

INSERT . . . SELECT with Join Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74

Array Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74

Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75

Aggregate Cache Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75

Considerations Before Increasing Aggregate Cache Size . . . . . . . . . . . . . . . . . . . . . . . . . . .75

Request Cache Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75

Optimized DROP Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76

Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76

Parameterized Statement Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76

IN-List Value Limit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77

Reducing Row Redistribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77

Extracting Combinations of Join Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77

Using the BETWEEN Clause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78

Merge Joins and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79

Merge Joins and Nested Join. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79

10 Performance Management

Page 11: Teradata Database Performance Management

Table of Contents

Merge Join with Covering NUSI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Hash Joins and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Hash Join Costing and Dynamic Hash Join . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Referential Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Benefits of RI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Overhead Cost of RI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Join Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Referential Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Standard RI and Batch RI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Secondary Indexes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Secondary Indexes and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Using NUSIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

NUSIs and Block Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Index Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Index Access Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Join Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Performance and Covering Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Multitable Noncovering Join Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Covering Bind Terms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Using Single-Table Join Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Defining Join Indexes with Outer Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Defining Join Indexes with Inequality Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Defining Join Indexes on UDT Columns and Expressions . . . . . . . . . . . . . . . . . . . . . . . . 89

Refreshing Join Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Using Aggregate Join Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Join Indexes and the Optimizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Protecting a Join Index with Fallback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Join Indexes and Collecting Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Cost Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Join Index Versus NUSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Join Index Optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Sparse Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Performance Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Joins and Aggregates On Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Views and Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Operations Available with Joins and Aggregates on a View. . . . . . . . . . . . . . . . . . . . . . . . 95

Partial GROUP BY and Join Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Large Table/Small Table Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Star Join Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Star Join Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Performance Value. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Partitioned Primary Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Performance Management 11

Page 12: Teradata Database Performance Management

Table of Contents

Nonpartitioned Primary Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99

Partitioned Primary Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99

Performance Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100

Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100

Multilevel Partitioned Primary Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101

Partition-Level Backup, Archive, Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101

Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101

Collecting Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101

Collecting Statistics and the Optimizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102

How the Optimizer Obtains Necessary Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102

AMP Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103

Frequency Distribution of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103

COLLECT STATISTICS Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103

Statistics on Skewed Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .104

Collecting Statistics for Join Index Columns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .105

Statistics Collection on Sample Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .105

PARTITION Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .105

Sampled Statistics Usage Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .106

Stale Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .106

Circumstances that Cause Stale Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .107

Refreshing Stale Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .107

EXPLAIN Feature and the Optimizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .108

EXPLAIN Output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .109

Using EXPLAIN to Detect Data Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .110

Identity Column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113

Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113

Disadvantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113

Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113

2 PC Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114

Performance Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114

Updatable Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114

Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114

Non-Key Access Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115

Chapter 8: Database Locks and Performance . . . . . . . . . . . . . . . . . .117

Locking Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117

Locking Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117

Guidelines for Preventing Excessive Deadlocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .118

Locking Logger and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .118

12 Performance Management

Page 13: Teradata Database Performance Management

Table of Contents

Transaction Rollback and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Effects on Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Rollback Priority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Minimizing or Avoiding Rollbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Detecting a Rollback in Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Example 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

Chapter 9: Data Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Data Distribution Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Identifying Uneven Data Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Using SQL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Using Teradata Viewpoint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Using Hash Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Impact of Uneven Data Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Check Periodically for Skewed Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Sample Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Parallel Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Join Processing and Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Referential Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Performance Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Primary Index and Row Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Effects of Skewed Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Performance Impact of a Primary Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

No Primary Index Table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Hash Bucket Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Chapter 10: Managing Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Tracking Data Space Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Running Out of Disk Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Low Cylinder Utilization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Frequently Updated Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Performance Degradation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Running Out of Free Cylinders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Ensuring Optimal Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Automatic Processes for Managing Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Freeing Cylinders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Performance Management 13

Page 14: Teradata Database Performance Management

Table of Contents

AutoCylPack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .132

MiniCylPack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .133

Defragmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .135

Freeing Space on Cylinders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .136

FreeSpacePercent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .137

Determining a Value for FSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .137

Operations Honoring the FSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .139

Operations that Disregard FSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .139

PACKDISK and FreeSpacePercent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .140

Running Other Utilities with PACKDISK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .140

Cylinder Splits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141

PACKDISK, FSP, and Cylinder Allocations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141

Data Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141

Multi-Value Compression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .142

Algorithmic Compression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .143

Choosing a Compression Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .143

Managing Data Block Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .143

Maximum Data Block Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .143

Minimum Data Block Allocation Unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .145

Datablock Merge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .145

Journal Data Block Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146

Adding Disk Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146

Managing Spool Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146

Spool Space and Perm Space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146

Spool Space Accounting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147

Increasing Spool Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147

Using Spool Space as a Trip Wire. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147

File System Fault Isolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147

Chapter 11: Using, Adjusting, and Monitoring Memory . . . . . .149

Using Memory Effectively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .149

Shared Memory Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .149

Memory Size. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .151

Reserving Free Memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .151

Monitoring Memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .151

Free Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152

ResUsage and Available Free Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152

Adjusting for Low Available Free Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152

Assured Minimum Non-FSG Cache Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152

14 Performance Management

Page 15: Teradata Database Performance Management

Table of Contents

Potential Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

Query Row Redistribution Memory Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Redistribution Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

FSG Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Space in FSG Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Calculating FSG Cache Size Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Cylinder Slots in FSG Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Calculating FSG Cache Read Misses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Using Memory-Consuming Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

File System and ResUsage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Managing I/O with Cylinder Read . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Cylinder Read Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Viewing the Cylinder Slot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Cylinder Read and I/O. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Using Cylinder Read for WAL Log Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Tracking Cylinder Read Resource Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Chapter 12: Performance Tuning and DBS Control Utility . . 161

DefragLowCylProd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

DictionaryCacheSize. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

FreeSpacePercent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

DisableMergeBlocks and MergeBlockRatio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

HTMemAlloc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

IdCol Batch Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

PermDBAllocUnit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

PermDBSize. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

PPICacheThrP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

RedistBufSize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

RollbackPriority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

SkewAllowance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Chapter 13: Teradata Active System Management . . . . . . . . . . . 169

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

Areas of Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Conceptual Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Performance Management 15

Page 16: Teradata Database Performance Management

Table of Contents

Teradata ASM Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .171

Following a Request in Teradata ASM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .172

Chapter 14: Optimizing Workload Management . . . . . . . . . . . . . . .175

Using Query Band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .175

Priority Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .175

Design Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .176

Chapter 15: Performance Reports and Alerts . . . . . . . . . . . . . . . . . .177

Using Performance Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177

Some Symptoms of Impeded System Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177

System Saturation and Resource Bottlenecks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177

Processing Concurrency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .178

Deadlocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .178

Measuring System Conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .178

Using Alerts to Monitor the System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .180

Suggested Alerts and Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .180

Weekly and/or Daily Reports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .181

Detecting Resource-Intensive Queries through Scripts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .182

Sample Script: High CPU Use. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .182

Sample Script: Active AMP Users with Bad CPU/IO Access Ratios . . . . . . . . . . . . . . . . .183

Sample Script: Hot / Skewed Spool in Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .184

Chapter 16: Baseline Benchmark Testing . . . . . . . . . . . . . . . . . . . . . . .185

Benchmark Test Suite Defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .185

Tips on Baseline Benchmarking Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .185

Baseline Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .186

Performance Metrics for Baseline Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .186

Chapter 17: Real-Time Tools for Monitoring System Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .187

Teradata Viewpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .187

16 Performance Management

Page 17: Teradata Database Performance Management

Table of Contents

Teradata Utilities and System Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

ctl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Query Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Show Locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Query Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Gateway Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

SHOWSPACE Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

TDP Monitoring Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

System Activity Reporter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

TOP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

BYNET Link Manager Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Interpreting the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

AWT Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Performance Benefit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

DBC.ResUsageSAWT Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

ampload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Differences Between ampload and awtmon Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Resource Check Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

CheckTable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Index Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Compressed Value Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Performance on Large Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Workload Management APIs and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

PM/APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

Open APIs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

Chapter 18: Troubleshooting Teradata Database Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Busy Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

CPU Saturation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Suggested Monitoring Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

High I/O Wait. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

Looking for the Bottleneck in Peak Utilization Periods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

Job Scheduling Around Peak Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

Teradata ASM and Concurrency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Managing I/O-Intensive Workloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Preventing Slow or Hang Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Handling Blocked Internal DBQL Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Canceling Rollbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Performance Management 17

Page 18: Teradata Database Performance Management

Table of Contents

Noticing a Slow Rollback of an ALTER TABLE Statement . . . . . . . . . . . . . . . . . . . . . . . .201

Skewed Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .202

Tables Prone to Skewing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .202

Possible Causes of Skewing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .203

Controlling Session Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .203

Identifying and Handling Resource-Intensive Queries in Real Time . . . . . . . . . . . . . . . . . . .204

Characteristics of a Resource-Intensive Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .204

Ensuring Parallel Node Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .205

AMP Efficiency and Balanced Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .206

Tools for Analyzing Lock Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .206

Appendix A: Performance and Database Redesign . . . . . . . . . . .207

Database Design Phases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .207

After Initial Design Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .208

Adding New Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .208

Shortcut Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .208

Caveats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .209

Appendix B: Performance and Capacity Planning . . . . . . . . . . . . .211

Solving Bottlenecks by Expanding Your Teradata Database Configuration . . . . . . . . . . . . . .211

Determining Resource Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .211

Adding Disk Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .212

Adding Vprocs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .212

Adding Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .213

Adding Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .214

Reconfiguring Your Teradata Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .214

Scaling Your Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .214

Performance Considerations When Upgrading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .215

Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .215

Appendix C: Performance Tools and Resources . . . . . . . . . . . . . . . .217

Performance Monitoring Tools and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .217

System Components and Performance Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .220

18 Performance Management

Page 19: Teradata Database Performance Management

Table of Contents

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Performance Management 19

Page 20: Teradata Database Performance Management

Table of Contents

20 Performance Management

Page 21: Teradata Database Performance Management

CHAPTER 1 Basic System Management

This chapter provides an introduction to Basic System Management Practices (BSMP).

Benefits of Managing Performance

Managing system performance provides important benefits:

• Maintaining efficient use of existing system resources

Managing the use of existing system resources includes, among other things, job mix tuning and resource scheduling to use available idle cycles effectively.

Managing resources to meet pent-up demand that may peak during prime hours ensures that the system operates efficiently to meet workload-specific goals.

• Helping identify system problems

Managing performance helps system administrators identify system problems.

Managing performance includes, among other things, monitoring system performance through real-time alerts and by tracking performance historically. Being able to react to changes in system performance quickly and knowledgeably ensures the efficient availability of the system. Troubleshooting rests on sound system monitoring.

• Aiding in capacity planning

If performance degradation is a gradual consequence of increased growth or higher performance expectations, all data collected over time can be used for capacity or other proactive planning. Since the onset of growth-related performance degradation can often be insidious, taking measurements and tracking both data and usage growth can be very useful.

Managing performance yields efficient use of existing system resources and can guide capacity planning activities along sound and definable lines.

For a discussion of capacity planning, see Appendix B: “Performance and Capacity Planning”.

Basic System Management Practices

The following figure illustrates Basic System Management Practices (BSMP):

Performance Management 21

Page 22: Teradata Database Performance Management

Chapter 1: Basic System ManagementBasic System Management Practices

Data Collection

As the figure shows, data collection is at the center of any system performance practice. Data collection supports the following specific performance management tasks:

System Performance

System performance entails the management of system resources, such as CPU, the I/O subsystem, memory, BYNET traffic, host network traffic (for example, channel or TCP-IP networks).

Reports and queries based on the standard resource usage (ResUsage) tables identify system problems, such as imbalanced or inadequate resources. An imbalanced resource is often referred to as skewed. Because Teradata Database is a parallel processing system, it is highly dependent upon balanced parallel processing for maximum throughput. Any time the system becomes skewed, the throughput of the system is reduced. Thus, the prime objective in all Teradata Database performance management is to balance for maximum throughput.

Inadequate resources refers to an operating condition that has caused some resource to become saturated. One example of a saturated resource is node-free memory reduced to a critically low point during high concurrent user activity. This results in system slowdowns.

Another example of a saturated resource is high network traffic in a host channel interface causing a node to become skewed and, as a result, creating an imbalance in system process. Such a skewed node is called a hot node.

1097A001

22 Performance Management

Page 23: Teradata Database Performance Management

Chapter 1: Basic System ManagementActivities Supporting BSMP

Workload Management

Workload management entails balancing the workload, managing concurrency levels, exorcising control over what work is allowed to run, and determining and tuning priorities.

Reports and queries based on the Database Query Log (DBQL) and AMPUsage data identify opportunities to improve response time consistency.

These can be realized by using Teradata Active System Management (ASM) filter, throttle, and workload definition rules set and monitored in the Teradata Viewpoint Workload Designer portlet.

Capacity Planning

Capacity planning entails analyzing existing workload historical trends in order to extrapolate future capacity needs.

Application Performance

Application performance entails managing both application and query performance.

Reports and queries based on the DBQL and AMPUsage data identify heavy resource usage queries as well as candidates for query tuning.

Teradata Viewpoint enables you to see where there is heavy resource usage by monitoring both system-wide metrics and the performance of individual queries on the Teradata Database.

Operational Excellence

Data collection and the four specific system management tasks support efforts to achieve database operational excellence.

Activities Supporting BSMP

The following management activities support BSMP:

• Data monitoring and collection, including system monitoring using Teradata Viewpoint

• Having clear performance expectations

• Establishing remote access to Teradata Support Center for troubleshooting

Data Monitoring and Collection

Monitoring and collecting data provides valuable information for:

• Performance analysis and tuning, including application tuning and system optimization.

• Workload management, which entails resource distribution and workload fairness.

• Performance monitoring., including anomaly identification and troubleshooting.

Performance Management 23

Page 24: Teradata Database Performance Management

Chapter 1: Basic System ManagementData Monitoring and Collection

• Capacity planning, which entails identifying the fitness of the system to handle workload demand and data and workload growth.

Data collection should be done for key user groups and for key applications. Moreover, it should be collected in real-time and historically.

Using Teradata Viewpoint to Monitor and Collect Data

Teradata Viewpoint includes several portlet bundles that enable you to monitor and collect data, including

• Self-service portlets: these include portlets that:

• Provide graphical display of current canary query responses.

• Provide information on query and session statistics.

• Enable execution and tracking of SQL statements.

• Indicate the health of system performance.

• Management portlets: these include portlets that:

• Provide interactive tools for analyzing system metrics over user-definable periods of time.

• Provide information on presetable lists of workload filters.

• Provide graphical representation of system metrics over time.

• Allow users to run queries against the database, monitor their progress, and see the results

• Display table locking information.

• Display trend reporting, including current versus historical trends with respect to key system metrics.

• Monitor data with respect to day-to-day windows, which include seasonal variations, end of season, month, or week processing, Monday morning batch (beginning of the week) processing, or weekend business peeks:

• Monitor data with respect to within-a-day windows

• Monitor data by workload, which includes by application (for example, by tactical queries, strategic queries, predefined and cyclical reporting) and by user area (for example, by web user, IT user, power user or ad hoc user, application developer, customer, partner, business divisions)

• Track the number of users using the system to help identify concurrency levels of logged on sessions and active sessions.

• Allow Database Administrators (DBAs) to set thesholds to identify problem queries.

• Allow DBAs to execute certain utilities against the database.

• Enable monitoring and managing of Teradata Database disk space usage.

• Display real-time, bucketized daily throughput statistics. For example, the number of queries completed in the last hour, or today, and so on.

• Display Teradata Database alerts.

24 Performance Management

Page 25: Teradata Database Performance Management

Chapter 1: Basic System ManagementData Monitoring and Collection

• Enable systems analysis using metrics from multiple Teradata Database systems displayed in a single view.

• Display data about such resources such as CPU, vprocs, AMP Worker Tasks (AWTs) with problematic areas highlighted for analysis.

• Monitor data about all nodes and vprocs in the system and identify any skewed resources.

• Teradata Multisystem Management (TMSM) portlets: these include portlets that:

• Show status of applications across “ecosystems” and alerts for applications resources. Ecosystems refer to a physical site or data center; the term identifies the subset of the site’s components that support Teradata system-based applications.

• Display alerts affecting the ecosystem.

• Show alerts over time for process resources.

• Show events and alerts across processes that are part of the same unit of work.

• Teradata ASM portlets: these include portlets that:

• Monitor the health of workloads using key indicators.

• Enable the flow of work to be tracked with respect to resource allocation and problem isolation.

• Enable filters, throttles, workload definitions, exceptions, exceptions actions, service level goals (SLGs) to be set and tracked.

For a general description of Teradata Viewpoint, see “Teradata Viewpoint” on page 187. For more information about Teradata Viewpoint, visit http://www.teradata.com/t/tools-and-utilities/database-management/Viewpoint/

Using Resource Usage Data to Evaluate Resource Utilization

You can use ResUsage usage data to see, for example, the details of system-wide CPU, I/O, BYNET, and memory usage. ResUsage data is point-in-time data.

ResUsage data provides a window on:

• Time-based usage patterns and peaks

• Component utilization levels and bottlenecks. That is, any system imbalance

• Excessive redistribution

For information on collecting and using ResUsage data, see Resource Usage Macros and Tables.

Using AMPUsage Data to Evaluate Workload Utilization

You can use AMPUsage data to evaluate CPU and I/O usage by workload and by time.

Such information provides a tuning opportunity. You can tune the highest consumer in the critical window so that CPU usage yields the highest overall benefit to the system.

For information on collecting AMPUsage data, see Chapter 5: “Collecting Other System Performance Data.”

Performance Management 25

Page 26: Teradata Database Performance Management

Chapter 1: Basic System ManagementData Monitoring and Collection

Using DBQL Tables

You can use DBQL tables to collect and evaluate:

• System throughput

• Response time

• Query details, such as query step level detail, request source, SQL, answer size, rejected queries, resource consumption

• Objects accessed by the query

Such information enables you to:

• Identify a workload that does not meet response time Service Level Goals (SLGs)

• Drill down into DBQL after targeting workloads using:

Kinds of Database Query Log Tables

Listed below, from the point of view of BSMP, are the two DBQL master tables and the kind of data they provide.

The following DBQL tables provide detailed performance information.

This resource... To identify details about...

AMPUsage Top consumers

Spool Log High spool users

The following... Provides...

DBQLogTbl Data on individual queries, including:

• Query origination, start, stop, and other timings

• CPU and logical I/O usage

• Error codes

• SQL (truncated)

• Step counts

DBQLSummaryTbl A summary of short-running queries.

For high-volume queries, it provides query origination, response time summaries, query counts, and CPU and logical I/O usage.

The following... Provides...

DBQLStepTbl Query step timings, CPU and I/O usage, row counts, and step actions, among other things

26 Performance Management

Page 27: Teradata Database Performance Management

Chapter 1: Basic System ManagementHaving Clear Performance Expectations

DBQL Collection Standards

• All usage should be logged in DBQL 24x7.

• Flush the DBQL caches at default rate of 10 minutes.

• Recommend logging usage in DBQL at the account string level:

Recommendation for account string setup is as follows: $H1$WXYZ&S&D&H, where:

• $H1$ = Performance Group (PG)

• WXYZ = Work Group Identifier

• &S&D&H = ASE variables - session, date, and hour

Collecting Useful DBQL Historical Data

DBQL History summaries key DBQL data to 1 row per user / account / application ID / client ID per time period. Teradata recommends retaining 13 months of copied detail. That is, Teradata recommends copying detail to another table daily. You should delete the source of copied detail daily to keep online summary collection efficient.

For DBQL logging best practices, a table showing the relationship between DBQL temporary tables and DBQL history tables, daily and monthly DBQL maintenance processes and sample maintenance scripts, CREATE TABLE statements for DBQL temporary tables and DBQL history tables, and DBQL maintenance macros, see Database Administration.

Establishing Account IDs for Workload and User Group Mapping to DBQL Tables

You should establish account IDs for workload and user group mapping to DBQL tables in order to collect time data.

For details on using ASE and LogonSource, see Database Administration.

For information on DBQL, see Database Administration.

Having Clear Performance Expectations

It is important to understand clearly the level of performance your system is capable of achieving. The system configuration consists of finite resources with respect to CPU and disk bandwidth.

DBQLObjTbl Usage information about database objects such as tables, columns, indexes, and databases.

DBQLSqlTbl The complete SQL text.

DBQLExplainTbl EXPLAIN output.

DBQLXMLTbl Query plans in Extensible Markup Language (XML).

The following... Provides...

Performance Management 27

Page 28: Teradata Database Performance Management

Chapter 1: Basic System ManagementEstablishing Remote Accessibility to the Teradata Support Center

Moreover, your system configuration can be further limited by performance trade-offs with respect to coexistence, where a small percentage of these resources are essentially unusable due to coexistence balancing strategies.

It is important to understand the performance expectations of your configuration, including how expectations change as the CPU to I/O balance of your workload changes throughout the day or week or month.

Establishing Remote Accessibility to the Teradata Support Center

Use the AWS to establish remote accessibility to Teradata Support Center. Remote accessibility makes it possible for the Teradata Support Center to troubleshoot system performance issues.

System Performance Resources and Documents

For specific system performance information, see the following:

• Teradata Viewpoint Online Help

• Teradata Active System Management: High Level Architectural Overview

• Teradata Active System Management: Usage Considerations & Best Practices

• Teradata Workload Analyzer: Architectural Overview

• Teradata Active System Management Tips and Techniques

• Using Teradata’s Priority Scheduler

• Understanding AMP Worker Tasks

• Practical Tips for Using the New ResUsage Tables

• Partitioned Primary Index Usage

Additional Information

For more information on... See...

Data collection Teradata Professional Services Data Collection Service

Workload management Teradata Professional Services Workload Optimization Service and Workload Management Workshop

Application performance Teradata Professional Services Application Performance Service and DBQL Workshop

28 Performance Management

Page 29: Teradata Database Performance Management

Chapter 1: Basic System ManagementAdditional Information

Capacity planning Teradata Professional Services Capacity Planning Service

System performance Teradata Customer Services System Performance Service

For more information on... See...

Performance Management 29

Page 30: Teradata Database Performance Management

Chapter 1: Basic System ManagementAdditional Information

30 Performance Management

Page 31: Teradata Database Performance Management

CHAPTER 2 Account String Expansion andSystem Performance

This chapter describes collecting data using account strings and the Account String Expansion (ASE) feature to provide information on system performance.

Account String and ASE

Each time the system determines that a new user/account string is active, the system begins collecting AMP usage and I/O statistics. The system stores the accumulated statistics for a user/account string pair as a row in the DBC.AMPUsage view. Each user/account string pair results in a new set of statistics and an additional row.

You can use the information in the DBC.AMPUsage view for capacity planning or in charge back and accounting software. ASE uses the AMPUsage mechanism, but by adding in the substitution variables, the amount of information recorded can greatly increase for the purpose of performance analysis or capacity planning.

You can specify the measurement rate by date (&D), time (&T), or a combination of both. Information can be written to AMPUsage based on the time the user logged on (&L). It can be directed to generate a row for each user, each session, or for an aggregation of the daily activities of the user.

At the finest granularity, ASE can generate a summary row for every SQL request, allowing precise measurements an individual SQL statement execution.

ASE Impact on PE and AMP Performance

• ASE has little impact on PE performance. The cost incurred for analyzing the account string amounts to only a few microseconds.

• ASE may have impact on AMP performance.

The AMP has the burden of additional DBC.AMPUsage logging. Depending on the number of users and the ASE options selected, the added burden may vary from very slight to enough to degrade performance. In general, the &D, &H, and &L options do not have major effects on performance.

Be cautious where you use the &T option. It can generate an AMPUsage row for virtually every Teradata SQL request. The &T option can have a much greater effect on performance. Therefore, do not use the &T option:

• In default account ID strings

Performance Management 31

Page 32: Teradata Database Performance Management

Chapter 2: Account String Expansion and System PerformanceAccount Strings and Performance Groups

• In conjunction with tactical queries

• With TPump

The &T option should not be a problem for long-running DSS requests, but could be a performance issue if users are running numerous small requests. &T is site-dependent; it should generate no more than 10 to 20 AMPUsage rows per minute.

Note: Because ASE causes the system to write more entries to DBC.AMPUsage, you must manage the table more often.

For more information on ASE, see Database Administration.

AMPUsage Logging with ASE Parameters

AMPUsage logging may have both performance and data storage impacts.

The following table summarizes potential impact.

Account Strings and Performance Groups

A Performance Group (PG) is explicitly specified for every account string that is created, except for DBC and SYSTEMUSERINFO. The account strings for these user IDs cannot be changed.

Priority Scheduler and Performance Groups

Priority Scheduler (PS) manages the allotment of system resources among concurrently running transactions based on their defined relative priority. This is accomplished through assigning each request to a default PG.

The definition of the PG within PS determines how that request obtains CPU resources over the course of the execution of the request.

ASE Parameter Performance Impact Data Capacity Impact

None Negligible 1 row per account per AMP

&D Negligible 1 row per account per day per AMP

&H Negligible 1 row per account per hour per AMP

&D&H Negligible 1 row per account per hour per day per AMP

&L Negligible 1 row per session pool

&I Negligible 1 row per SQL request

&S Negligible 1 row per session

&T Potentially non-negligible 1 row per query per AMP

32 Performance Management

Page 33: Teradata Database Performance Management

Chapter 2: Account String Expansion and System PerformanceCharge Back

Enabled Teradata ASM Workloads

When Teradata ASM workload definition (Category 3) is enabled, the Priority Scheduler PG portion of the account string is ignored for purposes of priority assignment. Instead, workload classification determines the priority of the request.

However, for the small amount of time that the first request of a session is being parsed prior to classification into a workload, the PG within the account string is used to establish priority.

For information on Teradata ASM workload definitions, see Chapter 14: “Optimizing Workload Management.”

Charge Back

ASE can be used to implement a simple charge back utility. It provides a way to determine how much system CPU time and disk activity is consumed by a user each day. By adding a few special characters to a user's account name, you can extract detailed information from system tables about what that user has done. Teradata Database expands these special characters into such things as the session number, request number, date or time when the account name is written to a system table.

At the completion of each SQL step, Teradata Database always updates the DBC.Acctg table with statistics about the request. These statistics include the total CPU time and number of disk I/Os used by the request. This statistical information is summarized by adding it to an existing row that contains the same username and account name.

Configuration

The following example modifies the account string of each user you want to track. You preface each account string with the text CB&D. You can add any additional account information after these four characters if you wish.

When you add a date to the account name, the account name effectively changes each day and a new row is written to the DBC.Acctg table. This row contains the total number of CPU seconds and total number disk I/Os for each request that was submitted on that date.

The &D is an ASE token that expands to the current date in the format YYMMDD. CB is an arbitrary text string that you chose to indicate that this account is being tracked for charge back.

You can modify an existing account string for a user using the following SQL command:

MODIFY USER JANETJONES AS ACCOUNT = ('CB&D');

Note: Priority control characters ($R, $H, $M, $L, and so on), if used, must be the first characters in the account string. An example of an account string that contains priority control and account string expansion would be: $MCB&D. The SQL query examples below would need to have the SUBSTR functions modified to account for the new offset of the ASE information.

Performance Management 33

Page 34: Teradata Database Performance Management

Chapter 2: Account String Expansion and System PerformanceCharge Back

Example

SELECT ACCOUNTNAME, USERNAME, SUM(CPUTIME), SUM(DISKIO) FROM DBC.AMPUSAGEWHERE SUBSTR(ACCOUNTNAME, 1, 2) = 'CB'GROUP BY USERNAME, ACCOUNTNAMEORDER BY USERNAME, ACCOUNTNAME;

*** Query completed. 11 rows found. 4 columns returned. *** Total elapsed time was 2 seconds.

AccountName UserName Sum(CpuTime) Sum(DiskIO)-------------- ------------- ------------------ ---------------CB060902 JANETJONES 1,498.64 3,444,236CB060903 JANETJONES 934.23 1,588,764CB060904 JANETJONES 883.74 924,262CB060905 JANETJONES 214.99 200,657CB060902 JOHNSMITH 440.05 396,338CB060903 JOHNSMITH 380.12 229,730CB060904 JOHNSMITH 112.17 184,922CB060905 JOHNSMITH 56.88 99,677CB060902 SAMOREILLY 340.34 410,178CB060903 SAMOREILLY 70.74 56,637CB060902 WEEKLY 3,498.03 7,311,733If we wanted to charge $0.25 per CPU second and bill for the month of September 2006, we could use the following query to generate the bill:

SELECT USERNAME, SUM(CPUTIME)*0.25 (FORMAT '$$ZZZ,ZZZ,ZZ9.99')FROM DBC.AMPUSAGEWHERE SUBSTR(ACCOUNTNAME, 1, 6) = 'CB0609'GROUP BY 1ORDER BY 1WITH SUM(CPUTIME)*0.25 (FORMAT '$$ZZZ,ZZZ,ZZ9.99', TITLE 'Grand Total:'); *** Query completed. 4 rows found. 2 columns returned. *** Total elapsed time was 2 seconds.

UserName (Sum(CpuTime)*0.25)------------------------------ -------------------JANETJONES $882.90JOHNSMITH $247.33SAMOREILLY $102.77WEEKLY $874.51 ------------------- Grand Total: $2,107.51

Charge Back and Overhead

From a CPU perspective charge back entails very little overhead. The accounting table is already being updated at the completion of each statement. The only cost is the creation of a new row in the table for each user each day. From a space perspective, the accounting table will grow by one row for each user each day. Periodic cleanup can constrain this growth.

Cleaning Up

You should periodically remove old information from the DBC.Acctg table. For example, the following command will delete entries for September 2006:

34 Performance Management

Page 35: Teradata Database Performance Management

Chapter 2: Account String Expansion and System PerformanceCharge Back

DELETE FROM DBC.ACCTG WHERE SUBSTR(ACCOUNTNAME, 1, 6) = 'CB0609';

Performance Management 35

Page 36: Teradata Database Performance Management

Chapter 2: Account String Expansion and System PerformanceCharge Back

36 Performance Management

Page 37: Teradata Database Performance Management

CHAPTER 3 Database Query Log Data andSystem Performance

Database Query Log (DBQL) provides a series of predefined tables and views that can store historical records of queries and their duration, performance, and target activity based on rules you specify.

You can use DBQL to log query processing activity, including:

• Capturing query/statement counts and response times.

• Discovering potential application improvements.

• Making further refinements to workload groupings and scheduling.

• Having SQL text and processing steps analyzed.

• Logging optimizer query plans as XML documents.

DBQL is flexible enough to log information on the variety of SQL requests, from short transactions to longer-running analysis and mining queries, that run on Teradata Database. In addition, DBQL provides key insights into other aspects of a query such as whether it was aborted or delayed, what was its start and end time, and so on.

Because DBQL operates asynchronously, the logging activity has a much lower impact on the overall response time of given transactions. Furthermore, DBQL writes its information to internal memory buffers or caches. These are flushed to disk when the buffer is full or at the time indicated by the DBS Control Record field DBQLFlushRate. The default rate is every 10 minutes for normal operation, but the Database Administrator (DBA) can change the rate to be more frequent to analyze performance issues.

For information on DBQLFlushRate, see Utilities.

For information on how to use DBQL, see Database Administration.

DBQL Logging Best Practices

Teradata recommends logging the three types of users are follows:

User Type 1: Short Subsecond Known Work (Tactical Work)

• Log as Summary

• BEGIN QUERY LOGGING LIMIT SUMMARY = 1,3,5 ON ALL ACCOUNT = 'acctname';

The numbers 1,3,5 are clock seconds not CPU seconds

• No SQL gets logged

Performance Management 37

Page 38: Teradata Database Performance Management

Chapter 3: Database Query Log Data and System PerformanceDBQL Logging Best Practices

• No Objects get logged

For this workload, high-speed performance and minimal response time are the primary objectives. Typically, this workload tends to be very predictable in nature with queries typically designed to be single AMP retrievals.

For this workload, capturing information at the request level may be unnecessary for two reasons:

• The transactions are well-tuned, known, and repeated over and over again.

• The additional overhead required to record SQL for each request would represent a meaningful portion of the overall work performed on behalf of the transaction, that is, the additional overhead could materially impact request response time.

The objective in this case is to capture only summarized information about these SQL requests. Since the expectation for this workload type is that the work is predictable, repeatable and does not vary much, summary logging is sufficient. If however, there could be unauthorized use of this ID, or on occasion, a longer running query is run, threshold logging could be used.

User Type 2: Mostly Long Running Work

• Log detail with SQL and objects

• BEGIN QUERY LOGGING WITH SQL, OBJECTS LIMIT SQLTEXT =0 ON ALL ACCOUNT = 'acctname';

If there are 10s of thousands of subsecond work, additional overhead will be incurred

Teradata recommends for this workload category that a high degree of detailed data be captured for analysis. In fact, the data generated from this DBQL logging option generates the critical detailed information needed to perform effective performance management and tuning.

The recommended level of logging ensures that the entire SQL text for each request is captured along with the objects that were used in processing the request. The SQL text and Object data is critical in performing query access path analysis. The DBQL data provided in the main detail data provides not only CPU and IO data, but also the data to calculate whether a query is skewed, does a large scan, or has the characteristics of a large product join, the keys to high impact performance tuning. While the volume of this level of logging may appear to be excessive, the minimal cost of the overhead combined with the volume of queries in these workload categories makes this level of logging acceptable. Experience shows that other comparable Teradata Database production environments have been logging a similar volume of queries with negligible overhead.

User Type 3: Mostly Short Subsecond Requests with Occasional Long Running or Unknown Work

• Log Threshold

• BEGIN QUERY LOGGING WITH SQL, OBJECTS LIMIT THRESHOLD =100 CPUTIME AND SQLTEXT =10000 ON ALL ACCOUNT = 'acctname';

38 Performance Management

Page 39: Teradata Database Performance Management

Chapter 3: Database Query Log Data and System PerformanceMultiple Logging Best Practices for a Single User ID

• The threshold number 100 represents hundredths of CPU seconds and causes queries that require more than one second of CPU time to be logged in DBQLSQLTbl, DBQLObjTbl, and DBQLogTbl with details. Queries with less than one second CPU time are summarized.

With threshold logging, DBQL cannot log to separate Explain and XML tables, even for those queries taking longer than the specified criteria. SQL, STEPINFO, and OBJECTS can be logged during threshold logging, even for those queries taking longer than the specified clock seconds.

For this workload type, it is assumed that millions of tactical known queries are running, but some unknown or long running request could occasionally run. It would in this case be appropriate to log at a threshold level in order to capture at a more detailed level the unknown or long running requests, while still logging the bulk of the work, the subsecond requests at a summary level.

Multiple Logging Best Practices for a Single User ID

There may be certain circumstances where a single user ID may have two different account strings. The need for two different account strings, however, is largely driven by a need to distinguish between workload categories.

As a result, although it may be possible for a single user ID to generate different levels of query logging detail, the level of detail generated should be consistent with the workload category identified. In other words, as long as the standard defined for logging remains consistent with the standard for user ids and account string definition, multiple account strings should present no problem.

Performance Management 39

Page 40: Teradata Database Performance Management

Chapter 3: Database Query Log Data and System PerformanceMultiple Logging Best Practices for a Single User ID

40 Performance Management

Page 41: Teradata Database Performance Management

CHAPTER 4 Collecting Resource Usage Data

This chapter describes specific topics for which collecting and using ResUsage data can help you:

• Measure system and system component performance.

• Identify potential performance impacts.

• Analyze performance degradation and improve.

• Identify problems such as bottlenecks, parallel inefficiencies, down components and congestion.

Resource Usage and Priority Scheduler Data

The ResUsageSps table allows you to see accumulated CPU, number of active processes, and other detail by Priority Scheduler Allocation Group (AG). It carries information that is similar to what is displayed in Priority Scheduler monitor output.

Information carried in the table is organized by:

• Collection date/time

• Node

• Vproc

• Performance Group

• Performance Period/AG

For those using Teradata ASM, each workload definition is the equivalent of one Performance Group in ResUsageSps.

The CpuTime and the IOBlks columns in ResUsageSps are collections of resource usage over the life of this particular logging interval.

No matter what your configuration size, care must be taken to manage what may be an exceptionally high volume of output from ResUsageSps table logging. Plan to offload ResUsageSps data frequently or turn on ResUsageSps logging at only limited times of the day, for example during the times of heaviest usage. To minimize the volume of the output, activate Active Row Filter Mode, when logging is enabled for ResUsageSps.

For ResUsageSps table column definitions, see Resource Usage Macros and Tables.

Performance Management 41

Page 42: Teradata Database Performance Management

Chapter 4: Collecting Resource Usage DataResUsage and Cache Hit Rates

ResUsage and Cache Hit Rates

ResUsage data can help determine cache hit rates by comparing total logical I/O requests to I/O requests that resulted in a physical I/O. Comparing these helps clarify any confusion that may arise when aligning DBQL data with ResUsage data, since DBQL data only tracks logical I/O.

An analogous circumstance occurs when attempting to differentiate between total BYNET message requests and BYNET message requests that result in a physical BYNET message being sent. Smaller systems see a greater difference between the two because of a higher percentage of point-to-point (PTP) messages with the sender and the receiver being the same node.

In ResUsageSpma table, a logical read count, that is, a count of the total number of requested disk segments for the interval, is represented in the FileAcqs column. The physical read count, or the number of requested disk segments that actually resulted in a read by the I/O subsystem, is represented in the AcqReads and the PreReads columns.

There are similar breakdowns for writes. BYNET logical messages are represented in the columns that begin with Msg, and the physical counterparts are represented with the columns that begin with NetMsg.

Normalized View for Coexistence

Teradata Database records both normalized and raw CPU measures. Normalized CPU time is derived by applying a CPU scaling factor to the node level raw CPU time.

The standard co-existence scaling factors for all node types are pre-defined in PDE startup parameter files in the Open PDE startup.txt file. The per-node values are added to the vconfig and tosgetpma() structures by PDE startup for use by other components. In this way, Teradata Database provides accurate performance statistics for mixed node systems, particularly with respect to CPU skewing and capacity planning, that is, usable capacity.

The formula is:

Node_Scaling_Factor * Node_CPU_Time

The scaling factor values are derived from:

• Measuring the raw capability of the Teradata compute node unconstrained by I/O or environment.

• Basing values on basic functionality, including data access rate, data load rate, row processing rate, raw transaction rate.

Currently, the 5100 is the base scaling factor value (1.00).

For AMP level reporting in DBQL and AMPUsage, the formula is:

Node_Scaling_Factor * AMP_CPU_Time

42 Performance Management

Page 43: Teradata Database Performance Management

Chapter 4: Collecting Resource Usage DataResUsage and DBC.AMPUsage View

ResUsage and DBC.AMPUsage View

The DBC.AMPUsage view displays CPU usage information differently than the way usage information is displayed in ResUsage data.

For more information on the DBC.AMPUsage view, see Data Dictionary.

ResUsage and Capacity Planning

Using the ResNode Macro Set

For capacity planning, generally only ResUsageSpma is required. This is the only table required to make use of the ResNode macro set.

Important information from ResNode includes:

• CPU utilization

• Parallel efficiency to show hot nodes or AMPs

• CPU to I/O balance

• OS busy versus DBS busy to see the characteristics of the workload

• Memory utilization

• Availability and process swapping (paging)

• Network traffic

Observing Trends

ResUsage data is most useful in seeing trends when there is a reasonably long history (more than a year) available for comparison. Use this data to answer questions, such as:

• How heavily is the system used at different times of the day or week?

• When are there peaks or available cycles in utilization?

This facility… Provides…

ResUsage metrics on the system, without making distinctions by individual user or account ID.

DBC.AMPUsage view AMP usage by individual user or account ID.

AMPUsage counts logical I/Os, not physical.

Some CPU used for the system cannot be accounted for in AMPUsage. Therefore, ResUsage CPU metrics will always be larger than AMPUsage metrics. Typically, AMPUsage captures about 70-90% of ResUsage CPU time.

Performance Management 43

Page 44: Teradata Database Performance Management

Chapter 4: Collecting Resource Usage DataResUsage and Host Traffic

ResUsage and Host Traffic

You can use the following macro to analyze the traffic flow between Teradata Database and the channel or network client.

For ResHostByLink columns, see Resource Usage Macros and Tables.

ResUsage and CPU Utilization

CPU Busy

The following macro reports CPU busy.

The above macro contain the Avg CPU Busy column, which is the average CPU utilization for all nodes. Avg CPU Busy % is a measure of how often multiple CPUs in a node were busy during a log period.

• In a DSS environment, a small number of jobs can easily bring the CPU close to 100% utilization.

• High CPU utilization consists of a Avg CPU Busy % of 70% over significant periods of time.

• The CPU is stressed differently in DSS and transaction processing environments.

For ResNode columns, see Resource Usage Macros and Tables.

CPU Tasks in a DSS Environment

The following lists how CPU tasks are carried out on the node during DSS operations.

1 Prepare for read:

• Memory management allocates memory for the data block.

• Database software communicates with the file system.

• File system communicates with the disk controller.

2 Qualify rows. Determine if the row satisfies the WHERE clause condition(s).

Macro Description Purpose

ResHostByLink By second averages General analysis

Macro Description Purpose

ResNode (System provided) by second averages

General system analysis

44 Performance Management

Page 45: Teradata Database Performance Management

Chapter 4: Collecting Resource Usage DataResUsage and CPU Utilization

Most DSS operations require full table scans in which the WHERE clause condition check is relatively time-consuming. Full table scans generally result from SQL statements whose WHERE clause does not provide a value for an index or partition elimination.

3 Process rows:

• Join

• Sort

• Aggregate

4 Format qualifying rows for spool output.

CPU Tasks during Transaction and Batch Maintenance Processing

The following describes how the CPU tasks are carried out on the node during a transaction batch maintenance processing.

Notice that the qualify rows activity is missing from the table. In transaction processing, it is more common for the WHERE clause to provide a value for the PI or USI. The read itself qualifies rows. Transaction processing typically avoids further conditional checks against non-indexed columns. All of these CPU tasks occur on the nodes.

1 Prepare for read:

• Memory management allocates memory for the data block.

• Database communicates with the file system.

• File system communicates with the disk controller.

2 Update row:

• Database locates row to be updated.

• Memory management allocates memory for the new data block to be built.

• Database updates the changed row and copies the old rows.

• Database communicates with the file system.

• File system communicates with the disk controller.

Parallel Node Efficiency

Parallel note efficiency is a measure of how evenly the workload is shared among the nodes. The more evenly the nodes are utilized, the higher the parallel efficiency.

Parallel node efficiency is calculated by dividing average node utilization by maximum node utilization. Parallel node efficiency does not consider the heaviness of the workload. It only looks at how evenly the nodes share that workload.

The closer parallel node efficiency is to 100%, the better the nodes work together. When the percentage falls below 100%, one or a few nodes are working much harder than the others in the time period. If node parallel efficiency is below 60% for more than one or two 10-minute log periods, Teradata Database is not getting the best performance from the parallel architecture.

Performance Management 45

Page 46: Teradata Database Performance Management

Chapter 4: Collecting Resource Usage DataResUsage and CPU Utilization

Poor Parallel Efficiency

Possible causes of poor node parallel efficiency include:

• Down node

• Uneven number of AMPs per node

• Skewed table distribution

• Skewed join or aggregation processing

• Non-Teradata Database application running on a TPA node

• Coexistence system with different speed nodes

Poor parallel efficiency can also occur at the AMP level. Common causes of poor AMP parallel efficiency include:

• Poor table distribution (You can check this in DBC.TableSizeV.)

• Skewed processing of an SQL statement:

• User CPU (You can check this in DBC.AMPusage.)

• Spool (You can check this in DBC.DiskspaceV.)

Parallel Disk Efficiency

Parallel AMP efficiency does not necessarily match parallel disk efficiency. Monitor parallel disk efficiency using ResUsageSvdsk and ResUsageSpdsk tables.

The following table lists common mistakes that can cause skewing and poor parallel efficiency and suggests solutions.

Mistake Solution

A user did not define a PI. The system uses the first column of the table as the default P1.

Define a PI with good data distribution.

A user used a null value as PI for the target table in a left outer join.

Perform any of the following:

• Choose a different PI.

• Handle NULL case separately.

• Use a multiset table.

46 Performance Management

Page 47: Teradata Database Performance Management

Chapter 4: Collecting Resource Usage DataResUsage and Disk Utilization

CPU Use

The following macros provide information on CPU use.

For information on the above macros, see Resource Usage Macros and Tables.

ResUsage and Disk Utilization

The following macro reports disk utilization.

A user performed a join on column with poor data distribution. For example, the user entered:

SELECT A.colname, B.x, B.yFROM A, BWHERE A.colname =

B.colname;

• Identify column value and counts. For example, enter:

SELECT colname, count(*) FROM T

HAVING count(*) > 1000GROUP BY 1 ORDER BY 1 Desc;

The following displays:

colname count(*)codeXYZ 720,000codeABC 1,200

• Break the query into two separate SQL statements. For example, handle codeXYZ only in one SQL statement; handle all other cases in another SQL statement.

• Collect statistics on the join column.

Mistake Solution

Macro Table Purpose

ResCPUByNode SPMA Report of how each individual node is utilizing CPUs

ResCPUByPE SVPR Report of how each Parsing Engine (PE) utilizes the CPUs on irrespective node

ResCPUByAMP SVPR Report of how each AMP utilizes the CPUs on the respective node

Macro Description Purpose

ResNode (System provided) by second averages

General system analysis

Performance Management 47

Page 48: Teradata Database Performance Management

Chapter 4: Collecting Resource Usage DataResUsage and BYNET Data Macro

ResUsage and BYNET Data Macro

For ResNode columns, see Resource Usage Macros and Tables.

Macro Description Purpose

ResNode (System provided) by second averages

General system analysis

48 Performance Management

Page 49: Teradata Database Performance Management

CHAPTER 5 Collecting Other SystemPerformance Data

This chapter describes collecting data associated with system performance using DBC.AMPUsage and canary queries. It also describes collecting data space data.

Using the DBC.AMPUsage View

AMPUsage provides cumulative information about the usage of each AMP for each user and account.

Since the system maintains data on a per-AMP basis, and includes normalized CPU, you can measure such things as CPU or I/O skew at a user level, even if you have coexistence (more than one node type in the configuration). The system collects and continually Inserts/Updates the rows in this table based on the ASE variables used.

AMPUsage counts logical I/Os, not physical.

DBC.AMPUsage View and ASE

Without ASE, AMPUsage will accumulate CPU and logical disk I/O usage by user and account from day one. It writes at minimum one row per user and account per AMP.

Including the recommended ASE variables (&S&D&H) in account strings increases the usefulness of AMPUsage by making AMPUsage more granular. Moreover, including the recommended ASE variable in account strings aids capacity planning analysis. For information on ASE, see Chapter 3: “Database Query Log Data and System Performance.”

Data is logged cumulatively, not in intervals as it is with ResUsage data. Since data is accumulated into the cache after a completed step, the one exception is aborted queries, which would not include the accumulated usage in the step that was actually aborted.

AMPUsage and Resource Usage

While resource usage is the primary data to use in assessing system usage at the system, node and AMP levels, AMPUsage data is required to see what users are doing.

With DBC.AMPUsage, one can identify:

• Heavy users of the system, over time and at the moment

Performance Management 49

Page 50: Teradata Database Performance Management

Chapter 5: Collecting Other System Performance DataUsing Canary Queries

• Users running skewed work

• Usage trends over time, by group or individual

For more information on DBC.AMPUsage, see:

• “ResUsage and DBC.AMPUsage View” on page 43

• Data Dictionary

Using Canary Queries

Canary, or heartbeat, queries help provide data on workload impacts to the system. Each canary query is fixed to do a consistent amount of work per execution.

You can use Teradata Viewpoint to send canary queries.

Use a canary query to:

• Measure response time as an indicator of system demand or system or database hangs.

• Initiate an alert system if response time degrades so that you can take appropriate action.

• Establish response time Service Level Agreements (SLAs) based on canary response times.

A canary query can be any SQL statement run at specific intervals whose response time is being monitored. You can use Teradata Viewpoint to send canary queries.

You can also define:

• System-wide canary queries running at the following default priority: $M.

• Canary queries by priority group. These queries should be run in the appropriate priority group and logged to DBQL.

System Canary Queries

Use system canary queries to check for overall system or database hangs, to take some kind of action when response times reach certain thresholds, or when stalled, such as send alert and/or capture system level information.

More than just a check, a system canary query should execute diagnostics that capture the state of the system if performance stalls.

System canary queries are intended specifically to focus on the core system of Teradata Database. They should be short-running (one second), low impact queries on tables that are normally not write locked.

System canary queries are most useful when run frequently. For example, some sites run them every 3 to 5 minutes; other sites find every 5 to 10 minutes adequate.

They should be run on a system node. This will eliminate other factors, such as middle tiers, network connections.

Depending on their makeup, canary queries can add to contention for resources. Use them selectively, where needed, with shorter queries preferable.

50 Performance Management

Page 51: Teradata Database Performance Management

Chapter 5: Collecting Other System Performance DataUsing Canary Queries

Sample System Canary Query

The simplest canary monitor query is the following:

SELECT * from DBC.DBCInfoV;

As the query runs, Teradata Viewpoint can monitor the query, logging start and end times. If the query runs longer than the indicated threshold, an alert and perhaps diagnostic scripts are automatically executed.

Production Canary Queries

Production canary queries may be used to:

• Take response time samplings, storing them for tracking purposes, or

• Monitor the expected response times of specific groups of queries, such as short-running tactical queries running in high priority.

Response times are an indicator of system demand. When system demand is high, canary response is high. You can expect all other queries running in the same PG to display similar elongations in response time.

From a user perspective, a sudden deviation in response times would have an immediate impact, since users of consistently short running queries would be the first to notice performance degradation.

Production canary queries have wider uses than system canary queries and can be used in a variety of ways. For example, they:

• Can be run on production user tables.

• Could be run from other endpoints in the system architecture, such as a network client PC or z/OS client to expand scope of monitoring.

• Monitor overall response.

• Monitor specific area of the job mix.

• Can be more complex and similar in nature to a particular type of production query, running in the same Priority Scheduler PG.

• Are run less frequently than system canary queries, usually once every 20 to 60 minutes.

In using a production query from a non-TPA node location, other things, such as network and middle-tier monitoring, are also covered, but when it stalls, you need to investigate further to determine where the bottleneck is located.

Once the response time for a canary query is stored in a table, it can be summarized for use in tracking trends.

Because production canary queries will use up production resources, frequency and scope of use should be balanced against the value gained from the results being analyzed. If you've got more canary queries than you have time to evaluate, cut back.

Performance Management 51

Page 52: Teradata Database Performance Management

Chapter 5: Collecting Other System Performance DataUsing Canary Queries

52 Performance Management

Page 53: Teradata Database Performance Management

CHAPTER 6 Query Analysis Resources andTools

This chapter describes Teradata Database query analysis resources and tools that help tune performance through application and physical database design.

Query Analysis Resources and Tools

You can use the following resources and tools to take best advantage of the query analysis capabilities of Teradata Database.

Each of the above is described in the sections that follow.

Query Capture Facility

The Query Capture Facility (QCF) allows the steps of query execution plans to be captured. Special relational tables that you create in a user-defined Query Capture Database (QCD) store the query text and plans.

Use This Tool... To...

Query Capture Facility (QCF) perform index analysis using an SQL interface to capture data demographics, collect statistics, and implement the results.

Target Level Emulation (TLE) replicate your production configuration in a safe test environment.

Teradata Visual EXPLAIN compare results from a query run at different times, on different releases, or with different syntax.

Teradata System Emulation Tool (SET) capture the complete environment for a specific SQL or database name with everything but the data itself.

Teradata Index Wizard perform SI analysis and offer indexing recommendations, using data captured via QCF and/or DBQL.

Teradata Statistics Wizard collect statistics for a particular workload or select tables or columns or indexes on which statistics are to be collected and recollected.

Performance Management 53

Page 54: Teradata Database Performance Management

Chapter 6: Query Analysis Resources and ToolsTarget Level Emulation

QCF and SQL EXPLAINs

The Optimizer produces the source of the captured data and outputs the text of SQL EXPLAINs detailing the final stage of optimization, although currently the data that QCF captures does not represent all the information reported by EXPLAIN.

Captured information becomes source input to:

• Teradata Visual EXPLAIN

See “Teradata Visual EXPLAIN” on page 54.

• Teradata Index Wizard

See “Teradata Index Wizard” on page 55.

For detailed information on QCF, see SQL Request and Transaction Processing.

Target Level Emulation

Target Level Emulation (TLE) allows the Teradata Support Center to emulate your production system for the purpose of query execution plan analysis.

Query plans are generated on the test system as if the queries were submitted on the production system. TLE achieves this by emulating the cost parameters and random AMP samples of your production system on the test system.

Performance Benefits

You can use TLE to validate and verify new queries in a test environment ensuring that your production work is not disrupted by problematic queries.

Caution: The Teradata Support Center should run TLE on a test system. Do not enable it on a production system.

For more information on TLE, see SQL Request and Transaction Processing.

Teradata Visual EXPLAIN

Teradata Visual EXPLAIN client-based utility is an interface for application performance analysis and comparison.

You can use Teradata Visual EXPLAIN to:

• Generate a description of the query processing sequence.

• Compare the same query run on different releases or operating systems.

• Compare queries that are semantically the same but syntactically different.

54 Performance Management

Page 55: Teradata Database Performance Management

Chapter 6: Query Analysis Resources and ToolsTeradata System Emulation Tool

Teradata System Emulation Tool

Teradata System Emulation Tool (SET), a client-based tool, is integrated with Teradata Visual EXPLAIN and Compare (VECOMP) and designed for application developer to simulate production environments on very small or disparate test systems.

If you have a test system with some of the data, you can use Teradata SET to import the production table detailed statistics, TLE data, all DDLs and AMP sampling statistics from the production system to the test system.

Performance Benefits

Teradata SET enables you to:

• Imitate the impact of environmental changes on SQL statement performance.

• Provide an environment for determining the source of Optimizer-based production database query issues using environmental cost data and random AMP sample-based statistical data.

For information on how to use Teradata SET, see Teradata System Emulation Tool User Guide.

Teradata Index Wizard

Teradata Index Wizard analyzes SQL statements in a workload, using the contents of the tables in QCD, and recommends the best set of indexes to use.

Index Wizard helps re-engineer existing databases by recommending, for example, secondary index (SI) definitions that may improve the overall efficiency of the workload. Recommendations can include adding or deleting SIs to or from an existing design.

Index Wizard creates workload statements, analyzes them, and then creates a series of reports and index recommendations that show various costs and statistics. The reports help you decide if the index recommendation is appropriate or not.

Index Wizard then validates the index recommendations so you can compare performance with existing physical database design to the recommended physical database design enhancements, that is, recommended indexes. Use these recommendations to evaluate potential performance improvements and modify the database accordingly.

Index Wizard Support for Partitioned Primary Index

Teradata Database supports INITIATE PARTITION ANALYSIS statement in Teradata Index Wizard tool. This statement makes recommendations with respect to table partitioning.

Note: The INITIATE PARTITION ANALYSIS statement is also a standalone SQL statement that you can execute without the Index Wizard.

The statement recommends potential performance benefits from adding a partitioning expression to one or more tables in a given workload. The INITIATE PARTITION ANALYSIS

Performance Management 55

Page 56: Teradata Database Performance Management

Chapter 6: Query Analysis Resources and ToolsTeradata Index Wizard

statement does not recommend the complete removal of any defined partitioning expressions. It considers, however, the alteration of an existing partitioning expression if a Partitioned Primary Index (PPI) table is explicitly specified in the table_list.

Because PPIs can dramatically improve query performance, but the process of deciding when to use them and how to define them is difficult for many users, support for PPIs within Index Wizard is particularly valuable.

Decisions about which class of PPIs to recommend are based on:

• Functionality that was deemed most useful for the majority of workloads,.

• Functionality that could be implemented within the Index Wizard architecture in a straightforward manner.

• Recommendations that were less likely to cause performance regressions outside of the defined workload.

For information on which class of PPIs Teradata Index Wizard considers and recommends, see SQL Request and Transaction Processing.

Final recommendations are stored in the PartitionRecommendations table in the Query Capture Database (QCD). A separate row is stored in this table for each workload statement that is impacted by the Index Wizard recommendation. The RangePartExpr Table stores the individual details of a recommended range partitioning expression that involves the RANGE_N function.

More workload cache memory (the memory used to cache information retrieved from the QCD) may be needed to support the PPI feature of Index Wizard for analysis and validation operations.

See “DBS Control Utility” in Utilities for information on VMaxWorkloadCache and IAMaxWorkloadCache.

Performance Impact

Teradata Index Wizard:

• Simulates candidate SIs without incurring the cost of creating them.

• Validates and implements SI recommendations.

• Provides automatic “what-if” analysis of user-specified index candidates.

• Interfaces with Teradata SET to allow workload analysis on test systems as if the workload had been analyzed on the production system.

• Interfaces with Teradata Visual EXPLAIN to compare query plans in the workloads.

For information on Teradata Index Wizard, see SQL Request and Transaction Processing.

For information on how to use Teradata Index Wizard, see Teradata Index Wizard User Guide.

56 Performance Management

Page 57: Teradata Database Performance Management

Chapter 6: Query Analysis Resources and ToolsTeradata Statistics Wizard

Teradata Statistics Wizard

Teradata Statistics Wizard assists in the process of collecting statistics for a particular workload or selecting arbitrary tables or columns or indexes on which statistics are collected or recollected.

In addition, the Statistics Wizard permits users to validate the proposed statistics on a production system. This feature enables the user to verify the performance of the proposed statistics before applying the recommendations.

As changes are made within a database, the Statistics Wizard identifies the changes and recommends:

• Which tables should have their statistics collected, based on the age of data and table growth, and

• What columns or indexes would benefit from having statistics defined and collected for a specific workload.

Performance Benefits

Teradata Statistics Wizard recommends the collection of statistics on specified tables, columns, or indexes, the collection of which may improve system performance. See “Collecting Statistics” on page 101.

For information on Teradata Statistics Wizard, see Teradata Statistics Wizard User Guide.

Performance Management 57

Page 58: Teradata Database Performance Management

Chapter 6: Query Analysis Resources and ToolsTeradata Statistics Wizard

58 Performance Management

Page 59: Teradata Database Performance Management

CHAPTER 7 System Performance and SQL

This chapter discusses system performance and SQL operations, including performance enhancements to the Optimizer.

CREATE TABLE/ALTER TABLE and Data Retrieval

Adjusting Table Size

The performance of data retrieval is in direct proportion to the size of the tables being accessed. As table size increases, the system requires additional I/O operations to retrieve or update the table.

Consider using compression on any large table. Compression is a key way to reduce table size, I/O and CPU usage and thus, response times. See “Altering Tables and Column Compression” on page 61. Also, indexes, such as PPI, can be used to access a subset of data for a query.

Keep in mind, of course, the requirements of other applications. They may need to join the table fragments to obtain needed data. You can use join indexes (see “Join Indexes” on page 87) to meet this need efficiently, but at the cost of some overhead and maintenance.

Reducing Number of Columns

As the number of columns increases, the row size increases and the table uses more datablocks for the same number of rows. More datablocks means that the number of I/O operations increases when scanning the table. Reducing the number of columns can improve scan performance by reducing the number of I/Os that are required.

If you define a table with indexes that you chose for maximum performance, and if users structure their statements to take advantage of those indexes, satisfactory response should be achieved even on very large tables of more than 100 columns.

Reducing Row Size

The size of a row is based on the total width of all the columns in the table, plus row overhead. An increase in row size may require more datablocks to contain the same number of rows.

A row cannot span datablocks. If a single row is longer than the current maximum size of a multirow data block, the system allocates a large data block (up to the system maximum block size) to accommodate this single large row.

If a single row exceeds the absolute maximum block size of 255 sectors, the system returns an error message to the session. The current recommended size is 254.

Performance Management 59

Page 60: Teradata Database Performance Management

Chapter 7: System Performance and SQLCREATE TABLE/ALTER TABLE and Data Retrieval

Altering Tables

You can use the ALTER TABLE statement to change the structure of an existing table to improve system performance and reduce storage space requirements.

Reduce the number of bytes in the table to reduce the number of I/O operations for that table.

The following summarizes the effect on table storage space of using ALTER TABLE to perform specific functions. (Resultant changes to the Data Dictionary have a trivial effect on performance.)

For more information, see Database Design and SQL Data Definition Language.

Function Performance ImpactSpace Requirement (Spool or Perm)

Add column (COMPRESS, NULL) All table rows are changed if a new presence byte is added.

Slight increase in perm space

Add column (NOT NULL, DEFAULT, and WITH DEFAULT)

All table rows are changed. Increase in perm space

Add column (NULL, fixed-length) All table rows are changed. Increase in perm space

Add column (NULL, variable length) All table rows are changed. Slight increase in perm space

Add FALLBACK option Entire table is accessed to create the fallback copy. Long-term performance effects.

Approximately doubled in perm space

Adding CHECK CONSTRAINT Takes time to validate rows, which impacts performance.

Unchanged

Adding Referential Integrity Takes time to check data. Impacts performance long term. Similar to adding indexes.

Possible great increase in:

• Spool space

• Perm space (for index if not soft batch)

Change format, title, default No impact. Unchanged

Changing cylinder FreeSpacePercent (FSP)

Raising the FSP can make inserting new data less expensive by reducing migration and cylinder allocations.

Lowering the FSP has the opposite effect.

Increase in perm space for operations such as default maximum, MultiLoad, restore

60 Performance Management

Page 61: Teradata Database Performance Management

Chapter 7: System Performance and SQLCREATE TABLE/ALTER TABLE and Data Retrieval

Altering Tables and Column Compression

The ALTER TABLE statement supports adding, changing, or deleting compression on one or more existing column(s) of a table, whether the table has data or is empty.

The ALTER TABLE statement enables users to:

• Make a noncompressed column compressed.

• Add, drop, or replace compress values using multi-value compression.

• Drop the COMPRESS attribute altogether.

Column compression does not change the table header format. Nor does it affect any Data Dictionary table definitions.

Compression reduces storage costs by storing more logical data per unit of physical capacity. Optimal application of compression produces smaller rows. This results in more rows stored per data block and thus fewer datablocks.

Compression also enhances system performance because there is less physical data to retrieve per row for queries.

Moreover, because compressed data may remain compressed while in memory, the file system segment (FSG) cache can hold more logical rows, thus reducing disk I/O.

DATABLOCKSIZE

You can control the default size for multirow datablocks on a table-by-table basis via the:

• DATABLOCKSIZE option

• MERGEBLOCKRATIO option

in the CREATE TABLE and ALTER TABLE statements, as follows.

Changing maximum multirow block size

If the table is not updated after the change, then there is no impact.

If the table is change, there can be performance impact whether or not the IMMEDIATE clause is specified.

Slight increase in perm space for smaller values. Slight decrease for larger values

Delete FALLBACK option FALLBACK subtable is deleted. Long-term performance effects.

Approximately half perm space

Drop column All table rows are changed. Decrease in perm space

Function Performance ImpactSpace Requirement (Spool or Perm)

Performance Management 61

Page 62: Teradata Database Performance Management

Chapter 7: System Performance and SQLCREATE TABLE/ALTER TABLE and Data Retrieval

To evaluate actual block sizes of tables, you can run the SHOWBLOCKS command of the Ferret utility. For more information on running this command, see Utilities. To specify the global data block size, use PermDBSize (see “PermDBSize” in Utilities).

Disk arrays are capable of scanning at higher rates if the I/Os are larger. But larger I/Os can be less efficient for row-at-a-time access which requires that the entire datablock be read for the relatively few bytes contained in a row. Cylinder reads allow smaller datablocks for row-at-a-time access and large reads for scans.

In general, the benefits of large datablocks with respect to scans outweigh, for the vast majority of workloads, the small penalty associated with row-at-a-time access up to 64 KB. Setting datablock size requires more consideration at 128 KB datablocks, where the penalty for row-at-a-time access becomes measurable.

IF you specify… THEN…

DATABLOCKSIZE in CREATE TABLE

the datablock can grow to the size specified in DATABLOCKSIZE instead of PermDBSize (see “PermDBSize” in Utilities).

If the rows in the block fit in a smaller block size, the block created will be smaller.

DATABLOCKSIZE in ALTER TABLE

the datablocks can grow to the size specified in DATABLOCKSIZE.

Whether they are adjusted to that new size gradually over a long period of time depends on the use of the IMMEDIATE clause.

MERGEBLOCKRATIO in CREATE TABLE

it limits attempts to combine blocks together if the resulting size is larger than the specified percent of the maximum multirow datablock size.

MERGEBLOCKRATIO in ALTER TABLE

the size of the resulting block when multiple existing blocks are being merged has a upper limit.

The limit depends on whether logically adjacent blocks are deemed mergeable with the single block being modified.

Blocks can still be initially loaded at the PermBDSize specification or the block size specified with the DATABLOCK option. Merges occur only during full table modifications.

MERGEBLOCKRATIO and the default size for multirow datablocks are unrelated.

the IMMEDIATE clause the rows in all existing datablocks of the table are repacked into blocks using the newly specified size. For large tables, this can be a time-consuming operation, requiring spool to accommodate two copies of the table while it is being rebuilt.

If you do not specify the IMMEDIATE clause, existing datablocks are not modified. As individual datablocks of the table are modified as a result of user transactions, the new value of DATABLOCKSIZE is used. Thus, the table changes over time to reflect the new block size.

62 Performance Management

Page 63: Teradata Database Performance Management

Chapter 7: System Performance and SQLTOP N Row Option

FREESPACE

You can specify the default value for free space left on a cylinder during certain operations on a table-by-table basis via the FREESPACE option in the CREATE TABLE and ALTER TABLE statements.

This allows you to select a different value for tables that are constantly modified versus tables that are only read after they are loaded. To specify the global free space value, use FreeSpacePercent (see “FreeSpacePercent” in Utilities).

ALTER TABLE Performance

The performance of ALTER TABLE statements have been enhanced, especially when column-level privileges are used extensively, by minimizing locks on dictionary tables, especially DBC.AcessRight, and by reducing the locking time on dictionary tables. This improves system concurrency when there are multiple ALTER TABLE statements that add or drop columns.

TOP N Row Option

As an option to the SELECT statement, the TOP N option automatically restricts the output of queries to a certain number of rows. This option provides a fast way to get a small sample of the data from a table without having to scan the entire table. For example, a user may want to examine the data in an Orders table by browsing through only 10 rows from that table.

The value of N can be passed into the operator by means of a macro, stored procedure, or USING request modifier parameter.

Performance Optimizations

TOP N option has been optimized for handling TOP N and “any N” requests. Optimizations include

• Adding an AMP runtime optimization for TOP n PERCENT operations.

• Extending “any N” optimization to INSERT … SELECT and CREATE TABLE … AS requests, views, and derived tables for all values of N.

• Adding an optimization that avoids redistributing the rows for the hash partitioning case when the grouping columns of a window function contain the PI columns of the source relation.

• Adding a RankLimit optimization for a TOP n operation that does not specify the WITH TIES option.

• Adding runtime optimizations for TOP n in a request that specifies an ORDER BY specification.

Performance Considerations

For best performance, use the TOP N option instead of the QUALIFY clause with RANK or ROW_NUMBER.

Performance Management 63

Page 64: Teradata Database Performance Management

Chapter 7: System Performance and SQLRecursive Query

• In best cases, the TOP N option provides better performance.

• In worse cases, the TOP N option provides equivalent performance.

See “The SELECT Statement” in SQL Data Manipulation Language.

If a SELECT statement using the TOP N option does not also specify an ORDER BY clause, the performance of the SELECT statement is better with BTEQ than with FastExport.

Recursive Query

A recursive query is a way to query hierarchies of data, such as an organizational structure, bill-of-materials, and document hierarchy.

Recursion is typically characterized by three steps:

• Initialization

• Recursion, or repeated iteration of the logic through the hierarchy

• Termination

Similarly, a recursive query has three execution phases:

• Initial result set

• Iteration based on the existing result set

• Final query to return the final result set

Ways to Specify a Recursive Query

You can specify a recursive query by:

• Preceding a query with the WITH RECURSIVE clause.

• Creating a view using the RECURSIVE clause in a CREATE VIEW statement.

For a complete description of the recursive query feature, with examples that illustrate how it is used and its restrictions, see SQL Fundamentals.

For information on the WITH RECURSIVE clause, see SQL Data Manipulation Language.

For information on the RECURSIVE clause in a CREATE VIEW statement, that is, for information on recursive views, see SQL Data Definition Language.

Performance Considerations

The following broadly characterizes the performance impact of recursive queries with respect to execution time:

• Using a recursive query shows a significant performance improvement over using temporary tables with a stored procedures. In most cases, there is a highly significant improvement.

• Using the WITH RECURSIVE clause has basically the same or equivalent performance as using the RECURSIVE VIEW.

64 Performance Management

Page 65: Teradata Database Performance Management

Chapter 7: System Performance and SQLCASE Expression

• In using a recursive query, it is important to put depth limits on the recursion to prevent infinite recursion when there are cycles in the underlying data.

CASE Expression

Effects on Performance

The CASE expression can provide performance improvements for the following queries:

• For multiple aggregates filtering distinct ranges of values. For example, total sales for several time periods.

• To create two-dimensional reports directly from Teradata Database. For example, balances in individual accounts held by all bank customers.

CASE expressions help increase performance. They return multiple results in a single pass over the data rather than making multiple passes over the data and then using the client application to combine them into a single report.

You can see performance improvements using the CASE expression as the following increase:

• Number of queries against the same source table(s)

• Volume of data in the source table

Valued and Searched CASE Expression

Use one of the following CASE expression forms to return alternate values based on search conditions.

This form… Tests… Example

Valued an expression against possible values.

Create a catalog entitled “Autumn Sale” that shows spring items marked 33% off and summer items marked 25% off.

SELECT item_number, item_description,item_price as “Current//Price”, CASE item_season

WHEN ‘spring’ THEN item_price *(1-.33)WHEN ‘summer’ THEN item_price *(1-.25)ELSE NULL

END AS “Sale//Price”FROM inventory_table;

Performance Management 65

Page 66: Teradata Database Performance Management

Chapter 7: System Performance and SQLCASE Expression

The following examples illustrate simple code substitution, virtual denormalization, and single pass examples that use the CASE expression.

Example 1: Simple Code Substitution

For example, instead of joining to a description table, use the CASE expression when the WHERE clause will not contain the case values, that is, the WHERE clause in the example below will not contain region_number.

It is important to not use CASE statements in a view, when the queries that access the view will use the values in a case in the WHERE clause. When this occurs, the Optimizer is unable to evaluate any statistics and a less than optimum query plan could be executed.

SELECT CASE region_numberWHEN 1 THEN "North"WHEN 2 THEN "South"WHEN 3 THEN "East"

ELSE "West" END,sum(sales)FROM sales_tableGROUP BY 1;

Example 2: Virtual Denormalization

ABC Telephone Company has a History table with n columns, plus call minutes and call type:

• 1 - daytime call

• 2 - night-time call

• 3 - weekend call

You want a summary of call minutes for each call type for each area code on a single line of output.

The standard solution is:

Searched arbitrary expression(s).

Repeat the query above, and mark down by 50% summer items with inventories of less than three.

SELECT item_number, item_description, item_price as “Current//Price”,CASEWHEN item_season = ‘summer' and item_count < 3

THEN item_price *(1-.50)WHEN item_season = ‘summer’ and item_count >= 3

THEN item_price *(1-.25)WHEN item_season = ‘spring’ THEN item_price *(1-.33)

ELSE NULL END AS “Sale//Price”FROM inventory_table WHERE item_season in (‘spring’ or ‘summer’);

This form… Tests… Example

66 Performance Management

Page 67: Teradata Database Performance Management

Chapter 7: System Performance and SQLCASE Expression

1 Do a GROUP BY on call_type and area code in the History table.

2 Do a self-join to get call_types 1 and 2 into the same row.

3 Do another self-join to get call_type 3 into the same row that contains all three call types.

In the classic denormalization solution, you would physically denormalize the History table by putting all three call types in the same row. However, a denormalized table requires more maintenance.

Instead, you can use the CASE expression to perform a virtual denormalization of the History table:

CREATE View DNVas Select Col1, ... , Col n,CASE WHEN call_type = 1

THEN call_minutes END (NAMED Daytime_Minutes),CASE WHEN call_type = 2

THEN call_minutes END (NAMED Nighttime_Minutes) ,CASE WHEN call_type = 3

THEN call_minutes END (NAMED Weekend_Minutes) FROM history;

Example 3: Single Pass

In this example, you want a report with five sales columns side by side:

• Current Year, Year to Date (Ytd)

• Current Year, Month to Date (Mtd)

• Last Year, Year to Date (LyYtd)

• Last Year, Month to Date (LyMtd)

• Last Year, Current Month (LyCm)

You currently execute five separate SQL statements and combine the results in an application program.

Select sum(sales) ... where sales_date between 060101 and date; [Ytd]Select sum(sales) ... where sales_date between 061001 and date; [Mtd]Select sum(sales) ... where sales_date between 050101 and ADD_MONTHS (date, -12); [LyYtd]Select sum(sales) ... where sales_date between 051001 and ADD_MONTHS (date, -12); [LyMtd]Select sum(sales) ... where sales_date between 051001 and 051031; [LyCm]

Instead, you can use the CASE expression to execute one SQL statement that only makes one pass on the Sales_History table.

Select ... sum(CASE WHEN sales_date between 060101 and date THEN sales ELSE 0

END), [Ytd]sum(CASE WHEN sales_date between 061001 and date THEN sales ELSE 0

END), [Mtd]sum(CASE WHEN sales_date between 050101 and ADD_MONTHS (date, -12)

THEN sales ELSE 0 END),[LyYtd]sum(CASE WHEN sales_date between 051001 and ADD_MONTHS (date, -

12)THEN sales ELSE 0 END),[LyMtd]

Performance Management 67

Page 68: Teradata Database Performance Management

Chapter 7: System Performance and SQLAnalytical Functions

sum(CASE WHEN sales_date between 051001 and 051031 THEN sales ELSE 0 END), [LyCm]from ... WHERE sales_date between 050101 and date ...

Analytical Functions

Teradata Database support for analytical functions allows you to perform computations at the SQL level rather than through a higher-level calculation engine.

Teradata Database supports:

• Ordered analytical syntax

• Random stratified sampling

• Multiple aggregate distincts

For complete information on analytic functions, see SQL Functions, Operators, Expressions, and Predicates.

Analytical Functions and Performance

Analytical functions, which are extremely helpful in general decision support, speed up order-based analytical type queries.

Using analytical functions, you can target the data analysis within the data warehouse itself. This provides several advantages, including:

• Improved processing performance.

• Less data movement across the network.

• Faster analysis than that performed by external tools and sort routines, but full access to ordered analytical functions by external tools such as Teradata Warehouse Miner.

For example, Teradata Warehouse Miner FREQ function uses CSUM, RANK, and QUALIFY in determining frequencies.

• Support for ANSI version of existing aggregate functions, enabling you to use RANK, SUM, AVG, and COUNT on multiple partitions within a statement select list.

• Simpler SQL programming, particularly because you can use:

• Nested aggregates with the HAVING clause

• Window functions

• QUALIFY, RANK, and ORDER BY clauses

For example, Teradata Database permits this query structure:

SELECT state, city, SUM(sale),RANK() OVER(PARTITION BY state ORDER BY SUM(sale))FROM Tbl1, Tbl2WHERE Tbl1.cityid = Tbl2.cityidGROUP BY state, cityHAVING MAX(sale) > 10QUALIFY RANK() OVER

68 Performance Management

Page 69: Teradata Database Performance Management

Chapter 7: System Performance and SQLAnalytical Functions

(PARTITION BY state ORDER BY MIN(sale)) > 10;

Example: Using Teradata RANK

RANK (sort_expression_list) returns the rank (1..n) of all rows in a group by values of sort_expression_list.

For example, assume you enter this query:

SELECT ProdId, Month, Sales, RANK(Sales)FROM SalesHistoryGROUP BY ProdIdQUALIFY RANK(Sales) <=3;

The rows of the response table are ranked as follows:

ProdId Month Sales RANK1234 0607 500 11234 0609 300 21234 0608 250 3...5678 0609 450 15678 0608 150 25678 0607 100 3

This opens up possibilities for applying RANK to non-analytical processing that may be cumbersome otherwise.

For example, RANK can:

• Process data sequentially.

• Generate unique sequential numbers on columns that uniquely define the row.

• Process consecutive rows in a predefined order, when you define a self-join on a ranked table.

For example:

1 Create a copy of the table with a new column containing the rank, based on some ordering criteria; for example: term_eff_date or load_event_id.

2 Define a self-join on the table similar to the following:

• WHERE A.rankvalue = B.rankvalue - 1

• AND A.policy_id = B.policy_id

3 Use the self-joined table to process all table rows in a single pass (proceeding from row number n to row number n+1). This offers significant performance improvement over making multiple passes to process just two rows at a time.

Random Sampling

Teradata Database supports extracting a random sample from a database table using the SAMPLE clause and specifying one of the following:

• The number rows

• A fraction of the total number of rows

Performance Management 69

Page 70: Teradata Database Performance Management

Chapter 7: System Performance and SQLAnalytical Functions

• A set of fractions as the sample

This sampling method assumes that rows are sampled without replacement and that they are not reconsidered when another sample of the population is taken. This method results in mutually exclusive samples when you request multiple samples. In addition, the random sampling method assumes proportional allocation of rows across the AMPs in the system.

Random Stratified Sampling

In addition to random sampling option, Teradata Database supports stratified sampling.

Random Stratified Sampling, also called proportional or quota random sampling, involves dividing the population into homogeneous subgroups and taking a random sample in each subgroup. Stratified sampling represents both the overall population and key subgroups of the population. The fraction specification for stratified sampling refers to the fraction of the total number of rows in the stratum.

The following apply to stratified sampling.

Distincts and Multiple Aggregate Distincts

Teradata Database supports the use of:

• One DISTINCT expression when performing an aggregation.

• Multiple aggregate distincts, which allow multiple DISTINCT expressions for aggregates.

For example:

SEL g, SUM(DISTINCT a), SUM(DISTINCT b)FROM TGROUP BY gHAVING COUNT(DISTINCT c) > 5;

The feature simplifies SQL generation.

Data in Partition By Column and System Resources

The columns specified in the PARTITION BY clause of a window specification determine the partitions over which the ordered analytical function executes.

For example, the following query specifies the StoreID column in the PARTITION BY clause to compute the group sales sum for each store:

SELECT StoreID, SMonth, ProdID, Sales,

You can specify… You cannot specify…

stratified sampling in derived tables, views, and macros.

stratified sampling with set operations or subqueries.

either a fraction or an integer as the sample size for every stratum.

fraction and integer combinations.

up to 16 mutually exclusive samples for each stratum.

70 Performance Management

Page 71: Teradata Database Performance Management

Chapter 7: System Performance and SQLUnique Secondary Index Maintenance and Rollback Performance

SUM(Sales) OVER (PARTITION BY StoreID)FROM sales_tbl;

At execution time, Teradata Database moves all of the rows that fall into a partition to the same AMP. If a very large number of rows fall into the same partition, the AMP can run out of spool space.

For example, if the sales_tbl table in the preceding query has millions or billions of rows, and the StoreID column contains only a few distinct values, an enormous number of rows are going to fall into the same partition, potentially resulting in out-of-spool errors.

To avoid this problem, examine the data in the columns of the PARTITION BY clause. If necessary, rewrite the query to include additional columns in the PARTITION BY clause to create smaller partitions that Teradata Database can distribute more evenly among the AMPs.

For example, the preceding query can be rewritten to compute the group sales sum for each store for each month:

SELECT StoreID, SMonth, ProdID, Sales,SUM(Sales) OVER (PARTITION BY StoreID, SMonth)FROM sales_tbl;

Unique Secondary Index Maintenance and Rollback Performance

Teradata Database processes Unique Secondary Index (USI) maintenance operations (INSERT . . . SELECT, Full-file Delete, Join Delete, and Update) block-at-a-time rather than row-at-a-time, whenever possible.

When the original index maintenance is processed block-at-a-time, the USI change rows are transient journaled block-at-a-time. As a result, the rollback of the USI change rows are block-at-a-time, that is, block optimized.

The USI change rows are redistributed to their owner AMP, sorted, and applied block-at-a-time to the USI subtable. That means the index datablocks are updated once rather than multiple times.

The performance improvements in index maintenance and rollback occur without requiring changes to user applications.

Nonunique Secondary Index Rollback Performance

Nonunique Secondary Index (NUSI) rollback logic is TJ (transient journal)-driven. These TJ records drive the rollback operation of NUSI rows, which occurs block-at-a-time whenever the TJ records are written block-at-a-time.

Performance Management 71

Page 72: Teradata Database Performance Management

Chapter 7: System Performance and SQLBulk SQL Error Logging

The performance improvements in index rollback occur without requiring changes to user applications.

Bulk SQL Error Logging

Teradata Database supports bulk SQL error handling for MERGE and INSERT . . . SELECT statements. This permits bulk SQL inserts and updates to be done without the target table restrictions that apply to Teradata Database load utilities.

Load utilities are restricted from having unique indexes, join or hash indexes, referential constraints, triggers, and LOBs on the target table.

USI and referential integrity (RI) violations cause the request to abort and rollback after all these violations and all other supported error conditions are logged.

For more information on creating and dropping the error table, see SQL Data Definition Language.

For information on the MERGE and INSERT . . . SELECT statements, see SQL Data Manipulation Language.

Optimized INSERT . . . SELECT Requests

Empty Table INSERT . . . SELECT Requests and Performance

An INSERT . . . SELECT optimizes performance when the target table is empty. If the target table has no data, INSERT . . . SELECT operates on an efficient block-by-block basis that bypasses journaling.

Normally, when the system inserts a row into a table, the system must make a corresponding entry into the TJ to roll back the inserted row if the transaction aborts. If a transaction aborts, the system deletes all inserts from the table one row at a time by scanning the TJ for RowIDs.

If the transaction aborts when the table into which rows are inserted is empty, the system can easily return the table to its original state by deleting all rows. Scanning the TJ is superfluous, and writing RowIDs to delete becomes unnecessary.

The advantages of using optimized INSERT . . . SELECTs are:

• Block-at-a-time processing

• Faster insert logic (that eliminates block merge complexity)

• Instantaneous rollback for aborted INSERT . . . SELECTs

Example

Using multiple Regional Sales History tables, build a single summary table by combining summaries from the different regions. Then insert these summaries into a single table via a multistatement INSERT . . . SELECT statement.

72 Performance Management

Page 73: Teradata Database Performance Management

Chapter 7: System Performance and SQLOptimized INSERT . . . SELECT Requests

All multistatement INSERT . . . SELECT statements output to the same spool table. The output is sorted and inserted into an empty table.

Form a multistatement request by semicolon placement in BTEQ as shown below, or by placing statements in a single macro.

Note: If you execute each of the statements as separate requests, only the first statement is inserted into an empty table.

INSERT into Summary_TableSELECT store, region,sum(sales),count(sale_item)FROM Region_1GROUP BY 1,2;INSERT into Summary_TableSELECT region2, sum (sales), count(sale_item)FROM Region_2GROUP BY 1,2

. . .;INSERT into Summary_TableSELECT region3, sum(sales), count(sale_item)FROM Region_NGROUP BY 1,2;

INSERT . . . SELECT Into an Empty SET Table

INSERT . . . SELECT into an empty SET table from a source known not to have duplicate rows avoids duplicate checking of the target table during insertion. This occurs even during direct insertion from another SET table.

This should offer significant performance improvement in cases where there is a NUPI that is relatively nonunique or has few values that are very nonunique.

KY01A011

Spool

Region_2

Optimized

Empty Target Table

Region_1 Region_N

Performance Management 73

Page 74: Teradata Database Performance Management

Chapter 7: System Performance and SQLArray Support

INSERT . . . SELECT with FastLoad

Use the optimized INSERT . . . SELECT to manipulate FastLoaded data:

1 FastLoad into a staging table.

2 INSERT . . . SELECT into the final table, manipulating the data as required.

FastLoad and INSERT . . . SELECT are faster than using an INMOD to manage data on the host. The host is a single bottleneck as opposed to parallel AMPs that populate temporary tables for reports or intermediate results.

Multiple source tables may populate the same target table. If the target table is empty before a request begins, all INSERT . . . SELECT statements in that request run in the optimized mode.

The staging table can be a NoPI (No Primary Index) table. FastLoad runs faster on a NoPI table than on a PI table because there is no sort or redistribution of rows. NoPI reduces the skew of intermediate tables and is a better alternative to defaulting to the first column or creating unnatural PIs.

An INSERT . . . SELECT from a NoPI table to a PI table can be slower than an INSERT . . . SELECT from a PI table to a PI table with the same PI.

INSERT . . . SELECT with Join Index

The fastest way of processing inserts into a table with a join index is as follows:

1 Use FastLoad to load the rows into an empty table with no indexes or join indexes defined.

2 Do an INSERT . . . SELECT from the freshly loaded table into the target table with the join index.

If the target table has multiple join indexes defined, the Optimizer may choose to use reusable spool during join index maintenance, if applicable.

Processing for these steps is performed a block at a time and should provide the best throughput.

Array Support

A data-driven iteration capability, called array support, allows SQL clients to iterate a parameterized Data Manipulation Language (DML) statement for multiple sets of parameter values within a single request.

This capability is referred to as data-driven iteration because it calls for the explicit presence of multiple input data records in the request to drive the iteration of a request specified only in its noniterated form.

Array support moves the explicit iteration out of the SQL request text and the input record descriptor and places it into the request data where it logically belongs.

Array Support is also effective for requests that reference data record fields in the USING modifier rather than with parameter tokens in the manner of embedded SQL.

74 Performance Management

Page 75: Teradata Database Performance Management

Chapter 7: System Performance and SQLAggregate Cache Size

For details, see SQL Fundamentals.

Performance

With array support, data insert performance improves, thus enabling users to load new data into Teradata Database faster. Data freshness enables better tactical business decision making.

Aggregate Cache Size

The valid range of values for aggregate cache size is from 1 MB to 8 MB. The default cache size is 4MB. DBS Control parameter, AggrCacheSize, controls the size of the aggregate cache.

Considerations Before Increasing Aggregate Cache Size

• If you have few groups and large numbers of rows per group in your aggregations, a larger aggregate cache size is not needed since your groups likely fits into a small or moderate-sized cache.

• If your aggregations are mostly on the PI, the aggregation algorithm keeps only one row in memory at a time. As a result, cache size is less important.

• If your aggregations have large numbers of groups and large numbers of rows per group, a larger cache is recommended.

• If you are doing lots of aggregations at the same time, defining a larger aggregate cache means less memory for all user combined. Thus, make sure you understand how your memory is being used before you make any changes to aggregate cache size.

Request Cache Entries

During database start-up (when the request cache is initialized), the number of cache entries is set to MaxRequestsSaved, a DBS Control Record field. The default value of MaxRequestsSaved is 600 and the value can be modified within the range of 300 - 2000 (inclusive).

The total size of the request cache that can be saved is limited to 100 MB. This limit is checked only if the number of request cache entries used is more than 300. If the number of request cache entries in use is 300 or less, then 100 MB limit is not checked.

By increasing the number of request cache entries, cache plans may stay in memory longer, improving processing time for requests that have been coded to take advantage of cached plans.

Performance Management 75

Page 76: Teradata Database Performance Management

Chapter 7: System Performance and SQLOptimized DROP Features

Optimized DROP Features

Any SQL applications that use CREATE and DROP objects such as a table, macro, view, or procedure are faster if privileges have not been granted on the object.

A full-file DELETE from the AccessRights dictionary table is a well-known performance problem because a table-level write lock is needed. That conflicts with DML operations that need to read AccessRights rows.

The costly full-file DELETE operation is eliminated in cases where you can use a rowhash-locked prime-index DELETE instead, specifically whenever you have performed no Grant requests on the object being dropped.

In such a case, all rows added to the AccessRights table by the CREATE statement have the same NUPI value (user ID and DatabaseID fields) and, as a consequence, you can delete these rows with a prime-index DELETE instead of a full-file DELETE.

Performance Considerations

One negative performance impact that may result from using rowhash locks rather than table-level locks is that deadlocks can occur when the Drop operation is running concurrently with another operation that requires a table-level lock on the AccessRights table.

Parameterized Statement Caching

The first time on an all-AMP query with a USING clause is seen, the Optimizer will peek at the USING values of a query and generate a specific plan that is not cached rather than generating a generic plan that is cached.

Peeking means looking at the USING values during query parsing and using those values when checking all potential optimizations, such as satisfiability, optimum single table access planning, partition elimination, and picking up join index(es). Peeking helps optimize a query based on its specific USING values.

Generic plans are cached, and reusing a cached plan saves parsing time, that is, the time the CPU takes to parse and generate a plan and send it to the Dispatcher. But reusing a cached plan may not provide the most efficient execution plan for all queries.

A specific plan is not cached since it cannot be reused for different USING values, although all other details such as the SQL text hash and the host character set, along with the estimated cost, parsing time, and run time are cached.

After both a specific and a generic plan have been generated, the Optimizer determines to always:

• Generate and use a specific plan to obtain the benefits of optimizing the query for specific values.

• Use the generic plan when the same query is repeated to obtain the benefits of caching.

76 Performance Management

Page 77: Teradata Database Performance Management

Chapter 7: System Performance and SQLIN-List Value Limit

This will be based on considerations such as the PE time required to produce the specific plan, how much CPU is saved on the AMP by the specific pan vs. the generic plan, and comparisons of the actual run time between the two plans.

The Parser reparses a request that generates a generic plan with estimated poor performance in order to generate a specific performance plan rather than executing the generic plan. Reparsing a request avoids executing an obviously under performing generic plan even once.

Peeking can be disabled using the DBS Control field, DisablePeekUsing. See “DisablePeekUsing” in Utilities.

With respect to queries that use the built-in functions DATE and CURRENT_DATE, the Optimizer generates a specific plan and caches it. But if DATE or CURRENT_DATE changes, the Optimizer disregards the cached plan and generates a new one.

See information on specific and generic plans, see SQL Request and Transaction Processing.

IN-List Value Limit

There is no arbitrary limit on the number of combined values in IN-Lists. Other existing limitations, such as the maximum number of characters in an SQL request, prevents the number of values from increasing without bound.

The former limit (1024) on the number of values in combined IN-List values has been removed.

Lifting the combined IN-List limit has the potential for supporting better performance for queries with more than 1024 combined values in two or more IN-Lists.

Reducing Row Redistribution

Extracting Combinations of Join Columns

Extracting all combinations of join columns, when these combinations are much less than the number of rows, helps achieve fewer row redistributions.

For example, one approach is as follows:

1 Load daily sales data (five million rows from 50 stores) into the Work table.

2 Join the Work table to the base reference tables to populate additional columns.

3 Eventually, insert the final Work table into the History table.

In this example, Step 2 joins the Work table to a reference Item table (120,000 rows):

1 Join the Work table to the Item table by redistributing the five million row Work table on item_no to get Item data.

2 Redistribute five million rows back to insert into another temporary Work table.

Another approach is as following:

Performance Management 77

Page 78: Teradata Database Performance Management

Chapter 7: System Performance and SQLReducing Row Redistribution

1 Extract the distinct item numbers from the temporary Work table (~100,000 rows).

2 Redistribute 100,000 rows instead of 5,000,000 rows to get the item data.

3 Redistribute 100,000 rows back to join to Work table.

Using the BETWEEN Clause

When considering a time interval, use the BETWEEN clause of MIN and MAX dates of an interval helps fewer row redistributions.

For example, a reference calendar contains 730 rows with:

• calendar_date

• fiscal_week

• fiscal_month

• fiscal_quarter

In this example, you want summary data from a History table for fiscal_quarter. A standard query would be:

SELECT H.item_code, SUM(H.items_sold)

, SUM(H.sales_revenue)FROM History H , Calendar CWHERE C.fiscal_quarter = “3Q06”AND C.calendar_date = H.sale_dateGROUP BY H.item_codeORDER BY H.item_code ;

From a performance perspective, this query would:

1 Build a spool table with dates from the reference calendar (90 days).

2 Duplicate the calendar spool. Either:

• Product join the calendar spool with the History table (90 compares/history table row).

• Sort both tables to do merge join.

Alternatively, redistribute the entire History table. Product join the large table with the calendar spool (~1 row /AMP).

Another approach is to denormalize the History table. Add fiscal_week, fiscal_month, fiscal_quarter to the History table. Qualify fiscal_month directly in the denormalized table. The penalties for using this approach include:

• Denormalization maintenance costs are higher.

• Extra bytes require more I/Os.

Example: BETWEEN Clause

The solution is to rewrite the query:

SELECT H.item_code, SUM(H.items_sold)

78 Performance Management

Page 79: Teradata Database Performance Management

Chapter 7: System Performance and SQLMerge Joins and Performance

, SUM(H.sales_revenue)FROM History H ,

(SELECT min(calendar_date), max(calendar_date)FROM Calendar WHERE fiscal_quarter = “3Q06”) AS DT (min_date, max_date)

WHERE H.sale_date BETWEENDT.min_date and DT.max_date

GROUP BY H.item_codeORDER BY H.item_code ;

From a performance perspective, the Optimizer could:

1 Build a spool table with a single row containing the first and last dates of fiscal_quarter.

2 Duplicate one row spool. Product join one row spool with the History table (2 compares/History table row).

One customer reported that a query that typically took three hours ran in about 12 minutes.

The benefits of using the BETWEEN date comparison are:

• Reducing multiple comparisons from as many dates in the date interval down to 2/row, and saving sort or redistribution of a large table.

• Not having to denormalize.

Using the BETWEEN data comparison is faster than reading extra denormalized table bytes.

In either case, the system must read all rows. The cost of reading extra denormalized table bytes is greater than the cost of building one row spool with MIN and MAX dates.

Merge Joins and Performance

Merge Joins and Nested Join

In a large join operation, a merge join requires less I/O and CPU time than a nested join. A merge join usually reads each block of the inner table only once, unless a large number of hash collisions occur.

A nested join performs a block read on the inner table for each outer row being evaluated. If the number of rows selected from the outer table is large, this can cause each block of the inner table to be read multiple times.

Merge Join with Covering NUSI

When large outer tables are being joined, a merge join of a table with a covering index of another table can realize a significant performance improvement.

The Optimizer considers a merge join of a base table with a covering NUSI, which gives the Optimizer an additional join method and costing estimate to choose from.

Performance Management 79

Page 80: Teradata Database Performance Management

Chapter 7: System Performance and SQLHash Joins and Performance

Hash Joins and Performance

The Optimizer may use a hash join instead of a merge join of tables for better performance:

• If at least one join key is not indexed.

• To provide a 10-40% performance improvement for the join step.

Note: Since the join is only part of a query, you may not see a 40% improvement in the entire query.

The hash join eliminates the sort used prior to the merge join by using a hash table instead.

You can enable hash joins with the following DBS Control Record fields:

• HTMemAlloc (see “HTMemAlloc” in Utilities)

• SkewAllowance (see “SkewAllowance” in Utilities)

Recommendations

Most sites should use the following values:

• HTMemAlloc = 2

• Skew Allowance = 75

Hash Join Costing and Dynamic Hash Join

The Optimizer costs hash joins. That is, the Optimizer evaluates the relative costs of available join methods to determine the least expensive method of joining two tables.

In addition, Teradata Database supports dynamic hash join. In this variation of the hash join, the row hash code is computed dynamically instead of the join creating a spool with the row hash code based on the join conditions. See “Hash joins” in SQL Request and Transaction Processing.

If the small table is small enough to fit into one hash partition and it is being duplicated to all AMPs, the redistribution of the large table can be eliminated by this type of dynamic hash join. In such a case, the large table is read directly, without spooling, redistributing or sorting, and the hash join is performed between the small table spool and the large table spool.

Expected performance improvements come from, but are not limited to, the following:

• Allowing hash joins and dynamic hash joins to be considered as a join option by costing.

• Using dynamic hash joins, which eliminate large table spooling.

Referential Integrity

Referential Integrity (RI) refers to relationships between tables based on the definition of a primary key (PK) and a foreign key (FK).

80 Performance Management

Page 81: Teradata Database Performance Management

Chapter 7: System Performance and SQLReferential Integrity

For more information on RI, see Database Design and SQL Fundamentals.

Benefits of RI

The following table lists and describes the benefits of RI.

Overhead Cost of RI

Overhead cost includes the building of reference index subtables and inserting, updating, and deleting rows in the referencing and referenced tables. Overhead for inserting, updating, and deleting rows in the referencing table is similar to that of USI subtable row handling.

The system redistributes a row for each reference to the AMP containing the subtable entry (USI or reference index). Specific processing differs thereafter; most of the cost is in message handling.

When implementing tables with RI:

• Consider the performance impact to update operations first.

• INSERTs will slow performance if RI is in the tables.

• Consider the cost of extra disk space for tables and extra cost for maintenance.

• Consider the cost of extra disk space for reference index subtables versus savings on program maintenance and increased data integrity.

• Compared with costs elsewhere (for example, SI), consider the cost of checking in the application especially via DML versus cost of not checking at all.

The following table describes the RI overhead for various operations.

Benefit Description

Maintains data consistency

The system enforces relationships between tables.

For example, Teradata Database enforces the relationship between a Customer ID and an application based on the definition of a PK or a FK.

Maintains data integrity When performing INSERT, UPDATE, and DELETE requests, Teradata Database maintains data integrity among referencing and referenced tables.

Increases development productivity

It is not necessary to code SQL statements to enforce referential constraints. Teradata Database automatically enforces RI.

Requires fewer programs to be written

Teradata Database ensures that update activities do not violate referential constraints.

Teradata Database enforces RI in all environments; you need no additional programs.

Performance Management 81

Page 82: Teradata Database Performance Management

Chapter 7: System Performance and SQLReferential Integrity

Join Elimination

Join elimination eliminates redundant joins based on information from RI.

The following conditions eliminate a join:

• RI exists between the two tables.

• Query conditions are conjunctive.

• The query does not contain reference columns from the PK table, other than the PK columns, including the SELECT, WHERE, GROUP BY, HAVING, ORDER BY, and so forth.

• PK columns in the WHERE clause appear only in PK-FK joins.

Operation Description

Building the reference index subtable

This is similar to executing the following statement:

SELECT I.Reference_Field, COUNT (*)FROM Referencing_table I, Referenced_table E WHERE I.Reference_Field = E.Reference_FieldGROUP BY I.Reference_Field;

Inserting a row into a referencing table

Teradata Database makes an RI check against the reference index subtable.

• If the referenced field is in the reference index subtable, Teradata Database increments the count in the reference index subtable.

• If the referenced field is not in the reference index subtable, Teradata Database checks the referenced table to verify that the referenced field exists. If it does, Teradata Database adds an entry with a count of 1 to the reference index subtable.

Deleting a row from the referencing table

Teradata Database makes an RI check against the reference index subtable, and decrements the count in the reference index subtable for the referenced field.

If the count becomes zero, Teradata Database deletes the subtable entry for the referenced field.

Updating a referencing field in the referencing table

Teradata Database makes an RI check against the reference index subtable and executes both the inserting-a-row and deleting-a-row operations on the reference index subtable, decrementing the count of the old referenced field value and incrementing the count of the new reference field value.

This is similar to changing the value of a USI column.

Deleting a row from the referenced table

Teradata Database checks the reference index subtable to verify that the corresponding referenced field does not exist. Assuming it does not exist, Teradata Database can delete the row from the referenced table. The reference index subtable check does not require the system to pass a message to another AMP, since the referenced field is the same value in the referenced table and the reference index subtable.

82 Performance Management

Page 83: Teradata Database Performance Management

Chapter 7: System Performance and SQLReferential Integrity

Referential Constraints

To maximize the usefulness of join elimination, you can specify Referential Constraints that Teradata Database does not enforce. This is sometimes called “soft RI.”

You can use the REFERENCES WITH NO CHECK OPTION clause to specify CREATE TABLE and ALTER TABLE statements with Referential Constraints and the Optimizer can use the constraints without incurring the penalty of database-enforced RI.

But when you use the REFERENCES WITH NO CHECK OPTION clause, the system does not enforce Referential Constraints. This implies that a row having a non-null value for a referencing column can exist in a table even if an equal value does not exist in a referenced column. Error messages, that would otherwise be provided when RI constraints are violated, do not appear when you specify Referential Constraints.

Moreover, with respect to Referential Constraints, if column_name is specified, then it need not refer to the single column PK of the referenced table or to a single column alternate key in the referenced table defined as UNIQUE. The candidate key acting as the PK in the referenced table need not be explicitly declared to be unique using the PRIMARY KEY or UNIQUE keywords or by declaring it to be a USI in the table definition.

However, the candidate key in the relationship actually must always be unique even if it is not explicitly declared to be so; otherwise, you can produce incorrect results and corrupt your databases in some situations when appropriate care is not taken to ensure that data integrity is maintained. As is always true when you specify Referential Constraints, you must assume responsibility for ensuring that any candidate key in a referential integrity relationship is unique, just as you must assume responsibility for ensuring that the referential relationship it anchors holds true in all cases, because Teradata Database enforces neither constraint.

Specifying Referential Constraints relies heavily upon your knowledge of the data. If the data does not actually satisfy the Referential Constraints that you provide and the Optimizer relies on Referential Constraints, then queries can produce incorrect results.

The Optimizer can use the constraints without incurring the penalty of database-enforced RI.

Standard RI and Batch RI

In standard RI, whether you are doing row-at-time updates or set-processing INSERT . . . SELECT requests, each child row will be separately matched to a row in the parent table, one row at a time. A separate select against the parent table is performed for each child row. Depending on your demographics, parent rows may be selected more than once.

IF… THEN…

the preceding conditions are met the PK join is removed from the query.

all references to the PK columns in the query are mapped to the corresponding foreign key columns.

foreign key columns are nullable the “NOT NULL” condition is added.

Performance Management 83

Page 84: Teradata Database Performance Management

Chapter 7: System Performance and SQLSecondary Indexes

With batch RI, all of the rows within a single statement, even if this is just one row, will be spooled and sorted, and will have their references checked in a single operation, as a join to the parent table. Depending on the number of rows in the INSERT . . . SELECT request, batch RI could be considerably faster, compared to checking each parent-child relationship individually.

If you plan to do row-at-time updates, there will be very little difference between standard RI and batch RI. But if you plan to load primarily using INSERT . . . SELECT requests, batch RI is recommended.

Secondary Indexes

Secondary Indexes and Performance

SIs supply alternate access paths. This increases performance. For best results, base SIs on frequently used set selections and on an equality search. The Optimizer may not use an SI if it is too weakly selective.

Statistics play an important part in optimizing access when NUSIs define conditions for the following:

• Joining tables

• Satisfying WHERE constraints that specify comparisons, string matching, or complex conditionals

• Satisfying a LIKE expression

• Processing aggregates

Because of the additional overhead for index maintenance, index values should not be subject to frequent change. When you change the value of an SI column, the system:

1 Deletes any SI references to the current value (AMP-local for NUSIs, and across AMPs for USIs).

2 Generates new SI references to the new value (AMP-local for NUSIs, and across AMPs for USIs).

Using NUSIs

The guiding principle for using NUSIs is that there should be fewer rows that satisfy the NUSI qualification condition than there are datablocks in the table. Whether the Optimizer uses a NUSI depends on the percent of rows per NUSI value, as follows.

IF this many rows qualify… THEN…

< 1 per block NUSI access is faster than full table scan. For example, if there are 100 rows per block and 1 in 1000 rows qualify, the Optimizer reads 1 in every 10 blocks. NUSI access is faster.

84 Performance Management

Page 85: Teradata Database Performance Management

Chapter 7: System Performance and SQLSecondary Indexes

In some instances, values of a NUSI are distributed unevenly throughout a table. At other times some values may represent a large percent of the table, while other values have few instances. When values are distributed unevenly, the system:

• Performs a full table scan on values that represent a large percent of table

• Uses the NUSI for the rest of the values

To query index nonuniqueness, you might enter the following:

.set retlimit 20sel <index column(s)>, count(*) from <tablename>

group by 1 order by 2 desc;

NUSIs and Block Size

For a noncovering NUSI to be effective, fewer rows must qualify than there are blocks in the base table.

Consider, for example, 100-byte rows; with a average block size of 31.5 KB, each multirow data block contains approximately 315 rows. This means fewer than one in 315 rows (that is, less than 0.32%) can qualify for a given NUSI value if the index is to be effective.

When the average block size is 63.5 KB, fewer than one in 635 rows (that is, less than 0.16%) can qualify for the NUSI to be effective.

When the average block size is 127.5 KB, fewer than one in 1275 rows (that is, less than 0.08%) can qualify for the NUSI to be effective. When the average block size is 1KB, fewer than one in 10 rows (that is, less than 10%) can qualify for the NUSI to be effective

To reset the global absolute-maximum size for multirow datablocks, see “PermDBSize” in Utilities.

Index Access

To determine if the Optimizer will use an index to process a request, include a WHERE constraint based on an index value and/or any covering column in the index, and use EXPLAIN to determine whether the value affects path selection. See “Teradata Statistics Wizard” on page 57.

Even when you use an index constraint, equivalent queries formulated with different syntax can result in different access plans. The Optimizer may generate different access paths based on the forms given below; depending on the expression, one form may be better than the other.

>= 1 per block full table scan is faster than NUSI access. For example, if there are 100 rows per block and 1% of the data qualifies, the Optimizer reads almost every block. Full table scan may be faster.

IF this many rows qualify… THEN…

Performance Management 85

Page 86: Teradata Database Performance Management

Chapter 7: System Performance and SQLSecondary Indexes

Form 1. (A OR B) AND (C OR D)

Form 2 (A AND C) OR (A AND D) OR (B AND C) OR (B AND D)

In expressions involving both AND and OR operators, the Optimizer generates the access path based on the form specified in the query. The Optimizer does not attempt to convert from one form to another to find the best path. Consider the following expression:

(NUSI = 7 OR NUSI = 342) AND (X = 3 OR X = 4)

In this case, Form 1 is optimal, because the access path consists of two NUSI SELECTs with values of 7 and 342. The Optimizer applies (X=3 OR X=4) as a residual condition. If the Optimizer uses Form 2, the access path consists of 4 NUSI SELECTs.

In the following expression:

(NUSIA = 1 OR NUSIA = 2) AND (NUSIB = 3 OR NUSIB = 4)

the collection of (NUSIA, NUSIB) comprises a NUSI. In this case, Form 2 is optimal because the access path consists of 4 NUSI SELECTs, whereas the Form 1 access path requires a full table scan.

Assume an expression involves a single field comparison using IN, such as the following:

Field IN (Value1, Value2, ...)

The Optimizer converts that expression to:

Field = Value1 OR Field = Value2 OR ...

Therefore, the Optimizer generates the same access path for either form. However, if an expression involves a multiple field comparison using IN, such as in the following query,

a. (Field1 IN (Value1 OR Value2 OR ...)AND Field2 IN (Value3 OR Value4 OR ...)

then the Optimizer converts the expression to:

b. (Field1 = Value1 OR Field1 = Value2 OR ...)AND (Field2 = Value3 OR ...)

Notice that the converted form differs from the following (which is in Form 2):

c. (Field1 = Value1 AND Field2 = Value3)OR (Field1 = Value2 AND Field2 = Value4)OR ...

Index Access Guidelines

Generally, Teradata Database follows these guidelines for index access.

Teradata Database uses… To…

PI satisfy an equality on an IN condition in a join.

UPI ensure fastest access to table data.

NUPI • Perform a single-disk row selection or join process

• Avoid sorting or redistributing rows.

86 Performance Management

Page 87: Teradata Database Performance Management

Chapter 7: System Performance and SQLJoin Indexes

For smaller tables, the Optimizer uses the index estimated to have the least amount of rows per index value.

Using appropriate SIs for the table can increase the retrieval performance for the table, but the trade-off is that the update performance can decrease.

Join Indexes

A join index is a data structure that contains data from 1 or more tables, with or without aggregation:

• Columns of two or more tables

• Columns of a single table

The guidelines for creating a join index are the same as those for defining any regular join query that is frequently executed or whose performance is critical. The only difference is that for a join index the join result is persistently stored and automatically maintained.

Performance and Covering Indexes

Queries that use join indexes can run many times faster than queries that do not use them. Moreover, query performance improves any time a join index can be used instead of the base tables.

A join index is most useful when its columns can satisfy, or cover, most or all of the requirements in a query. For example, the Optimizer may consider using a covering index instead of performing a merge join.

Covering indexes improve the speed of join queries. The extent of improvement can be dramatic, especially for queries involving complex, large-table, and multiple-table joins. The extent of such improvement depends on how often an index is appropriate to a query.

Covering indexes should perform at the higher end. Aggregate join indexes perform much better than join indexes in all areas.

In-place join indexes (where the columns of the covering index and the columns of the table to which it is to be joined both reside on the same AMP) outperform indexes which require row redistribution. An in-place, covering, aggregate join index that replaces two or more large

USI process requests that employ equality constraints.

UPIs to match values in one table with index values in another

ensure optimal join performance.

index based on more than one column (a composite index) only

process requests that employ equality constraints for all fields that comprise the index.

bitmapping process requests only when equality or range constraints involving multiple NUSIs are applied to very large tables.

Teradata Database uses… To…

Performance Management 87

Page 88: Teradata Database Performance Management

Chapter 7: System Performance and SQLJoin Indexes

tables in queries with complex joins, aggregations, and redistributions can cause a query to run hundreds of times faster.

Multitable Noncovering Join Index

Teradata Database optimizes queries to use a join index on a set of joined tables, even if the index does not cover completely the columns referenced in the table, when:

• The index includes either the RowID or the columns of a unique index of the table containing a noncovered column referenced by the query.

• The cost of such a plan is less than other plans.

A multitable, noncovering join index provides some of the query improvement benefits that join indexes offer without having to replicate all the columns required to cover the queries.

Additional overhead of accessing the base table row occurs when a noncovered column is required in the query.

Covering Bind Terms

If the connecting condition of a subquery is IN and the field it is connecting to in the subquery is unique, you can define a join index on the connected fields. This provides one more type of index for the Optimizer to consider using in place of multiple base tables.

Using Single-Table Join Indexes

Single-table join indexes are very useful in tactical applications because they can support alternative PI access to data. This is a good approach to consider when the tactical query carries a value in an equality condition for a column (such as a customer phone) that is in the table but is not the PI column of the table (which might be customer key, for example). A single-table join index can be constructed using the available non-indexed column (customer phone) as its PI, thereby enabling single-AMP access to the data, and avoiding more costly all-AMP non-PI access of the base table.

Single-table join indexes are also valuable when your applications often join the same large tables, but their join columns are such that some row redistribution is required. A single-table join index can be defined to contain the data required from one of the tables, but using a PI based on the FK of the table (preferably the PI of the table to which it is to be joined).

Use of such an index greatly facilitates join processing of large tables, because the single-table index and the table with the matching PI both hash to the same AMP.

The Optimizer evaluates whether a single-table join index can replace or partially cover its base table even when the base table is referenced in a subquery (unless the index is compressed and the join is complex, such as an outer join or correlated subquery join).

Defining Join Indexes with Outer Joins

If there is a need for a non-aggregate multitable join index between two or more large tables, considering using an outer-join join index. This approach offers the following benefits:

88 Performance Management

Page 89: Teradata Database Performance Management

Chapter 7: System Performance and SQLJoin Indexes

• For queries that reference only the outer tables, an outer-join join index will be considered by the Optimizer.

• Unmatched rows are preserved within the join index structure.

Defining Join Indexes with Inequality Conditions

In defining join indexes, inequality conditions are allowed.

To define inequality conditions between two columns of the same type, either from the same table or from two different tables, you must AND these with the rest of the join conditions.

This capability expands the usefulness of join indexes because the Optimizer can more often choose to resolve a query with a join index rather than by accessing the data tables.

Defining Join Indexes on UDT Columns and Expressions

You can define a join index, including an aggregate join index, on:

• UDT columns

• The following expressions:

• Non-aggregate expressions

• Non-OLAP expressions

• Non-UDF expressions

in the select-list and single-table conditions in the WHERE clause and ON clause.

This includes allowing a join index to be defined using a UDM that is associated with a UDT column.

With this general support for UDT columns and expressions, PDT (Period Data Type) columns and BEGIN, END and P_INTERSECT expressions on a PDT column can be used in the select-list, WHERE clause and ON clause of a join index.

These increased capabilities expand the usefulness of join indexes in improving system performance because the Optimizer can more often than not choose to resolve a query with a join index rather than by accessing the base tables.

Refreshing Join Indexes

The ALTER TABLE TO CURRENT statement for a join index allows you to refresh the content of a join index without having to drop it and recreate it.

The efficiency of the ALTER TABLE alternative compared with dropping and recreating a join index depends on how often an ALTER TABLE statement is executed and the type of current date condition defined in the join index.

If the join index is refreshed very infrequently and the current date condition requires a large volume of old rows to be removed and a large volume of new rows to be inserted, it may be more efficient to drop and recreate the join index.

Performance Management 89

Page 90: Teradata Database Performance Management

Chapter 7: System Performance and SQLJoin Indexes

Using Aggregate Join Indexes

Aggregate join indexes offer an extremely efficient, cost-effective method of resolving queries that frequently specify the same aggregate operations on the same column or columns. When aggregate join indexes are available, the system does not have to repeat aggregate calculations for every query.

You can define an aggregate join index on two or more tables or on a single table. A single-table aggregate join index includes:

• A columnar subset of a base table

• Additional columns for the aggregate summaries of the base-table columns

You can create an aggregate join index using the:

• SUM function

• COUNT function

• GROUP BY clause

The following restrictions apply to defining an aggregate join index:

• Only COUNT and SUM are valid in any combination. (COUNT DISTINCT and SUM DISTINCT are invalid.)

• To avoid overflow, always type the COUNT and SUM fields as FLOAT.

The system enforces this restriction as follows.

Many aggregate functions are based on SUM and COUNT, so even though many aggregate functions cannot be used in an aggregate join index, the join index may be used to resolve queries using such aggregate functions.

Join Indexes and the Optimizer

For each base table in a query, the Optimizer performs the following.

IF you … THEN the system …

do not define an explicit data type for a COUNT or SUM field

assigns the FLOAT type to it automatically.

define a COUNT or SUM field as anything other than FLOAT

returns an error and does not create the aggregate join index.

In this phase… The system...

Qualification evaluates up to 10 join indexes to choose the one with the lowest cost.

Qualification for the best plan includes one or more of the following benefits:

• Smallest size to process

• Most appropriate distribution

• Ability to take advantage of covered fields within the join index

90 Performance Management

Page 91: Teradata Database Performance Management

Chapter 7: System Performance and SQLJoin Indexes

Subsequent action depends on this analysis, as follows.

Protecting a Join Index with Fallback

You can define fallback protection for a simple or an aggregate join index.

With fallback, you can access a join index and the base table it references if an AMP fails with little impact on performance.

Without fallback, an AMP failure has significant impact on both availability and performance, as follows:

• You cannot update the base table referenced by a join index, even if the base table itself is defined with fallback.

• A join index cannot be accessed by queries. Performance may be degraded significantly.

The cost is a slight degradation when processing a DML statement that modifies a base table referenced by the join index because the fallback copy of the join index must also be maintained.

Join Indexes and Collecting Statistics

Single-table join indexes that are not defined as sparse join indexes and hash indexes inherit all statistics from the base table, including AMP samples and collected statistics.

Only sparse join indexes and multitable join indexes require statistics collection. It will be particularly important that statistics be collected on the sparse-defining column of a sparse join index or the sparse join index may not be selected for use.

Consider collecting statistics to improve performance during:

• Creation of a join index

• Update maintenance of a join index

You need to submit separate COLLECT STATISTICS statements for the columns in the join index and the source columns in the base tables. Join index tables and data tables are seen as separate entities to the Optimizer. (Also see “Collecting Statistics” on page 101.) This should

Analysis (of results)

determines if this plan will result in unique results, analyzing only those tables in the query that are used in the join index.

IF the results will be... THEN the Optimizer...

Unique skips the sort-delete steps used to remove duplicates.

Nonunique determines whether eliminating all duplicates can still produce a valid plan, recognizing any case where:

• No field_name parenthetical clause exists

• All logical rows will be accessed

In this phase… The system...

Performance Management 91

Page 92: Teradata Database Performance Management

Chapter 7: System Performance and SQLJoin Indexes

not exact a very high cost because Teradata Database can collect statistics while queries are in process against the base table.

Cost Considerations

Join indexes, like SIs, incur both space and maintenance costs. For example, Insert, Update, and Delete requests must be performed twice: once for the base table and once for the join index.

Space Costs

The following formula provides a rough estimate of the space overhead required for a join index:

Join Index Size = U * ( F + O + (R * A)

where:

Updates to the base tables can cause a physical join index row to split into multiple rows. The newly formed rows each have the same fixed field value but contain a different list of repeated field values. This applies specifically when the compressed join index format is being used.

The system, however, does not automatically recombine logically related split rows. To re-compact such rows, you must drop and recreate the join index.

Maintenance Costs

The use of a join index entails:

• Initial time consumed to calculate and create the index

• Whenever a value in a join-index column of the base table is updated, the join index must also be updated, including any required aggregation or pre-join effort.

However, if join indexes are suited to your applications, the improvements in query performance can far outweigh the costs.

Join indexes are maintained by generating additional AMP steps in the base table update execution plan. Those join indexes defined with outer joins will usually require additional steps to maintain any unmatched rows.

Parameter Description

F Length of the fixed field <join-index-field1>

R Length of a single repeating field <join-index-field2>

A Average number of repeated fields for a given value in<join-index-field1>

U Number of unique values in the specified <join-index-field1>

O Row overhead (assume 14 bytes)

92 Performance Management

Page 93: Teradata Database Performance Management

Chapter 7: System Performance and SQLJoin Indexes

Expect a single-table join index insert operation to have similar maintenance overhead as would an insert operation with an equivalent NUSI. Updates or deletes, however, may incur greater overhead with a single-table join index, unless a value for the PI of the join index is available at the time of the update.

Overhead for an in-place aggregate join index can be perhaps three times more expensive than maintaining the same table without that index. For an aggregate join index that redistributes rows, the maintenance overhead can be several times as expensive.

Maintenance overhead for multitable join indexes without aggregates can be small or very large, depending on the pre-join effort involved in constructing or changing a join index row. This could be up to 20 times or more expensive than maintaining the table without the index. The overhead is greater at higher hits per block, where hits means the number of rows in a block are touched.

Since Teradata Database writes a block only once regardless of the number of rows modified, as the number of hits per block increases:

• The CPU path/transaction decreases (faster for the case with no join index than for the case with a join index)

• Maintenance overhead for aggregate join indexes decreases significantly

If a DELETE or UPDATE statement specifies a search condition on the PI or SI of a join index, the join index may be directly searched for the qualifying rows and modified accordingly.

This direct-update approach is employed when the statement adheres to these requirements:

• A PI or SI access path to the join index

• If a <join-index-field2> is defined, little or no modification to the <join-index-field1> columns

• No modifications to the join condition columns in the join index definition

• No modifications to the PI columns of the join index

It is not necessary to drop the join index before a backup. It is important, however, to drop join indexes before the underlying tables and databases are restored, should a restore ever be required. Otherwise an error is reported and the restore will not be done.

Join Index Versus NUSI

A join index offers the same benefits as a standard SI in that it, like the standard SI, is:

• Optional

• Defined by you

• Maintained by the system

• Transparent to the user

• Immediately available to the Optimizer

• If a covering index, considered by the Optimizer for a merge join

However, a join index offers the following performance benefits.

Performance Management 93

Page 94: Teradata Database Performance Management

Chapter 7: System Performance and SQLJoin Indexes

See also “Secondary Indexes” on page 84.

For more information on the syntax, applications, restrictions, and benefits of join indexes, see SQL Data Manipulation Language and Database Design.

Join Index Optimizations

The Optimizer:

• Selects cost-based query rewrites using the best available aggregate join index when several possible aggregate join indexes are available.

• Provides a larger number of opportunities to perform cost-based rewrites of requests using aggregate join indexes (AJIs) for queries with subqueries, spooled derived tables, outer joins, count(distinct), and extended grouping sets.

When a user creates multiple AJIs, the creation of the current AJI makes use of an existing AJI that is the most efficient for the calculation of this AJI so that the CREATE JOIN INDEX statement will have better performance.

With the existence of multiple JIs, including AJIs and non-AJIs, aggregate queries will perform better with the cost-based rewrite and more chances to use an AJI.

• Uses join indexes with Partial Group By (PGB) optimizations during join planning, making it possible to produce better join plans.

IF a join index is… THEN performance improves by…

defined using joins on one or more columns from two or more base tables

eliminating the need to perform the join step every time a joining query is processed.

used for direct access in place of some or all of its base tables, if the Optimizer determines that it covers most or all of the query.

eliminating the I/Os and resource usage required to access the base tables.

limited to only certain data types of your choice, such as Date

allowing direct access to the join index rows within the specified value-order range.

a single-table join index with a FK PI reducing I/Os and message traffic because row redistribution is not required, since the following are hashed to the same AMP:

• A single-table join index having a PI based on the base table foreign key.

• The table with the column(s) making up the foreign key.

defined with an outer join • Giving the same performance benefits as a single-table join index, for queries that reference only outer tables.

• Preserving unmatched rows.

created using aggregates eliminating both the aggregate calculation(s) and the join step for every query requiring the join and aggregate.

94 Performance Management

Page 95: Teradata Database Performance Management

Chapter 7: System Performance and SQLSparse Indexes

Sparse Indexes

Using sparse indexes (a form of join indexes), you can index a portion of the table using WHERE clause predicates to limit the rows indexed. This capability is implemented using join index technology.

Allowing constant expressions in the WHERE clause of the CREATE JOIN INDEX statement gives you the ability to limit the rows that are included in the join index to a subset of the rows in the table based on an SQL query result. This capability in effect allows you to create sparse indexes.

When base tables are large, you can use this feature to reduce the content of the join index to only the portion of the table that is frequently used if the typical query only references a portion of the rows.

It is important for statistics to be collected on the sparse-defining column of the join index, or the join index may not be selected for use.

Performance Impact

A sparse index can focus on the portions of the tables that are most frequently used. This capability:

• Reduces the storage requirements for a join index.

• Makes the costs for maintaining an index proportional to the percent of rows actually referenced in the index.

• May make query access faster because the join index is smaller.

Joins and Aggregates On Views

Views and Performance

Teradata Database performs joins and aggregate queries on views containing aggregates, eliminating the need for temporary tables.

The overhead associated with temporary tables decreases because you can eliminate:

• Creating temporary tables

• Deleting temporary tables

• Having a set of I/Os from, and to, the temporary table

Operations Available with Joins and Aggregates on a View

When you perform joins and aggregate queries on a view, you can:

• Use aggregated values in arithmetic expressions

• Perform an aggregate on aggregations

• Perform an aggregate before a join to replace code values with names

Performance Management 95

Page 96: Teradata Database Performance Management

Chapter 7: System Performance and SQLJoins and Aggregates On Views

• Control the join order of some of the tables

• Save building some temporary tables

A view might contain a date, a source and a destination, a count, and some sums.

Example 1

You want to create a report for a set of times and for each destination that includes an average and a maximum value of the count and sums. The purpose of the report is to determine potential loss of revenue by destination. To create this report, enter:

CREATE VIEW Loss_Summary_View(week, from_code, to_code, count_a, sum_x, sum_y, sum_z)AS SELECT

C.week, H.from_code, H.to_code, COUNT(H.a),SUM(H.x), SUM(H.y), SUM(H.z)

FROM History H , Calendar CWHERE C.month = 100610AND C.day = H.dayGROUP BY 1, 2, 3 ;SELECT LSV.week, LD.to_location, AVG(LSV.count_a), MAX(LSV. count_a), AVG(LSV. sum_x), MAX(LSV. sum_x), AVG(LSV. sum_y), MAX(LSV. sum_y), AVG(LSV. sum_z), MAX(LSV. sum_z) FROM Loss_Summary_View LSV, Location_Description LDWHERE LSV.to_code = LD.to_codeGROUP BY 1, 2;

Example 2

In the following example, join the CustFile table with the CustProdSales view (which contains a SUM operation) to determine which companies purchased more than $10,000 worth of item 123:

CREATE VIEW CustProdSales (custno, pcode, sales)AS

SELECT custno, pcode, SUM(sales)FROM SalesHistGROUP BY custno, pcode;

SELECT company_name, salesFROM CustProdSales a, CustFile b

WHERE a.custno = b.custnoANDa.pcode = 123ANDa.sales > 10000;

Business View Columns

Airline week, from_city, to_city, # flights, # passengers, # empty seats, revenue

Telco month, from_city, to_city, # calls, # minutes, # dropped calls, revenue

Manufacturer day, from_city, to_city, # shipments, # items shipped, # returns, revenue

96 Performance Management

Page 97: Teradata Database Performance Management

Chapter 7: System Performance and SQLPartial GROUP BY and Join Optimization

Partial GROUP BY and Join Optimization

The Optimizer considers when early aggregations can be done and whether they are cost- optimal. In other words, applying early partial Group By is considered part of query optimization.

Applying a partial Group By pushes aggregations before joins in order to optimize query execution. In formulating a query execution plan, the Optimizer automatically considers applying a partial GROUP BY as soon as possible in order to reduce the number of working rows.

As soon as the number of working rows is reduced, not only is the problem of running out of spool space avoided (a problem if a table is very large), but join steps, once a partial GROUP BY is applied, run faster without requiring the user to rewrite the query.

The Optimizer will apply a partial GROUP BY to query execution if both of the following conditions are satisfied:

• If it is semantically correct to apply early aggregations.

• If it is estimated to be more cost effective.

Large Table/Small Table Joins

Large Table/Small Table (LT/ST) joins combine three or more small tables with one large table.

The Optimizer algorithm:

• Looks for the large relation

• Analyzes connections to each index

• Analyzes non-indexed case

It is important to collect statistics on:

• Join and select indexes

• Small table PIs

• Selection columns, especially if the join is highly selective

• Join columns, especially if the join to the large table is weakly selective

Consider the following points about LT/ST and indexes:

• Indexes are an important factor in join performance.

• Consider the choice of indexes.

• Consider indexes on common-join column sets in large tables.

If the PI of a large table can be made up of elements from the small tables, the Optimizer uses a product join on the small tables. With the PI of the large table, the Optimizer can do a merge join and not read the entire large table, which is much more efficient use of system resources.

Performance Management 97

Page 98: Teradata Database Performance Management

Chapter 7: System Performance and SQLLarge Table/Small Table Joins

Example

For example, you want to examine the sales of five products at five stores for a one-week time period. This requires joining the Stores table (Table B), the Week_Ending_Date table (Table A), and the Product_List table (Table C) with the Daily_Sales table (Table L). The following figure illustrates this join.

Selected portions of the Stores table, Week_Ending_Date table and Product_List table are product-joined. The result creates the PI for the Daily_Sales table. The joined small tables are then joined with the large table, and an answer set is returned. This plan uses significantly fewer system resources and requires less processing time.

KY01A010

Spool

1 2NUPI UPI

3UPI

Week_Ending_Date Stores Product_List

Product Joins

Merge Joins

Daily_Sales

1UPI

2 3

98 Performance Management

Page 99: Teradata Database Performance Management

Chapter 7: System Performance and SQLStar Join Processing

Star Join Processing

A star join schema is one in which one of the tables, called the fact table, is connected to a set of smaller tables, called the dimension tables, between which there is no connection.

The fact table has a multipart key. The set of smaller dimension tables has a single-part PK that corresponds exactly to one of the components of the multipart key in the fact table.

Star Join Queries

Star join queries do not place selection criteria directly on a dimension table. Rather they place an IN condition on the PK of the dimension table that is stored in the fact table. The IN list behaves as if it were a dimension table, thus allowing star join processing to occur in cases where normally the dimension table would have been required.

Performance Value

The Optimizer can apply star join processing to queries that join a subset of PI/NUSI columns of a large table to small tables and qualify the remaining PI/NUSI columns with IN conditions.

Partitioned Primary Index

The PPI feature enables you to set up databases that provide performance benefits from data locality, while retaining the benefits of scalability inherent in the hash architecture of Teradata Database.

This is achieved by hashing rows to different AMPs, as is done with a normal PI, but creating local partitions within each AMP.

Nonpartitioned Primary Index

A traditional nonpartitioned PI allows the data rows of a table to be:

• Hash partitioned (that is, distributed) to the AMPs by the hash value of the PI columns

• Ordered by the hash value of the PI columns on each AMP

Partitioned Primary Index

PPI allows the data rows of a table to be:

• Hash partitioned to the AMPs by the hash of the PI columns

• Partitioned on some set of columns on each AMP

• Ordered by the hash of the PI columns within that partition

PPI introduces syntax that you can use to create a table or noncompressed join index with a PPI and to support the index. The table may be a base, global temporary, or volatile table.

Performance Management 99

Page 100: Teradata Database Performance Management

Chapter 7: System Performance and SQLPartitioned Primary Index

One or more partitioning expressions can be defined for a PI. The syntax supports altering a PPI along with changes, for example, to the output of various support statements. You can use two functions, RANGE_N and CASE_N, to simplify the specification of a partitioning expression.

Performance Impact

PPI improves performance as follows:

• Uses partition elimination (static, delayed, or dynamic) to improve the efficiency of range searches when, for example, the searches are range partitioned.

• Provides an access path to the rows in the base table while still providing efficient join strategies

Moreover, if the same partition is consistently targeted, the part of the table updated may be able to fit largely in cache, significantly boosting performance.

Performance Considerations

Performance tests indicate that the use of PPI can cause dramatic performance improvements both in queries and in table maintenance.

For example, NUSI maintenance for insert and delete requests can be done a block at a time rather than a row at a time. Insert and delete requests done this way show a reduction in I/O per transaction. The reduction in I/O in turn reduces the CPU path need to process the I/Os.

But be aware of the following:

• While a table with a properly defined PPI allows overall improvement in query performance, certain individual workloads involving the table, such as PI selects, where the partitioning column criteria is not provided in the WHERE clause, may become slower.

• There are potential cost increases for certain operations, such as empty table INSERT . . . SELECTs.

• Performance slowdown varies directly with the number of partitions.

In cases where little or no partition elimination can be done via conditions supplied in the WHERE clause, Teradata Database has to execute the join step on each partition (this is called the “sliding window”). In cases where the table is being joined to another table that is partitioned in the same way and there are equality conditions between the partitions, the “slide” can effectively be eliminated because each partition only needs to be joined with its counterpart in the other table. In all other cases, performance slowdown increases proportionally with the number of partitions.

• You must carefully implement the partitioning environment to gain maximum benefit. Benefits that are the result of using PPI vary based on:

• The number of partitions defined.

• The number of partitions that can be eliminated given the query workloads.

• Whether or not you follow an update strategy that takes advantage of partitioning.

For detailed information on PPIs, see the Orange Book, Partitioned Primary Index Usage.

100 Performance Management

Page 101: Teradata Database Performance Management

Chapter 7: System Performance and SQLMultilevel Partitioned Primary Index

Multilevel Partitioned Primary Index

Multilevel partitioning allows each partition of a partitioned PI table to be subpartitioned. Each level must define at least two partitions. The number of levels of partitioning cannot exceed 15. The limit is 65, 535 combined partitions, that is, the product of the number of partitions at each level. The number of levels of partitioning may be further restricted by other limits such as, for example, the maximum size of the table header or data dictionary entry sizes.

A Multilevel Partitioned Primary Index (MLPPI) can be used to improve query performance via partition elimination at each of the levels or a combination of levels. Like a single-level PPI table, an MLPPI provides an access path to the rows in the base table. As with other indexes, the Optimizer determines if the index is usable for a query and, if usable, whether its use provides the best cost plan for executing the query. No modification of the query is required.

For information on creating MLPPI tables, see the CREATE TABLE statement in SQL Data Definition Language. For design issues, see Database Design.

For issues with respect to using a RANGE_N or CASE_N function to build a partitioning expression, see SQL Functions, Operators, Expressions, and Predicates and, to a lesser extent, in the description of the ALTER TABLE statement in SQL Data Definition Language.

For more information on partitioning, see SQL Request and Transaction Processing, SQL Data Definition Language, and Database Design.

Partition-Level Backup, Archive, Restore

Backup, Archive, Restore (BAR) for table partitions allows backup and restore of selected table partitions for tables with a PPI.

Restore means that users can restore into a populated table and only overwrite those partitions indicated by a database administrator in the ARCMAIN script, using the PARTITIONS WHERE option.

Performance Considerations

Performance improves when partitions of a table rather than the entire table are involved in an operation.

Collecting Statistics

Statistics are data demographics input that the Optimizer uses. Collecting statistics, and keeping them fresh, ensures that the Optimizer will have the most accurate information to create the best access and join plans.

Performance Management 101

Page 102: Teradata Database Performance Management

Chapter 7: System Performance and SQLCollecting Statistics

Collecting Statistics and the Optimizer

Without collected statistics, the Optimizer uses very non granular random AMP-sampled data as well as fixed formulas, to guess at data demographics.

In queries that perform multiple table joins, statistics help the Optimizer to sequence the joins and to chose the most efficient join geography, as well as helping to produce accurate estimates of the spool files resulting from the joins.

Statistics are especially informative if index values are distributed unevenly. For example, when a query uses conditionals based on nonunique index values, Teradata Database uses statistics to determine whether indexing or a full search of all table rows is more efficient. You can use the EXPLAIN modifier for information on the proposed processing methods before submitting a request.

If Teradata Database determines that indexing is the best method, it uses the statistics to determine whether spooling or building a bitmap would be the most efficient method of qualifying the data rows. Teradata Database may consider bitmapping under certain conditions, such as if multiple NUSIs exist on a table and each NUSI is non-selective by itself and the query passes values for each NUSI.

The statistics are located in the DBC.TVFields and the DBC.Indexes tables of the Data Dictionary.

How the Optimizer Obtains Necessary Information

To create the AMP steps to carry out an SQL request in Teradata Database, the Optimizer needs to have the following information:

• Number of rows in the tables involved in the request

• Distribution of the specific data values for table selection

The Optimizer can obtain this information from either of two sources, one that is efficient, but less complete, and one that is less efficient, but more complete.

Source Comments

Random AMP Samples

If no statistics are available, the Optimizer uses random AMP samples for:

• Table row counts

• Number of distinct NUSI values

If statistics are not available on SIs, the Optimizer must make a rough estimate of the selectivity of indexed values and the possible number of distinct values in any NUSIs. This may result in inefficient use of, or no use of, SIs, especially in join processing.

Random AMP samples are less detailed and less reliable (especially if the index is nonunique and the distribution of primary data rows is lumpy) than collected statistics.

The Optimizer performs random AMP sampling dynamically during the parsing procedure and caches the samples for up to four hours.

102 Performance Management

Page 103: Teradata Database Performance Management

Chapter 7: System Performance and SQLCollecting Statistics

AMP Sampling

When statistics are not available, the Optimizer can obtain random samples from one or more AMPs when generating row count estimates for a query plan.

An AMP sample includes row count, row size, and rows per value estimates for a given table. These are passed to the Optimizer, resulting in improved join plans and better query execution times.

A one-AMP sampling of a table with heavily skewed data may result in wrong estimates of the row count information being passed to the Optimizer. With a multiple-AMP sampling, the Optimizer receives better estimates with which to generate a query plan, although a multiple-AMP sampling is more expensive than a one-AMP sampling.

All-AMP sampling is the default for volatile tables. A one-AMP sampling is the default for all other tables. Contact the Teradata Support Center for assistance in enabling multiple-AMP sampling.

Frequency Distribution of Data

Using the COLLECT STATISTICS option amasses statistics that include frequency distribution of user data.

Frequency distribution organizes the distinct values of an index or column into a group of intervals.

COLLECT STATISTICS Guidelines

Follow the guidelines below to use COLLECT STATISTICS.

Collected statistics

The Optimizer always uses available collected statistics because they are detailed and are more likely to be accurate than a random AMP sample.

However, you must actively request collection by submitting a COLLECT STATISTICS statement. Otherwise, they may become stale. Stale statistics may cause inefficient plans to be generated.

Although columns can be accessed by queries at the same time as statistics are being collected on them, COLLECT STATISTICS is resource intensive and can slow down the performance of queries running at the same time.

Source Comments

Task Guideline

Collect UPI statistics on small tables

If you collect no other statistics on the table, collect UPI statistics. The table is small (that is, 100 rows/AMP).

Performance Management 103

Page 104: Teradata Database Performance Management

Chapter 7: System Performance and SQLCollecting Statistics

Statistics on Skewed Data

When you collect statistics for skewed data, the Optimizer can accommodate exceptional values.

Statistics reveal values that include the most nonunique value in the table and the most nonunique value per value range. The system maintains statistics on the most nonunique value/range.

Collect NUPI statistics The NUPI is:

• Fairly or highly unique, and

• Used commonly in joins, or

• Skewed

• Small

Collect NUSI statistics on all NUSIs

The Optimizer can use the NUSI in range scans (BETWEEN... AND...).

With statistics available, the system can decide to hash on the values in the range if demographics indicate that to do so would be less costly than a full table scan.

Without statistics, the index will likely not be used and, in some cases, the NUSI may be used when it is not efficient to do so.

Collect NUSI statistics on covering (ALL option) NUSIs

If an SI is defined with the intent of covering queries (the ALL option is specified), you should consider collecting statistics even if the indexed columns do not appear in WHERE conditions.

Collecting statistics on a potentially covering NUSI provides the Optimizer with the total number of rows in the NUSI subtable and allows the Optimizer to make better decisions regarding the cost savings from covering.

Collect NUSI statistics on NUSIs with ORDER BY

If a sort key is specified in a NUSI definition with the ORDER BY option, collect statistics on that column so the Optimizer can compare the cost of using a NUSI-based access path in conjunction with a range or equality condition on the sort key column.

Collect non-index column statistics on all non-indexed columns used for table selection

Collecting statistics on a group of columns allows the Optimizer to estimate the number of qualifying rows for queries that have search conditions on each of the columns or that have a join condition on each of the columns.

Refresh statistics after updates When:

• The number of rows changed is greater than 10%.

• For PPI tables >10% for any partition.

• The demographics of columns with collected statistics changes.

Drop statistics If statistics are no longer needed because no queries are using them, they should be dropped.

Task Guideline

104 Performance Management

Page 105: Teradata Database Performance Management

Chapter 7: System Performance and SQLCollecting Statistics

Without statistics, the system relies on random AMP sampling. It can be quite inaccurate when the table is small or unevenly distributed. Small tables often distribute unevenly.

Collecting Statistics for Join Index Columns

• For non-sparse join and hash indexes, the Optimizer inherits the rowcount and all histograms from the base tables. Teradata recommends that you collect statistics on base tables only and not on the non-sparse join or the hash indexes.

• For sparse join indexes, Teradata recommends that you collect statistics on base tables and sparse join indexes separately.

• For non-compressed join indexes, the Optimizer performs all-AMP sampling to estimate the join index rowcount accurately.

• For compressed sparse join indexes, Teradata recommends that you collect PI statistics. The All-AMP random sampling that the Optimizer performs may not reflect an accurate rowcount because of the compression.

Statistics Collection on Sample Data

Without collected statistics, query performance can suffer because the Optimizer does not have the information it needs to choose access paths efficiently. However, collecting statistics can be very time consuming on large tables because the task performs a full-table scan and sorts the data to determine the number of occurrences of each distinct value.

Collecting statistics on small tables is important, but should be fairly fast. Also, collecting statistics on the system-derived PARTITION column is fairly fast without respect to table size.

Given this, you may choose to specify statistics collection on a sample of the data, instead of all the data. Collecting statistics on a sample significantly reduces the disk I/O required to read the data and the CPU time required to sort it.

You can specify optional USING SAMPLE keywords in the COLLECT STATISTICS statement to reduce the time consumed to collect statistics, but this comes at a loss of statistical accuracy.

PARTITION Statistics

Teradata Database provides the user with the ability to collect with “PARTITION statistics” based on partition numbers rather than column values. This enables the Optimizer to estimate cost operations more accurately involving PPI table.

The Optimizer is provided with:

• The number of partitions that are non-empty

• How the rows are distributed among partitions.

PARTITION is a system-derived column whose value is dynamically generated for all tables having a PPI. The RowID of each row in the table contains the internal partition number in which that particular row is stored. The system dynamically converts this internal partition number to the external partition number seen by a user as the value of the PARTITION column.

Performance Management 105

Page 106: Teradata Database Performance Management

Chapter 7: System Performance and SQLCollecting Statistics

Collecting statistics on PARTITION enables the collection of internal partition numbers for the individual rows of a table defined with a PPI. Partition statistics can be collected for just the PARTITION column (single-column partition statistics) or on the PARTITION column and other table columns (multicolumn PARTITION statistics). Collecting statistics on the actual partitioning columns themselves is also recommended. When the Optimizer has this information, it is better able to calculate the relative cost of various methods of optimizing a query over a PPI table.

Having PARTITION statistics allows the Optimizer to generate a better plan with respect to PPI tables. For example, the Optimizer can cost joins involving PPI tables more accurately with PARTITION statistics.

Sampled Statistics Usage Considerations

Sampled statistics are generally appropriate for:

• Very large tables

• Uniformly distributed data

• Indexed or non-indexed column(s)

You should consider sampled statistics, as specified by the USING SAMPLE option, when collecting statistics on very large tables and where resource consumption from the collection process is a performance concern.

Do not use sampled statistics on small tables or as a wholesale replacement for existing collections. Rather, consider sampling whenever the overhead from full scan statistics collection (most notably CPU costs) is of great concern to the customer.

Sampling may degrade the quality of the resulting statistics and the subsequent query plans the Optimizer chooses. Thus, sampled statistics are generally more accurate for data that is uniformly distributed. For example, columns or indexes that are unique or nearly unique are uniformly distributed. Do not consider sampling for highly skewed data because the Optimizer needs to be fully aware of such skew.

In addition to uniformly distributed data, sampling can be more accurate for indexes than non-indexed column(s). For indexes, the scanning techniques employed during sampling can take advantage of the hashed organization of the data to improve the accuracy of the resulting statistics.

When sampling, you need not specify the percentage of rows of the table to sample. By default, Teradata Database uses a 2% sample. If, USING sample collections on a NUPI Teradata Database detects any skewing, it will increase the sample bit by bit to as high as 50% until it determines that the sample percent is in line with the observed skewing.

Stale Statistics

Statistics provide more detailed information and include an exact row count as of the time that the statistics were gathered.

If the statistics are stale, however, that is, if the table's characteristics (distribution of data values for a column or index for which statistics have been collected, number of rows in the

106 Performance Management

Page 107: Teradata Database Performance Management

Chapter 7: System Performance and SQLCollecting Statistics

table, and so on) have changed significantly since the statistics were last gathered, the Optimizer can be misled into making poor join plans. This results in the poor performance of queries which use the stale statistics.

Example

If for table A, statistics were gathered when table had 1,000 rows but now the table has 1,000,000 rows (perhaps statistics were gathered during the prototyping phase), and if for table B no statistics were gathered but now the table has 75,000 rows, then if a product join between table A and table B is necessary for a given query, and one of the tables must be duplicated on all AMPs.

Then the Optimizer will choose table A to be duplicated, since 1,000 rows (from the stale statistics) is much less than 75,000 rows.

Since in reality Table A now has 1,000,000 rows, the Optimizer will make a very poor decision (duplicating 1,000,000 rows instead of 75,000), and the query will run much longer than necessary.

Circumstances that Cause Stale Statistics

Statistics are stale when:

• The number of rows in the table or a partition has changed significantly.

The number of unique values for each statistic on a table, as well as the date and time the statistics were last gathered, can be obtained by:

HELP STATISTICS tablename;

More detailed information can be obtained by specifying a specific column or set of columns for which statistics have been collected.

For statistics on unique indexes, HELP STATISTICS can be cross-checked by comparing the row count returned by:

SELECT COUNT(*) FROM tablename;

For statistics on nonunique columns, the HELP STATISTICS result can be cross-checked by comparing the count returned by:

SELECT COUNT(DISTINCT columnname) FROM tablename;

• The range of values for an index or column of a table for which statistics have been collected has changed significantly.

Sometimes you can infer this from the date and time the statistics were last collected, or by the very nature of the column.

For example, if the column in question holds a transaction date, and statistics on that column were last gathered a year ago, it is almost certain that the statistics for that column are stale.

Refreshing Stale Statistics

Teradata recommends that you re-collect statistics if as little as a 10% change (rows added or deleted) in a table or partition has occurred.

Performance Management 107

Page 108: Teradata Database Performance Management

Chapter 7: System Performance and SQLEXPLAIN Feature and the Optimizer

For high volumes of very nonunique values such as dates or timestamps, it may be advantageous to recollect at 7%.

If the statistics for a table are stale, they can be easily re-collected. The following statement:

COLLECT STATISTICS ON tablename;

will re-collect statistics on all indexes and columns for which previous COLLECT STATISTICS statements were done (and for which DROP STATISTICS statements have not been done).

Because collecting statistics involves a full-table scan, collecting them may take a significant amount of time. Collecting statistics should, therefore, be done off-hours for large tables.

You may want to execute the Help Statistics statement before and after re-collecting statistics to see what, if any, difference the recollect makes.

Moreover, for frequently executed queries, requesting an EXPLAIN before and after recollecting statistics may show differences in join plans and/or spool row count/processing time estimates.

EXPLAIN Feature and the Optimizer

The EXPLAIN request modifier is one of the most valuable tools in Teradata Database for understanding how the Optimizer works. When you put EXPLAIN in front of any SQL request and execute the request, the Optimizer returns a description of how the request is broken into AMP steps for processing.

EXPLAIN quickly highlights missing statistics, unused indexes, and so on. By utilizing the information available from EXPLAIN, you may be able to prevent some performance problems from occurring.

Use EXPLAIN to… Comments

identify SIs by index number.

EXPLAIN output identifies SIs used for joins and accesses by internal Teradata Database index number, as well as by column name.

To obtain a list of SIs by internal number for each of your tables, run the SELECT statement. For example, you might enter the following SELECT statement for a list of SIs on the Message table in the RST database.

SelectColumnName,IndexNumber

FromDBC.IndicesVWhereDatabaseName = ‘RST’And TableName = ‘Message’Order ByIndexNumber;

check if AMP plans for your joins are what you expected.

It is crucial that you develop the skill of following row counts through an EXPLAIN to identify where the Optimizer has made an assumption different than the actual data.

108 Performance Management

Page 109: Teradata Database Performance Management

Chapter 7: System Performance and SQLEXPLAIN Feature and the Optimizer

The following may change and affect EXPLAIN output:

• Data volatility

• Distribution

• Software release level

• SIs, space requirements and maintenance

• Design (see “Database Design Phases” on page 207)

• Collecting or dropping statistics

• Changes in the data demographics

• Actual value of USING variables, parameters, and built-in functions

Keep a file of EXPLAIN output over time to identify processing changes and revisit index selection accordingly.

For more information on EXPLAIN, see SQL Data Manipulation Language and Database Design.

EXPLAIN Output

Teradata Database supports the following Optimizer plan detail in EXPLAIN output:

• Cost estimates in milliseconds for Merge, Merge Delete, Merge Update steps, Insert, Update, Upsert, Delete steps.

• Spool size estimates. The number of rows the spool contains and its size in bytes.

• View names either instead of, or as well as, table names.

• Indication of which column or columns are used for resequencing and/or redistributing spool results. This aids in debugging complex queries, especially those suspected of skewing intermediate results onto a single AMP.

• Conditions evaluated by steps.

• Indication of grouping columns for aggregates

evaluate the use of common and parallel steps.

EXPLAIN identifies serial, parallel, and common steps, including:

• Any indexes that the Optimizer will use to select rows

• The sequence of intermediate spool files that might be generated for joins

The Optimizer creates common steps when different SQL statements need the same steps. EXPLAIN does not note common steps. You must recognize that a spool is being reused.

uncover hidden or nested views.

EXPLAIN displays the resolution of views down to the base tables so that you can identify obscure or nested views.

detect data movement. See “Using EXPLAIN to Detect Data Movement” on page 110.

Use EXPLAIN to… Comments

Performance Management 109

Page 110: Teradata Database Performance Management

Chapter 7: System Performance and SQLEXPLAIN Feature and the Optimizer

Using EXPLAIN to Detect Data Movement

Data movement includes duplication and redistribution in join plans.

Example 1

To look at data movement of a join with duplication, you enter an EXPLAIN request modifier, as follows:

EXPLAINSELECT a.c1, a.c4, a.c7 FROM facttable aINNER JOIN dimension1 b ON a.c1 = b.c1INNER JOIN dimension2 c ON a.c2 = c.c2

The output would be similar to the following:

Explanation-------------------------------------------------------

1 First, we lock a distinct JCH_STAR."pseudo table" for read on a RowHash to prevent global deadlock for JCH_STAR.c.

2 Next, we lock a distinct JCH_STAR."pseudo table" for read on a RowHash to prevent global deadlock for JCH_STAR.b.

3 We lock a distinct JCH_STAR."pseudo table" for read on a RowHash to prevent global deadlock for JCH_STAR.a.

4 We lock JCH_STAR.c for read, we lock JCH_STAR.b for read, and we lock JCH_STAR.a for read.

5 We do an all-AMPs RETRIEVE step from JCH_STAR.b by way of an all-rows scan with no residual conditions into Spool 2, which is duplicated on all AMPs. Then we do a SORT to order Spool 2 by row hash. The size of Spool 2 is estimated to be 768 rows. The estimated time for this step is 0.04 seconds.

6 We execute the following steps in parallel.

a We do an all-AMPs JOIN step from Spool 2 (Last Use) by way of an all-rows scan, which is joined to JCH_STAR.a by way of a traversal of index # 4 extracting row ids only. Spool 2 and JCH_STAR.a are joined using a nested join with a join condition of ("JCH_STAR.a.c1 = Spool_2.c1"). The input table JCH_STAR.a will not be cached in memory. The result goes into Spool 3,

IF the table is a... THEN it...

facttable has columns c1, c2, c3, c4, c5, c6, c7, and c8 with PI c1, c2, c3, c7 and SI c1.

dimension1 has columns c1 and c4 with PI c1.

dimension2 has columns c2 and c5 with PI c2.

facttable, dimension1, and dimension2

belongs to database jch_star.

110 Performance Management

Page 111: Teradata Database Performance Management

Chapter 7: System Performance and SQLEXPLAIN Feature and the Optimizer

which is built locally on the AMPs. Then we do a SORT to order Spool 3 by field Id 1. The result spool file will not be cached in memory. The size of Spool 3 is estimated to be 2,574,931 rows. The estimated time for this step is 5 minutes and 34 seconds.

b We do an all-AMPs RETRIEVE step from JCH_STAR.c by way of an all- rows scan with no residual conditions into Spool 4, which is duplicated on all AMPs. The size of Spool 4 is estimated to be 768 rows. The estimated time for this step is 0.04 seconds.

7 We do an all-AMPs JOIN step from Spool 3 (Last Use) by way of an all- rows scan, which is joined to JCH_STAR.a. Spool 3 and JCH_STAR.a are joined using a row id join, with a join condition of ("JCH_STAR.a.c1 = Spool_3.c1"). The input table JCH_STAR.a will not be cached in memory. The result goes into Spool 5, which is built locally on the AMPs. The size of Spool 5 is estimated to be 2,574,931 rows. The estimated time for this step is 7 minutes and 45 seconds.

8 We do an all-AMPs JOIN step from Spool 4 (Last Use) by way of an all-rows scan, which is joined to Spool 5 (Last Use). Spool 4 and Spool 5 are joined using a single partition hash join, with a join condition of ("Spool_5.c2 = Spool_4.c2"). The result goes into Spool 1, which is built locally on the AMPs. The size of Spool 1 is estimated to be 2,574,931 rows. The estimated time for this step is 44.37 seconds.

9 Finally, we send out an END TRANSACTION step to all AMPs involved in processing the request.

10 The contents of Spool 1 are sent back to the user as the result of statement 1. The total estimated time is 0 hours and 14 minutes and 4 seconds.

Example 2

To look at data movement of a join with redistribution, you might use an EXPLAIN modifier, as follows:

EXPLAINSELECT a.c1, a.c4, a.c7 FROM facttable aINNER JOIN dimension1 b ON a.c4 = b.c4INNER JOIN dimension2 c ON a.c5 = c.c5;

The output would be similar to the following:

Explanation-------------------------------------------------------

1 First, we lock a distinct JCH_STAR."pseudo table" for read on a RowHash to prevent global deadlock for JCH_STAR.c.

2 Next, we lock a distinct JCH_STAR."pseudo table" for read on a RowHash to prevent global deadlock for JCH_STAR.b.

Performance Management 111

Page 112: Teradata Database Performance Management

Chapter 7: System Performance and SQLEXPLAIN Feature and the Optimizer

3 We lock a distinct JCH_STAR."pseudo table" for read on a RowHash to prevent global deadlock for JCH_STAR.a.

4 We lock JCH_STAR.c for read, we lock JCH_STAR.b for read, and we lock JCH_STAR.a for read.

5 We execute the following steps in parallel.

a We do an all-AMPs RETRIEVE step from JCH_STAR.c by way of an all-rows scan with no residual conditions into Spool 2, which is duplicated on all AMPs. The size of Spool 2 is estimated to be 768 rows. The estimated time for this step is 0.04 seconds.

b We do an all-AMPs RETRIEVE step from JCH_STAR.a by way of an all-rows scan with no residual conditions into Spool 3, which is built locally on the AMPs. The input table will not be cached in memory, but it is eligible for synchronized scanning. The result spool file will not be cached in memory. The size of Spool 3 is estimated to be 7,966,192 rows. The estimated time for this step is 9 minutes and 44 seconds.

c We do an all-AMPs RETRIEVE step from JCH_STAR.b by way of an all-rows scan with no residual conditions into Spool 4, which is redistributed by hash code to all AMPs. The size of Spool 4 is estimated to be 96 rows. The estimated time for this step is 0.04 seconds.

6 We do an all-AMPs JOIN step from Spool 2 (Last Use) by way of an all-rows scan, which is joined to Spool 3 (Last Use). Spool 2 and Spool 3 are joined using a single partition hash join, with a join condition of ("Spool_3.c5 = Spool_2.c5"). The result goes into Spool 5, which is redistributed by hash code to all AMPs. The size of Spool 5 is estimated to be 95,797 rows. The estimated time for this step is 5.34 seconds.

7 We do an all-AMPs JOIN step from Spool 4 (Last Use) by way of an all-rows scan, which is joined to Spool 5 (Last Use). Spool 4 and Spool 5 are joined using a single partition hash join, with a join condition of ("Spool_5.c4 = Spool_4.c4"). The result goes into Spool 1, which is built locally on the AMPs. The size of Spool 1 is estimated to be 10,505 rows. The estimated time for this step is 0.60 seconds.

8 Finally, we send out an END TRANSACTION step to all AMPs involved in processing the request.

9 The contents of Spool 1 are sent back to the user as the result of statement 1. The total estimated time is 0 hours and 9 minutes and 50 seconds.

112 Performance Management

Page 113: Teradata Database Performance Management

Chapter 7: System Performance and SQLIdentity Column

Identity Column

Identity Column (IdCol) is a column attribute option. When this attribute is associated with a column, it causes the system to generate a table-level unique number for the column for every inserted row.

IdCol values are returned as part of the response, if an AGKR (Auto Generated Key Retrieval) option flag in the request-level Options parcel is set, when an INSERT or INSERT . . . SELECT statement is executed.

Advantages

The main advantage of an IdCol is its ease of use in defining a unique row identity value. IdCol guarantees uniqueness of rows in a table when the column is defined as a GENERATED ALWAYS column with NO CYCLE allowed.

For some tables, it may be difficult to find a combination of columns that would make a row unique. If a composite index is undesirable, you can define an IdCol as the PI.

An IdCol is also suited for generating unique PK values used as employee numbers, order numbers, item numbers, and the like. In this way, you can get a uniqueness guarantee without the performance overhead of specifying a unique constraint.

Disadvantages

One disadvantage of an IdCol is that the generated values will have identity gaps whenever an Insert into a table having an IdCol is aborted or rows are deleted from the table.

Sequence is not guaranteed nor do IdCol values reflect the chronological order of the rows inserted.

Moreover, once a table with an IdCol is populated, deleting all the rows in the table and reinserting new rows will not cause the numbering to restart from 1. Numbering will continue from the last generated number of the table.

To restart numbering from 1, drop the table and re-create it before reloading the rows. Do not use IdCols for applications that cannot tolerate gaps in the numbering. Identity gaps are more of an issue with applications using IdCols for auto-numbering employees, orders, and so on.

Performance Considerations

There is minimal cost with respect to system performance when using IdCol. However, the initial load of an IdCol table may create an initial performance hit since every vproc that has rows will need to reserve a range of numbers at about the same time.

When the table to be updated has a NUSI or a USI, there will be performance degradation for Inserts and Updates if the IdCol is on the PI. When the IdCol is on a column other than the PI, the performance cost is negligible.

For users writing applications that use IdCol, having the IdCol values returned improves open access product performance.

Performance Management 113

Page 114: Teradata Database Performance Management

Chapter 7: System Performance and SQL2 PC Protocol

2 PC Protocol

Two-Phase Commit (2PC) is an IMS and CICS protocol for committing update transactions processed by multiple systems that do not share the same locking and recovery mechanism.

For detailed information on 2PC, see Teradata Director Program Reference.

Performance Impact

Consider the following disadvantages of using the 2PC protocol:

• Performance may decrease because, at the point of synchronization, up to two additional messages are exchanged between the coordinator and participant, in addition to the normal messages that update the database

If your original Teradata Database SQL request took longer to complete than your other requests, the performance impact due to the 2PC overhead will be less noticeable.

• • If Teradata Database restarts, and a session using the 2PC protocol ends up in an IN-DOUBT state, Teradata Database holds data locks indefinitely until you resolve the IN-DOUBT session. During this time, other work could be blocked if it accesses the same data for which Teradata Database holds those locks.

To resolve this situation, perform the following steps:

• Use the COMMIT/ROLLBACK command to resolve manually the IN-DOUBT sessions.

• Use the RELEASE LOCKS command.

• Use the RESTART command to restart your system.

2PC causes no system overhead when it is disabled.

Updatable Cursors

In ANSI mode, you can define a cursor for the query results and for every row in the query results in order to update or delete the data row via the cursor associated with the row.

This means that update and delete operations do not identify a search condition; instead, they identify a cursor (or a pointer) to a specific row to be updated or deleted.

Pocketable cursors allows you to update each row of a select result independently as it is processed.

Recommendations

To reap the full benefit from the updatable cursor feature, you should minimize:

• The size of query result and number of updates/transaction

• The length of time you hold the cursor open

Using many updates per cursor may not be optimal because:

114 Performance Management

Page 115: Teradata Database Performance Management

Chapter 7: System Performance and SQLNon-Key Access Paths

• They block other transactions.

• The system requires longer rollbacks.

In this case, use the MultiLoad utility to do updates.

Non-Key Access Paths

The following improve Optimizer access path planning. Support for:

• Constraint scans for single SELECT statements and join statements on base tables and join indexes.

• Integration of LikeScan into constraint scan.

• Integration of RangeScan for non-value-ordered (non-VO) column SIs into constraint scan.

• BitMap scan on indexes used for constraint scan.

• IN-list access path for aggregation statements and joins between tables when the IN-list is on the NUSI of a base table and join index.

To enable the above, contact the Teradata Support Center.

Better use of access paths to base tables and join indexes improves query performance.

Performance Management 115

Page 116: Teradata Database Performance Management

Chapter 7: System Performance and SQLNon-Key Access Paths

116 Performance Management

Page 117: Teradata Database Performance Management

CHAPTER 8 Database Locks and Performance

This chapter provides information on handling database locks in order to improve performance.

Locking Overview

When multiple transactions need to perform work that requires a nonshareable lock on the same object, Teradata Database controls concurrency by:

• Granting a lock to the transaction that requests access first.

• Queuing subsequent transactions.

• Releasing the lock and granting a new lock to the oldest transaction in the queue, when the current transaction completes.

For a complete discussion of locks and locking, see Database Administration.

Locking Mechanisms

On Teradata Database, the following mechanisms exist:

• A mechanism that determines whether the transaction limit for a locking queue has been reached and, if so, sends an error message to the session owning any transaction needing to be added to the same queue.

• A hung-transaction detection mechanism that detects and aborts transactions hung due to system errors.

• A deadlock detection mechanism that detects and aborts and rolls back the youngest transaction and sends an error message to the session owning that transaction.

• A user-tunable value that determines the time interval between deadlock detection cycles.

• The system includes two internal timers, but no time-out mechanism exists for transactions waiting in a queue. However, the MultiLoad client utility can time-out MLOAD transactions waiting for over 50 seconds (see Teradata MultiLoad Reference).

Most transactions on Teradata Database are processed without incurring a deadlock. For detailed information on lock compatibility and contentions, as well as information on the Lock Display (LokDisp) utility, see Utilities.

Performance Management 117

Page 118: Teradata Database Performance Management

Chapter 8: Database Locks and PerformanceGuidelines for Preventing Excessive Deadlocks

Guidelines for Preventing Excessive Deadlocks

• Except with CREATE INDEX, use LOCKING FOR ACCESS whenever dirty reads are acceptable.

• Beware of BTEQ handling of transaction processing. After transaction rollback, BTEQ continues the transaction from the point of failure, not at the beginning of the transaction!

• Set the DeadLockTimeout field via the DBS Control to 30 seconds if you have a mix of DSS and PI updates on fallback tables.

• Be sure to use RELEASE LOCKS on Archive/Recovery jobs.

• Use the Locking Logger utility to monitor and detect locking problems.

• Use the LOCKING ROW [FOR] WRITE/EXCLUSIVE phrase preceding a transaction. This phrase does not override any lock already being held on the target table. LOCKING ROW is appropriate only for single table selects that are based on a PI or SI constraint. For example:

.

.LOCKING ROW FOR WRITESELECT y FROM tableA WHERE pi =1;UPDATE tableA SET y=0 WHERE pi =1;..

• In macros, use multistatement requests instead of Begin Transactions (BT)/End Transactions (ET) to minimize table-level deadlocking. For example:

.

.LOCKING ROW FOR WRITESELECT y FROM tableA WHERE pi =1; UPDATE tableA SET y=0 WHERE pi =1 ;..

This causes all the necessary locks to be applied at the start, which avoids the potential for a deadlock. Use the EXPLAIN modifier to check out the processing sequence.

• Be aware that when you are updating tables where several rows in the base table share a row in the join index subtable (such as in most aggregate join index cases), there can be considerable collision on the same row.

Locking Logger and Performance

Do not run Logging Logger with a continuous download of a buffer into a table. You only want to offload the circular buffer periodically, not continuously.

You can allocate more sessions to perform the download if you think you need them, but the download can be successfully with only 2 sessions, which usually results in the download of the buffer taking a couple of minutes.

118 Performance Management

Page 119: Teradata Database Performance Management

Chapter 8: Database Locks and PerformanceTransaction Rollback and Performance

If one session blocks another and kills one session, that does not cause an entry to be made in the Locking Logger log. Global deadlocks are supposed to get written to the log.

Because Locking Logger records at a very granular level, which can result in a great deal of output to review, use the Locking Logger filter. You get more useful data fitting into 1 buffer.

Transaction Rollback and Performance

A rollback is a reversal of an incomplete database transaction. If a transaction fails to complete because the database restarts or the transaction is aborted, the system must remove any partially completed database updates from the affected user tables to assure data integrity.

Teradata Database maintains transaction integrity via the TJ (dbc.transientjournal). The TJ contains data about incomplete transactions, recording a “before image” of each modified table row.

Effects on Performance

Because a rollback can conceivably involve millions or even billions of rows, a rollback can affect the performance and availability of resources in Teradata Database while the rollback is in progress.

The rollback affects the performance of the system because the rollback competes for CPU with other users. Moreover, a rollback can keep locks on affected tables for hours, or even for days, until the rollback is complete. During a rollback, there is a trade-off between overall system performance vs. table availability.

Rollback Priority

In the event of a rollback, these rows are re-applied to the table at the priority specified in the DBS Control Record field, RollbackPriority.

How RollbackPriority affects performance is not always straightforward and is related to the Priority Scheduler configuration, job mix, and other processing dynamics. For a discussion of see “RollbackPriority” on page 166.

Minimizing or Avoiding Rollbacks

If you are doing a lot of deletes on rows from a table, consider using MultiLoad rather than BTEQ. MultiLoad completely avoids use of the TJ and is restartable.

To minimize the number of TJ rows being generated on an INSERT . . . SELECT request, consider doing a multistatement INSERT . . . SELECT request into an empty table from both of the other tables. This only puts one entry into the transient journal to let the DBS know that the target table is an empty table and to drop all the rows in the target table if the need for a rollback is encountered. After the new table is created, drop the old table and rename the new table to the name of the old table.

Performance Management 119

Page 120: Teradata Database Performance Management

Chapter 8: Database Locks and PerformanceTransaction Rollback and Performance

More details on both the delete rows and INSERT . . . SELECT request are available online in Support Link Knowledge Base.

Detecting a Rollback in Progress

Sessions in rollback may appear to have logged off in both DBC.LogOnOffV and DBC.AccessLogV, but this is not always the case. The logoff would depend on the manner in which the job was aborted. If you specify one of the following:

ABORT SESSION hostid.username LOGOFFABORT SESSION *.username LOGOFFABORT SESSION hostid.* LOGOFF

then the LOGOFF option would terminate the session.

Without it, the session should continue until the abort completes or the Supervisor issues a LOGOFF request. Unless an SQL job is explicitly coded to do otherwise, a session will also appear to have logged off if the system has undergone a restart.

The rollback or abort is independent of the session. It is actually handled by a completely different mechanism with internally allocated AMP worker tasks.

Example 1

To activate the RCVmanager, go to the Database Window and type "start rcvmanager". Then issue the "list rollback tables" command. It will show you each table that is being rolled back at that point in time, how many TJ rows have been rolled back and how many rows are remaining.

If you run this command twice, you can then make an estimate how long it will take the rollback to complete, based on the rows processed and rows remaining and the time between the two snapshots.

list rollback tables;

TABLES BEING ROLLED BACK AT 10:01:26 04/09/20

ONLINE USER ROLLBACK TABLE LIST

Host Session User ID Workload Definition AMP W/Count---- -------- --------- ----------------------------- 1 234324 0000:0001 24

TJ Rows Left TJ Rows Done Time Est.------------- ------------- --------- 53638 1814 00:09:51

Table ID Name--------- --------------------------0000:16A6 "FINANCE_T"."Order_Header"

SYSTEM RECOVERY ROLLBACK TABLE LIST

Host Session TJ Row Count---- -------- -------------

120 Performance Management

Page 121: Teradata Database Performance Management

Chapter 8: Database Locks and PerformanceTransaction Rollback and Performance

Table ID Name--------- ------------- ----

Enter command, "QUIT;" or "HELP;" :list rollback tables;

TABLES BEING ROLLED BACK AT 10:01:37 04/09/20

ONLINE USER ROLLBACK TABLE LIST

Host Session User ID Workload Definition AMP W/Count---- -------- --------- ------------------- ----------- 1 234324 0000:0001 24

TJ Rows Left TJ Rows Done Time Est.------------- ------------- --------- 52663 2789 00:09:45

Table ID Name--------- ---------------------------0000:16A6 "FINANCE_T"."Order_Header"

SYSTEM RECOVERY ROLLBACK TABLE LIST

Host Session TJ Row Count---- -------- -------------

Table ID Name--------- ------------------

Enter command, "QUIT;" or "HELP;" :

Example 2

A second way to identify the existence of a rollback exists is shown below:

Issue the command:

# rallsh -sv "/usr/ntos/bin/puma -c | grep -v ' 0 ' | grep MSGWORKABORT "

If this command returns any lines like:

MSGWORKABORT 3 999 1 2MSGWORKABORT 3 999 1 2MSGWORKABORT 3 999 1 2<... etc...>

on multiple successive samplings, then there is very likely a session in rollback. In a short, one-time-only sample, the tasks in MSGWORKABORT could also have been finishing up an END TRANSACTION and are not actually aborting. Since you are looking for high-impact, long-running aborts, look for some vproc(s) with tasks in MSGWORKABORT for several minutes.

The actual output of the "puma -c" command is:VPROC = 6WorkType Min Max Inuse PeakMSGWORKNEW 3 50 0 8MSGWORKONE 3 999 0 7

Performance Management 121

Page 122: Teradata Database Performance Management

Chapter 8: Database Locks and PerformanceTransaction Rollback and Performance

MSGWORKTWO 3 999 0 2MSGWORKTHREE 3 999 0 2MSGWORKFOUR 0 999 0 0MSGWORKFIVE 0 999 0 2MSGWORKSIX 0 999 0 1MSGWORKSEVEN 0 999 0 0MSGWORKEIGHT 0 999 0 0MSGWORKNINE 0 999 0 0MSGWORKTEN 0 999 0 0MSGWORKELEVEN 0 999 0 0MSGWORKABORT 3 999 2 2 <=Inuse shows 2 AWTs doing ABORTMSGWORKSPAWN 3 999 0 2MSGWORKNORMAL 3 999 0 3MSGWORKCONTROL 3 999 0 2

Look for a nonzero value in the inuse column for MSGWORKABORT. In the example above, two AMP Worker Tasks are being used in the abort process.

Following a restart, additional details on the rollback can be obtained from the RcvManager utility. For a description of this utility, see Utilities.

122 Performance Management

Page 123: Teradata Database Performance Management

CHAPTER 9 Data Management

This chapter discusses the impact of data management on performance.

Data Distribution Issues

The following table lists data distribution issues, causes, and results.

Identifying Uneven Data Distribution

Using SQL

You can use an SQL statement similar to the following to determine if data for a given table is evenly distributed across all AMP vprocs. The SQL statement displays the AMP with the most-used through the AMP with the least-used space, investigating data distribution in the Message table in database RST.

SELECT vproc,CurrentPermFROM DBC.TableSizeVWHERE Databasename = ‘RST’AND Tablename = ‘Message’ORDER BY 2 desc;

Issue Cause Results

Same row hash value for an excessive number of rows

• Rows with same row hash value cannot fit in a single data block.

• Rows spill over into additional data blocks.

• Highly nonunique PIs. As an estimate, more than 1000 occurrences/NUPI value begin to cause performance degradation problems. This figure is based on all the rows for the same NUPI value spilling over into more than five datablocks.

• Size of the data block.

• Increased I/Os for updates.

• Increased compares for inserts and FastLoad (more Central Processing Unit (CPU) and I/Os).

• Performance degradation in the Restore and Table Rebuild utilities.

Some AMPs have many more rows of a table than do other AMPs.

One or a few NUPI values have many more rows than all the other NUPI values.

• Poor CPU parallel efficiency on the AMPs during full table scans and bulk inserts.

• Maintenance on the lumps involves increased I/Os for updates, and increased compares for inserts (more I/Os).

Performance Management 123

Page 124: Teradata Database Performance Management

Chapter 9: Data ManagementIdentifying Uneven Data Distribution

Using Teradata Viewpoint

You can use Teradata Viewpoint to examine uneven data distribution.

Although even data distribution has many advantages in a parallel system, at times you must sacrifice level-data distribution to reap the other benefits of PIs, specifically of a PI in join operation.

Using Hash Functions

Use the following functions to identify uneven data distribution.

HASHAMP Example

If you suspect distribution problems (skewing) among AMPS, the following is a sample of what you might enter for a three-column PI:

SELECT HASHAMP (HASHBUCKET (HASHROW (col_x, col_y, col_z))), count (*)

FROM hash15 GROUP BY 1ORDER BY 2 desc;

HASHROW Example

If you suspect collisions in a row hash, the following is a sample of what you might enter for a three-column PI:

SELECT HASHROW (col_x, col_y, col_z), count (*)FROM hash15 GROUP BY 1ORDER BY 2 descHAVING count(*) > 10;

Impact of Uneven Data Distribution

Uneven data distribution results in:

• Poor CPU parallel efficiency on full table scans and bulk inserts

• Increased I/Os for updates and inserts of over-represented values

If you suspect uneven data distribution:

• Run the ResVproc macro to check the AMP parallel efficiency so that you can identify the AMP affecting that node.

Function Definition

HASHAMP AMP that owns the hash bucket

HASHBACKAMP Fallback AMP that owns the hash bucket

HASHBUCKET Grouping for the specific hash value

HASHROW 32 bits of row hash ID without the uniqueness field

124 Performance Management

Page 125: Teradata Database Performance Management

Chapter 9: Data ManagementIdentifying Uneven Data Distribution

• Check table distribution information at the AMP level by running an SQL query against DBC.TableSizeV (see “Identifying Uneven Data Distribution” on page 123 for a sample query). This query identifies the number of bytes/AMP vproc for a given table.

Check Periodically for Skewed Data

The parallel architecture of Teradata Database distributes rows across the AMPs using hash buckets. Each AMP has its own set of unique hash buckets. Skewed or lumpy data means that many rows are hashed to the same AMP because of their highly nonunique index(es), compared to the other AMPs on the system, and therefore the distribution is very uneven.

Although skewed data does not always cause problems, nor can it always be avoided, having skewed data may result in hot AMPs during access, and on a very busy system, can use a lot of resources.

Sample Scripts

Below is a set of useful scripts written to check for skewed data:

Note: Running a script that checks for skewed data is a good performance practice when new applications are being loaded on the system or when data changes in major ways.

/* *//* LUMPY – identified those tables that are not evenly distributed *//* Variance should ideally be less than 5%. Here we have *//* it set to 1000% which will usually indicate that some *//* or many vprocs do not have any data from the table at *//* all. You can use “RETLIMIT” to limit the number of *//* rows returned. */SEL (MAX(CurrentPerm) – MIN(CurrentPerm)) * 100/(NULLIF(MIN(currentperm),0))(NAMED variance)(FORMAT ‘zzzzz9.99%’),MAX(CurrentPerm)(TITLE ‘Max’)(FORMAT ‘zzz,zzz,zzz,999’),MIN(currentperm)(TITLE ‘Min’)(FORMAT ‘zzz,zzz,zzz,999’),TRIM(DatabaseName)||’.’||TableName(NAMED Tables)FROM DBC.TableSizeVGROUP BY DatabaseName, TableNameHAVING SUM(CurrentPerm)> 1000000AND variance> 1000WHERE DatabaseName NOT IN(‘CrashDumps’,’DBC’)ORDER BY Tables;/* *//* Once you have identified a target table, you can display *//* the detailed distribution with the following query. *//* */SELECT vproc, CurrentPerm FROM DBC.TableSizeVWHERE DatabaseName = ‘xxxx’AND TableName = ‘yyyy’ORDER BY 1;/* *//* The following table will list the row distribution by amp *//* for a given table. *//* */sel dt1.a (title’AMP’),dt1.b (title’Rows’),((dt1.b/dt2.x (float)) – 1.0) * 100(format’+++9%’,title’Deviation’)

Performance Management 125

Page 126: Teradata Database Performance Management

Chapter 9: Data ManagementParallel Efficiency

from(sel hashamp(hashbucket(hashrow(<index>))),count(*)from <databasename>.<tablename>group by 1)dt1 (a,b),(sel (count(*) / (hashamp()+1)(float))FROM <databasename>.<tablename>)dt2(x)order by 2 desc,1;/* *//* The following query will provide the distribution by amp *//* for a given index or column. *//* */sel hashamp(hashbucket(hashrow(index or column))),count(*)from database.tablegroup by 1order by 2 desc;/* *//* The following query will provide the number of collisions *//* for row hash. *//* */sel hashrow(index or column), count(*)from database.tablegroup by 1order by 1having count(*) > 10;/* *//* The following query will provide the number of amps and *//* the number of rows impacted by a query. *//* */LOCKING TABLE <table> FOR ACCESSSELECT COUNT(DT.ampNum) (TITLE ‘#AMPS’),SUM(DT.numRows) (TITLE ‘#ROWS’)FROM(SELECT HASHAMP(HASHBUCKET(HASHROW(<index>))),count(*)FROM <table>WHERE <selection criteria>GROUP BY 1)DT (ampNum, numRows);

Parallel Efficiency

If the system exhibits poor parallel efficiency and data is not skewed, you should look for signs of skewing during processing.

Join Processing and Aggregation

Join processing and aggregation both may involve row redistribution. An easy way to find out if rows are redistributed during an operation is to check for high BYNET read and write activity.

126 Performance Management

Page 127: Teradata Database Performance Management

Chapter 9: Data ManagementPrimary Index and Row Distribution

In join processing, poor parallel efficiency occurs when the join field is highly skewed. Since rows are redistributed to AMPs based on the hash value of the join column, a disproportionate number of rows may end up on one AMP or on a few AMPs.

For example, you perform a join on city code with a large number of instances of New York and Los Angeles. A large number of rows would be redistributed to two AMPs for the join. Those AMPs would show much higher CPU utilization during the operation than the other AMPs.

Referential Integrity

Skewed processing can also occur with RI when the referencing column has skewed demographics, for example, when the referenced column is city code.

Performance Impact

Both skewed data distribution and skewed processing can adversely affect node, as well as AMP, parallel efficiency because the CPU activity of a node is a direct reflection of the CPU activity of vprocs.

Primary Index and Row Distribution

The hash value of the PI for a row determines where the row is stored on the AMP. In a normal environment, hash values are evenly distributed across the nodes and across the AMPs within a node.

The less unique the values for the index, the less evenly the rows of that table are distributed across the AMPs. If a table has a NUPI with thousands of instances of a single value, the table can become skewed.

Effects of Skewed Data

The effects of a skewed table appear in several types of operations. For example:

• In full table scans, the AMPs with fewer rows of the target table must wait for the AMPs with disproportionately high numbers of rows to finish. Node CPU utilization reflects these differences because a node is only as fast as the slowest AMP of that node.

• In the case of bulk inserts to a skewed table, consider the extra burden placed on an AMP with a high number of multiple rows for the same NUPI value.

For example, assume you have a 5 million row table, with 5,000 rows having the same NUPI value. You are inserting 100,000 rows into that table, with 100 of those insert rows having the same NUPI value. The AMP holding the 5,000 rows with that NUPI value has to perform one half million duplicate row checks (5,000 * 100) for this NUPI. This operation results in poor parallel efficiency.

Performance Impact of a Primary Index

Keep in mind the following:

Performance Management 127

Page 128: Teradata Database Performance Management

Chapter 9: Data ManagementNo Primary Index Table

• The more unique the PI, the more unique the row hash value.

• The more unique the row hash value, the more even the data distribution across all the AMP vprocs. Even data distribution enhances parallel efficiency on full table scan operations.

• UPIs generate the most even distribution of table rows across all AMP vprocs.

• NUPIs generate even row distribution to the same degree that values for a column or columns are unique. Rows with the same row hash always go to the same AMP, whether they are from the same table or from different tables.

If a PI causes uneven data distribution, you may decide to change it. To determine the best PI for a table, factor in the:

• Extent of update activity

• Number of full table scans

• Join activity against the PI definition

• Frequency of PI as a selectivity column and, therefore, a potential access path

No Primary Index Table

By definition, a NoPI table has no PI and thus the hash value, as well as AMP ownership of a row, is random. This means rows can be stored in any AMP. Moreover, a NoPI table has no row ordering constraints and, therefore, rows can be appended to the end of the table as if it were a spool.

Data can be loaded into a NoPI table quickly using FastLoad or through SQL sessions via TPump Array INSERT. A NoPI table can then be used as a staging table from which data is applied to a target table through an INSERT . . . SELECT, an UPDATE FROM or a MERGE INTO statement.

Hash Bucket Expansion

Teradata Database supports either 65,536 or 1,048,576 hash buckets for a system. The larger number of buckets primarily benefits systems with thousands of AMPs, but there is no disadvantage to using the larger number of buckets on smaller systems.

The hash bucket is an array indexed by hash bucket number. Each entry of the array contains the number of the AMP that processes the rows in the corresponding hash bucket. The RowHash is a 32-bit result obtained by applying the hash function to the PI of the row.

• On systems with 65,536 hash buckets, the system uses 16 bits of the 32-bit RowHash to index into the hash map.

• On systems with 1,048,576 hash buckets, the system uses 20 bits of the 32-bit RowHash as the index.

128 Performance Management

Page 129: Teradata Database Performance Management

Chapter 9: Data ManagementHash Bucket Expansion

Two fields added to the DBS Control, only one of which a user can set, manage the number of hash buckets:

• CurHashBucketSize

An 8-bit unsigned integer that indicates the number of bits used to identify a hash bucket in the current system configuration. Users cannot change this field. The default value for sysinit is 20.

• NewHashBucketSize

An 8-bit unsigned integer that indicates the number of bits used to identify a hash bucket. Users can choose this value prior to starting a system reconfiguration. A value of 20 causes the reconfiguration to create the larger hash map, with additional redistribution of data during the reconfiguration.

New systems are delivered with the larger number of buckets, regardless of system size. This allows maximum flexibility for future system growth.

Performance Management 129

Page 130: Teradata Database Performance Management

Chapter 9: Data ManagementHash Bucket Expansion

130 Performance Management

Page 131: Teradata Database Performance Management

CHAPTER 10 Managing Space

This chapter discusses space management in Teradata Database.

Tracking Data Space Data

You can track data space data using Teradata Viewpoint to measure:

• Current permanent, spool, and temporary data usage by each database/user.

• Permanent space skew by each database/user and table.

• Permanent space usage trends by each database/user

• Current data usage compared to the allocated maximum and historical peak usage.

• Permanent space usage and availability by each vproc.

Running Out of Disk Space

Teradata Database does not run out of disk space until it allocates and fully utilizes all cylinders.

Low Cylinder Utilization

Performance degradations can occur, however, as soon as the system gets close to exhausting the free cylinder pool. This happens because the system performs MiniCylPacks on cylinders with low utilization in order to reclaim the unused disk space. Therefore, you should be aware if you are running out of space due to a preponderance of under-utilized cylinders.

Low utilization of cylinders can occur when:

• You FastLoad a table using a small FreeSpacePercent (FSP) and then insert additional data to the table that is greater than the FSP.

• You delete a significant percent of a table but have not yet run Ferret PACKDISK to reclaim the space.

Frequently Updated Tables

With frequently updated tables, the free space on the cylinder can become so fragmented that it cannot be used.

When this occurs, the system could allocate additional cylinders to the table. To avoid this problem, the system sometimes performs a cylinder defragmentation to make the free space on the cylinder usable again.

Performance Management 131

Page 132: Teradata Database Performance Management

Chapter 10: Managing SpaceRunning Out of Free Cylinders

Performance Degradation

While AutoCylPacks, MiniCylPacks and defragmentation help the system reclaim free disk space for further use, they incur a performance degradation. Properly size and tune the system to avoid this overhead.

Running Out of Free Cylinders

Teradata Database requires free cylinders to ensure that there is enough:

• Permanent space

• Temporary space

• Spool space

Ensuring Optimal Performance

To ensure optimal performance, Teradata Database uses both.

Automatic Processes for Managing Space

Teradata Database has three automatic processes to deal with space issues.

• Minicylinder pack (MiniCylPack)

• Automatic background cylinder packing (AutoCylPack)

• Defragmentation

Rather than free cylinders, defragmentation creates more space on the currently used cylinders, diminishing the need for empty cylinders.

Freeing Cylinders

AutoCylPack

AutoCylPack (automatic background cylinder packing) manages space on cylinders, finding cylinders with low or high utilization and returning them to their user-defined FSP (Free

Item Description

Contiguous sectors on a cylinder

datablocks are stored on adjacent sectors in a cylinder.

If a cylinder has 20 available sectors, but only 10 are contiguous, a 15-sector block must be stored on another cylinder.

Free cylinders Teradata Database performs better if permanent data is distributed across multiple cylinders. However, permanent data and spool data cannot share the same cylinder.

Therefore, a system must always have empty cylinders that can be used for spool space.

132 Performance Management

Page 133: Teradata Database Performance Management

Chapter 10: Managing SpaceFreeing Cylinders

Space Percent), that is, the percentage of storage space within a cylinder that AutoCylPack leaves free of data to allow for future table growth.

There is a system wide default for AutoCylPack. It can be overridden on a table-by-table basis by specifying the FSP clause in a CREATE TABLE or ALTER TABLE statement.

It is important for Teradata Database file system to maintain enough free cylinders and to manage the space on the cylinders.

Although AutoCylPack runs as a background task issuing I/Os, you can adjust its level of impact. For the DBS Control fields that support AutoCylpack, see Utilities.

Performance

AutoCylPack helps reduce:

• The chances that MiniCylPacks run. MiniCylPacks are strictly concerned with reclaiming whole cylinders.

• The need for you to perform regular Ferret PACKDISK operations.

PACKDISK can be performed on a table, a range of tables or an entire system, but it affects performance of the foreground tasks and thus system performance.

If cylinders have higher utilization:

• System performance improves because there is a higher datablock to cylinder index ratio and more effective use of Cylinder Read. There will be less unoccupied sectors read in.

• A table occupies less cylinders. This leaves more cylinders available for other uses.

MiniCylPack

A MiniCylPack moves datablocks in logical sequence from cylinder to cylinder, stopping when the required number of free cylinders is available. A single MiniCylPack may effect two to 20 cylinders on an AMP.

The process continues until one cylinder is completely emptied. The master index begins the next required MiniCylPack at the location that the last MiniCylPack completed.

Teradata Database file system will start to MiniCylPack when the number of free cylinders drops to the value set by MiniCylPackLowCylProd. The default is 10.

The File Information Block (FIB) keeps a history of the last five cylinders allocated to avoid MiniCylPacks on them.

Note: Spool files are never cylinder packed.

Use the DBS Control (see “MiniCylPackLowCylProd” in Utilities) to specify the free cylinder threshold that causes a MiniCylPack. If the system needs a free cylinder and none are available, a MiniCylPack occurs spontaneously.

Performance Management 133

Page 134: Teradata Database Performance Management

Chapter 10: Managing SpaceFreeing Cylinders

Migrating Datablocks

1 If space can be made available either by migrating blocks forward to the next cylinder or backwards to the previous cylinder, choose the direction that would require moving the fewest blocks.

If the number of blocks is the same, choose the direction of the cylinder with the most number of free sectors.

2 If Step 1 fails to free the desired sectors, try migrating blocks in the other direction.

3 If space can be made available only by allocating a new cylinder, allocate a new cylinder. The preference is to add a new cylinder:

• Before the current cylinder for permanent tables.

• After the current cylinder for spool tables and while performing FastLoads.

When migrating either forward or backward, the number of blocks may vary because the system considers different blocks for migration.

Because of the restriction on key ranges within a cylinder, the system, when migrating backward, must move tables and rows with the lowest keys. When migrating forward, the system must move tables and rows with the largest keys.

The system follows special rules for migrating blocks between cylinders to cover special uses, such as sort and restore. There are minor variations of these special rules, such as migrating more datablocks than required in anticipation of addition needs, and looking for subtable breaks on a cylinder to decide how many datablocks to attempt to migrate.

Performance

Although cylinder packing itself has a small impact on performance, it often coincides with other performance impacting conditions or events. When Teradata Database file system performs a MiniCylPack, the operation frees exactly one cylinder.

The cylinder packing operation itself runs at the priority of the user whose job needed the free cylinder. The cylinder packing operation is the last step the system can take to recover space in order to perform a write operation, and it is a signal that the system is out of space.

Needing to pack cylinders may be a temporary condition in that a query, or group of queries, with very high spool usage consumes all available free space. This is not a desirable condition.

If space is a problem,

• Enable AutoCylPack if you have not done so and specify an FSP of 0% for read-only tables using the CREATE TABLE or ALTER TABLE statement

• Run the Ferret PACKDISK command.

Error Codes

MiniCylPacks are a natural occurrence and serve as a warning that the system may be running short on space. Tightly packed data can encourage future cylinder allocation, which in turn triggers more MiniCylPacks.

The system logs MiniCylPacks in the Software_Event_LogV with the following error codes.

134 Performance Management

Page 135: Teradata Database Performance Management

Chapter 10: Managing SpaceFreeing Cylinders

Frequent 340514200 or 340514300 messages indicate that the configuration is under stress, often from large spool file requirements on all AMPs. MiniCylPacks tend to occur across all AMPs until spool requirements subside. This impacts all running requests.

If table data is skewed, you might see MiniCylPacks even if Teradata Database has not used up most of the disk space.

Defragmentation

As random updates occur over time, empty gaps become scattered between datablocks on the cylinder. This is known as fragmentation. When a cylinder is fragmented, total free space may be sufficient for future updates, but the cylinder may not contain enough contiguous sectors to store a particular data block. This can cause cylinder migrates and even new cylinder allocations when new cylinders may be in short supply. To alleviate this problem, the file system defragments a fragmented cylinder, which collects all free space into contiguous sectors.

Use the DBS Control (see “DefragLowCylProd” in Utilities) to specify the free cylinder threshold that causes defragmentation. When the system reaches this free cylinder threshold, it defragments cylinders as a background task.

Code Description

340514100 Summary of MiniCylPacks done at threshold set via the DBS Control.

340514200 A MiniCylPack occurred during processing and a task was waiting for it to complete.

340514300 The system could not free cylinders using MiniCylPack. The MiniCylPack failed. This means that the system is either getting too full or that the free cylinder threshold is set unreasonably high. Investigate this error code immediately.

KY01A018

FreeCylinder

Data Block

Free Space

New DataBlock

Performance Management 135

Page 136: Teradata Database Performance Management

Chapter 10: Managing SpaceFreeing Space on Cylinders

To defragment a cylinder, the file system allocates a new cylinder and copies data from the fragmented cylinder to the new one. The old cylinder eventually becomes free, resulting in a defragmented cylinder with no change in the number of available free cylinders.

Since the copy is done in order, this results in the new cylinder having a single, free-sector entry that describes all the free sectors on the cylinder. New sector requests on this cylinder are completed successfully, whereas before they may have failed.

Ferret Defragmentation

The Ferret Defrag command is a defragmentation command. It can be used to scan the entire set of user cylinders and then to defragment the qualified cylinders.

Running Ferret Defrag command, however on a busy customer system affects the performance of the foreground tasks and thus system performance. Moreover, Ferret Defrag has no way to know if there are enough free cylinders already available in the system, so that no more defragmentation is required.

Freeing Space on Cylinders

In addition to MiniCylPack and AutoCylPack, you can use the following to free space on cylinders or to make more cylinders available to the system:

• FreeSpacePercent (FSP) (see “PACKDISK and FreeSpacePercent” on page 140)

• PACKDISK (see Utilities)

• Cylinders to Save for Perm (see “Cylinders Saved for PERM” in Utilities)

• Temporary space limits on users/profiles.

• Spool space limits on user profile definitions:

• Set larger spool space limits.

• Minimize the number for individual users to limit runaway query spool space.

Both of the above will not necessarily stop space management routines such as MiniCylPack from running, but it could help manage the area.

• Archival and deletion of aged data

KY01A019

ContiguousFree Space

FreeCylinder

New DataBlock

Data Blocks

136 Performance Management

Page 137: Teradata Database Performance Management

Chapter 10: Managing SpaceFreeSpacePercent

Planned deletion of obsolete rows facilitates space availability. Depending on the nature of your data, you may want to archive before deleting. For more information, see Teradata Archive/Recovery Utility Reference.

• Appropriate data compression (see “Data Compression” on page 141)

• Additional disk space (see “Adding Disk Space” on page 146)

• Appropriate data block sizing:

• Maximum block size allocation

• Minimum block size allocation

• Journal data block size allocation

FreeSpacePercent

FreeSpacePercent (FSP) is a system-wide parameter. Use the CREATE TABLE statement to define the free space percent for a specific table. FSP does not override a value you specify via a CREATE or ALTER TABLE request.

MiniCylPack operations attempt to honor the FSP specified in a CREATE TABLE and ALTER TABLE statements or, if no FSP was specified, the system default FSP. If MiniCylPack cannot reclaim space while honoring those values, it will keep trying to use a degraded (smaller FSP) value until space can be reclaimed. This continues until MiniCylPack attempts to use an FSP value of 0 for all cylinders.

In some situations, Teradata Database runs out of free cylinders even though over 20% of the permanent disk storage space is available. This is due to:

• A higher FSP setting than necessary, which causes the system to allocate unrequired space

• A lower FSP setting than necessary, which causes the system to allocate new cylinders excessively

• Low storage density (utilization) on large tables due to cylinder splits

Determining a Value for FSP

Use the data in the following table to determine a value for FSP.

IF the majority of tables are… THEN…

read-only set the default system-wide FSP value to 0.

You can override the FSP for the remaining modifiable tables on an individual table basis with the ALTER TABLE statement.

Performance Management 137

Page 138: Teradata Database Performance Management

Chapter 10: Managing SpaceFreeSpacePercent

Because the system dynamically allocates free cylinder space for storage of inserted or updated data rows, leaving space for this during the initial load allows a table to expand with less need for cylinder splits and migrates. The system uses free space for inserted or updated rows. However, if you do not expect table expansion, that is, the majority of tables are read-only, use the lowest value (0%) for FSP.

When the system default FSP is zero, use the information in the following table to minimize problems.

If you set FSP to a value other than 0, tables are forced to occupy more cylinders than necessary. The extra space is not reclaimed until either you insert rows into the table, use the Ferret utility to initiate PACKDISK on a table, or until MiniCylPack is performed due to a lack of free cylinders.

When the system default FSP is greater than 0, use the information in the following table to minimize problems.

NOT read-only the FSP value depends on the percentage of increase in table size due to the added rows.

Thus, set the FreeSpacePercent parameter to a value that reflects the net growth rate of the data tables (inserts minus deletes). Common settings are5 to 15%. A value of 0% would be appropriate for tables that are not expected to grow after initially being loaded.

For example, if the system keeps a history for 13 weeks, and adds data daily before purging a trailing 14th week, use an FSP of at least 1/13 (8%).

To accommodate minor data skews and any increase in weekly volume, you can add an extra 5% (for a total of 13%) FSP.

IF you… THEN…

use read-only tables

change nothing.

At load time, or PACKDISK time, the system stores tables at maximum density.

add data via BTEQ or a CLI program

set the FREESPACE value on the CREATE TABLE statement to an appropriate value before loading the table.

If the table is loaded, use the ALTER TABLE statement to change FREESPACE to an appropriate value before running PACKDISK.

IF the majority of tables are… THEN…

138 Performance Management

Page 139: Teradata Database Performance Management

Chapter 10: Managing SpaceFreeSpacePercent

Operations Honoring the FSP

When adding rows to a table, the file system can choose either to use 100% of the storage cylinders available or to honor the FSP. The following operations honor FSP:

• FastLoad

• MultiLoad into empty tables

• Restore

• Table Rebuild

• Reconfiguration

• SQL to add fallback

• SQL to create an SI

Operations that Disregard FSP

The following operations disregard FSP:

• SQL inserts and updates

• Tpump

• MultiLoad inserts or updates to populated tables

If your system is tightly packed and you want to apply or reapply FSP, you can:

• Specify the IMMEDIATE clause with the ALTER TABLE statement on your largest tables.

• DROP your largest tables and FastLoad them.

• DUMP your largest tables and RESTORE them.

• In Ferret, set the SCOPE to TABLE and PACKDISK FSP = xxxx

In each case, table re-creation uses utilities that honor the FSP value and fills cylinders to the FSP in effect. These options are only viable if you have the time window in which to accomplish the processing. Consider the following guidelines:

• If READ ONLY data, pack tightly (0%).

• For INSERTs:

• Estimate growth percentage to get FSP. Add 5% for skewing.

• After initial growth, FSP has no impact.

• Reapply FSP with DROP/FASTLOAD, DUMP/RESTORE or PACKDISK operations.

IF you… THEN…

use read-only tables set FREESPACE on the CREATE TABLE statement to 0 before loading the table.

If the table is loaded, use the ALTER TABLE statement to change FREESPACE to 0 before running PACKDISK.

add data via BTEQ or a CLI program

change nothing.

The system adds rows at maximum density.

Performance Management 139

Page 140: Teradata Database Performance Management

Chapter 10: Managing SpacePACKDISK and FreeSpacePercent

• Experiment with different FSP values before adding nodes or drives.

PACKDISK and FreeSpacePercent

When Teradata Database runs out of free cylinders, you can run Ferret PACKDISK, an expensive overhead operation, to compact data to free up more cylinders.

To reduce the frequency of PACKDISK operations:

• When FastLoading tables to which rows will be subsequently added, set FSP to 5-20% to provide enough free space to add rows.

• For historical data, where you are adding and deleting data, provide enough free space to add rows.

For example, you add up to 31 days before deleting on a table with six months history.

• Add one month to six months: 1/7 = 14.3%

• Safety - plan on 1.5 months, 1.5 / 7.5 = 20%

Set Free Space Percent to 20%.

• For historical data and fragmented cylinders:

• For large tables, either set FSP to 20 - 35%, or set MaxBlockSize to smaller size (16 KB, for example).

• Translate free space to the number of datablocks. Plan on at least 6-12 blocks worth of free space.

• Specify the IMMEDIATE clause with the ALTER TABLE statement.

The table header contains the FSP for each table. If you change the default FSP, the system uses the new default the next time you modify the table. FSP has no effect on block size.

Running Other Utilities with PACKDISK

If you run PACKDISK frequently, use the following tools, two of which are utilities, to determine the amount of free space:

• DBC.DiskSpaceV

• SHOWSPACE, a Ferret command, shows you the percent of free space per cylinder.

If this figure is low, it will impact performance by performing dynamic cylinder packs when the system needs contiguous space.

• SHOWFSP, a Ferret command like SHOWSPACE, is useful in discovering specific tables that need packing.

SHOWFSP shows the number of cylinders that can be freed up for individual tables by specifying a desired free space percent.

SHOWFSP is useful in discovering which tables would free the most cylinders if PACKDISK were run on them. Certain tables exist that can free up a large percentage of cylinders.

140 Performance Management

Page 141: Teradata Database Performance Management

Chapter 10: Managing SpaceData Compression

Cylinder Splits

A FreeSpacePercent value of 0% indicates that no empty space is reserved on Teradata Database disk cylinders for future growth when a new table is loaded. That is, the current setting causes each data cylinder to be packed 100% full when a new table is loaded.

Unless data is deleted from the table prior to subsequent row inserts, this situation will guarantee that a cylinder split will be necessary the first time an additional row is to be inserted into the table (following the initial load). Cylinder splits consume system I/O overhead and result in poor utilization of data cylinders in most circumstances.

PACKDISK, FSP, and Cylinder Allocations

Packing data for tables being modified too tightly can result in too many cylinder allocations, thus depleting the number of free cylinders. For example, if the data on a table occupying 10 cylinders is packed to 0% free space, inserting one row on each cylinder (for a total of only 10 rows) may require Teradata Database to allocate up to 10 additional new cylinders.

If, however, a PACKDISK was run on the table with an FSP of 10%, the original table would occupy 11 cylinders. When the same 10 rows are inserted into this table, there will most likely be space available on the existing cylinders and the table would remain, consuming only 11 cylinders. This will leave the other 9 cylinders that were previously used free and thus available for future new cylinder requests. Another way to accomplish the same goal would be to update the table first and then perform the PACKDISK operation to free up any extra cylinders that were not really required.

Running the PACKDISK command in this example saved 9 free cylinders, but required manual intervention. Another way to have accomplished this would have been to specify an FSP of 10% in the CREATE TABLE or ALTER TABLE statement and to have had AutoCylPack adjust the table as part of its background activity. If new rows are added to the table in this example on a regular basis, then the AutoCylPack solution would be preferable. If, however, new rows are only added rarely, then it would be better to leave the table packed with 0% free space for most of its existence and just run a PACKDISK when required.

Data Compression

Implementing data compression on a grand scale will help most operations by making row sizes smaller, allowing more rows per block in a single I/O. This means less I/O and fewer blocks for the table. Many tests have shown great improvements gained through the more-rows-per-block concept, thus reducing I/Os in the full table scan processes.

The presence of compression has not been shown to degrade query response time since the uncompressed values are held in the table header in memory and can be accessed very quickly. However, with more data able to be read per data block due to the smaller row sizes, it is possible for some queries to become more CPU-intensive when using compression extensively.

Performance Management 141

Page 142: Teradata Database Performance Management

Chapter 10: Managing SpaceData Compression

Multi-Value Compression

Multi-value Compression (MVC) is a dictionary-based compression scheme at the granularity of column in a row. The best candidates for compression are the most frequently occurring values in each column. MVC is a good compression scheme when there are lots of repeating values for a column.

MVC provides Teradata Database with the capacity to support multiple value compression for fixed-width columns. In addition MVC supports variable-length column data types, including VARCHAR, VARBYTE, and VARGRAPHIC. MVC can be applied to larger columns of up to 510 character. MVC on variable length columns allows NULLs to be compressed.

When you specify a values or values, the system suppresses any data matching a compress value from a row. Up to 255 distinct values (plus NULL) may be compressed per fixed-width column. This saves disk space.

Smaller physical row size results in less datablocks and fewer I/Os and improved overall performance.

For information on implementing MVC, see Database Administration.

Performance Impact

MVC improves performance as follows:

• Reduces the I/O required for scanning tables when the tables have compressible values in their columns.

• Reduces disk space because rows are smaller.

• Permits joins to look up the tables to be eliminated.

• Improves data loading because more rows may fit into one data block after compression is applied.

Performance Considerations

MVC enhances the system cost and performance of high data volume applications, like call record detail and click-stream data, and provides significant performance improvement for general ad hoc workloads and full table scan applications.

MVC improves performance depending upon amount of compression achieved by the revised DDL. There appears to be a small cost in terms of processing the compressed data. Select and delete operations show a proportionate improvement in all cases. Inserts and updates show mixed results. The load utilities benefit from the compressed values.

Tables with large numbers of rows and fields with limited numbers of unique values are very good candidates for compression. With very few exceptions, the CPU cost overhead for compression processing is minimal. The reduction in the table size depends upon the number of fields compressed and the frequency of the compressed values in the table column. The reduced table size directly translates into improved performance.

142 Performance Management

Page 143: Teradata Database Performance Management

Chapter 10: Managing SpaceManaging Data Block Size

Algorithmic Compression

Algorithmic Compression (ALC) enables users to define their own compression and decompression algorithms, as well as providing two fast path compression and decompression algorithms for a BYTE, CHARACTER, GRAPHIC, VARBYTE, VARCHAR, or VARGRAPHIC column, regardless of whether value compression is defined for the column.

The algorithms are implemented with UDFs at the column level. The algorithms are defined as regular scalar UDFs and specified in the column definition during a CREATE TABLE or ALTER TABLE statement. Teradata Database then invokes the algorithms internally to compress or decompress the column data when the data is moved into tables or when data is retrieved from tables.

Users can have ALC, as well as MVC, on the same column. In such a case, ALC will compress only values that are not specified to be compressed by MVC.

Performance

Algorithmic compression helps minimize storage for data and thus trades off CPU performance in doing so. Compression itself tends to use more CPU time, but I/O takes also CPU time.

Compression tends to reduce I/O. One could conceivably use less CPU with a fast extreme compression algorithm. More likely, however, is a case where CPU time increases, but the DBS is I/O bound. In such a case, response can improve even though it takes more CPU.

Because ALC algorithms are defined as UDFs, their performance depends on UDF performance. If the compression or decompression UDFs contains bugs, then data corruption may result. Corruption is limited to the field that was compressed; the rest of the row will be unaffected.Any extra cost of compression and decompression depends on the nature of the algorithm chosen.

Choosing a Compression Scheme

In some cases, such as when column values are mostly unique, ALC can provide better compression results than MVC. Because MVC can coexist with ALC, users have the flexibility of an IF-ELSE option: either compressing values from a dictionary or using a custom compression algorithm.

Managing Data Block Size

Maximum Data Block Size

You can set maximum block size two ways.

Performance Management 143

Page 144: Teradata Database Performance Management

Chapter 10: Managing SpaceManaging Data Block Size

Larger block sizes enhance full table scan operations by selecting more rows in a single I/O. The goal for DSS is to minimize the number of I/O operations, thus reducing the overall time spent on transactions.

Smaller block sizes are best used on transaction-oriented systems to minimize overhead by only retrieving what is needed.

For more information on data block size, see “PermDBSize” in Utilities.

Rows cannot cross block boundaries. If an INSERT or UPDATE causes a block to expand beyond the defined maximum block size, the system splits the block into two or three blocks depending on the following.

Operation Comments

Set PermDBSize via the DBS Control

When you set maximum block size at the system level, a table utilizes a value only until a new value is set system-wide. This control helps organize the table for better performance for either Decision Support System (DSS) or Online Transaction Processing (OLTP) operations.

Use the CREATE or ALTER TABLE command

When you set maximum block size at the table level, this value remains the same until you execute an ALTER TABLE command to change it.

IF… AND… THEN…

the new or changed row belongs in the beginning or end of a block

a block containing only that row is larger than the defined maximum size block for the table

the row is placed in a block by itself.

the new or changed row belongs in the middle of a block

a block containing only that row would be larger than the defined maximum size for the table

then a three-way block split is performed.

The existing block is split into two blocks at the point where the row being modified belongs and a new block is created between them containing only the modified row.

the existing block size can be changed to accommodate modification and still not be larger than the defined maximum size for the table

case is empty a single new block with the existing rows and the new or changed row is created.

144 Performance Management

Page 145: Teradata Database Performance Management

Chapter 10: Managing SpaceManaging Data Block Size

Additional special rules exist that take precedence over the above. For example:

• Rows of different subtables never coexist in datablocks.

• Spool tables are almost always created as whole blocks all at once with many rows in them with the maximum size defined for spool (as long as there are enough rows).

Minimum Data Block Allocation Unit

Set minimum block allocation via the PermDBAllocUnit field (see “PermDBAllocUnit” in Utilities).

Although this field may cause more unused space in blocks in the system, data can be maintained longer within a block without getting a larger block or doing a block split. This value also determines additional block growth (that is, a block must ultimately be a multiple of this value).

Datablock Merge

The Datablock Merge operation reduces the number of very small datablocks in table(s) by combining them with logically adjacent datablocks. It enables the Teradata file system to merge dynamically up to 8 small datablocks residing on a single cylinder into a single large data block for each block containing a row that will be modified (or inserted).

The Merge Datablock operation:

• Runs automatically.

• Does not require manual investigation of block size histograms.

• Does not require any specific AMP level locking.

• Searches continuously for small blocks to merge together, even if some of those blocks do not require updates.

• Only affects the performance of workloads that modify data. Read-only workloads, insert operations into new subtables, and prime key modification requests are not affected.

Moreover, Merge Datablock is only applicable to permanent and permanent journal tables, not to global temporary or volatile tables.

The Datablock Merge operation enhances the performance of future query and modification operations by automatically merging smaller datablocks into a single large data block. Having fewer blocks reduces the number of I/O operations required to read and modify a large set of rows from disk.

the existing block size cannot be changed to accommodate modification and still not be larger than the define maximum size for the table

case is empty the modification is applied and then the block is split into as many parts as required (and as few parts as possible) so that each part is not larger than the defined maximum for the table.

IF… AND… THEN…

Performance Management 145

Page 146: Teradata Database Performance Management

Chapter 10: Managing SpaceAdding Disk Space

For the DBS Control fields that support merge datablock, including altering its behavior as well as disabling it, see Utilities.

For the option, MergeBlockRatio, that defines the size of the resulting block when multiple

existing blocks are being merged, see “CREATE TABLE” and “ALTER TABLE” in SQL Data Definition Language Syntax and Examples.

Journal Data Block Size

Journal data block sizes may also affect the I/O of a system. Set the journal data block size via the JournalDBSize field (see “JournalDBSize” in Utilities).

A larger journal size may result in less I/O or cause wasted block space on the database. Size your journal datablocks as accurately as possible.

Adding Disk Space

As your business grows, so does your database. Depending on the amount of historical data you wish to maintain online, your database may need to grow even if your business is not growing as quickly. With Teradata Database, you can add storage to existing nodes, or add storage as well as nodes.

Consider the following:

• Current performance of the existing nodes

• Existing bottlenecks

• Amount of space managed by an AMP

• Number of AMPs on the existing nodes

Managing Spool Space

Managing spool space allocation for users can be a method to control both space utilization and potentially bad (that is, unoptimized) queries.

Spool Space and Perm Space

Spool space is allocated to a user. If several users are active under the same logon and one query is executed that exceeds the limit of the spool space allocated, all active queries for that user that require spool will likewise be denied additional spool and will be aborted.

If space is an issue, it is better to run out of spool space than to run out of permanent space. A user requesting additional permanent space will do so to execute queries that modify tables (inserts or updates, for example). Additional spool requests are almost always done to support a SELECT. Selects are not subject to rollback.

To configure this, see “Cylinders Saved for PERM” in Utilities.

146 Performance Management

Page 147: Teradata Database Performance Management

Chapter 10: Managing SpaceFile System Fault Isolation

Note: Permanent and spool limits per user span the entire system. When the configuration is expanded, the limit is spread across all AMPs. If the system size in AMPs has increased by 50%, then both permanent and spool space limits are spread 50% thinner across all AMPs. This may require that the spool space limit for some users, and possibly permanent space limit, be raised if the data in their tables is badly skewed (that is, lumpy).

Spool Space Accounting

Spool space for users is updated in the DatabaseSpace table. However, given the possibility of phantom spool, running UPDATESPACE is necessary to clear spool space for users that have logged off, whether they were aborted users or not. After UPDATESPACE has been run, spool space values in DBC.DatabaseSpace reflects actual spool usage.

Phantom spool cases are those in which the DBC.DatabaseSpace table indicates that there is spool, although no spool exists on the disk. Phantom spool cases are not the same as “left-over spool” cases. Left-over spool cases are those in which spool is actually created and exists on the disk, although a request completed its execution.

Increasing Spool Space

To increase spool space and increase performance:

• Create a spool reserve (see Database Administration)

• Compress recurring values

• Eliminate unnecessary fallback

• Eliminate unnecessary indexes

• As a last resort, add hardware

Using Spool Space as a Trip Wire

Lowering spool space may be a way to catch resource-intensive queries that will never finish or that will run the entire system out of free space if the user is allocated a very high spool space limit.

In the interest of system performance, do not allocate high spool space limits to all users and, in general, be very conservative in setting spool space limits.

File System Fault Isolation

Teradata Database can isolate some file system errors to a specific data or index subtable, or to a contiguous range of rows (“region”) in a data or index subtable. In these cases, Teradata Database marks only the affected subtable or region down. This improves system performance and availability by allowing transactions that do not require access to the down subtable or rows to proceed, without causing a database crash that would require a system restart.

However, if several regions in a subtable are marked down, it could indicate a fundamental problem with the subtable itself. Therefore, when a threshold number of down regions is

Performance Management 147

Page 148: Teradata Database Performance Management

Chapter 10: Managing SpaceFile System Fault Isolation

exceeded, the entire subtable is marked down on all AMPs, making it unavailable to most SQL queries. This threshold can be adjusted by means of the DBS Control utility.

For more information, see MaxDownRegions in the DBS Control chapter of Utilities.

148 Performance Management

Page 149: Teradata Database Performance Management

CHAPTER 11 Using, Adjusting, and MonitoringMemory

This chapter discusses using, adjusting, and monitoring memory to manage system performance.

Using Memory Effectively

To determine if your system is using memory effectively:

1 Start with a value for FSG Cache percent. See “Reserving Free Memory” on page 151.

2 Adjust the value based on available free memory. See “Reserving Free Memory” on page 151 and “Free Memory” on page 152.

You should also consider adjusting the values of the following fields of the DBS Control Record:

• DBSCacheThr

• RedistBufSize

• SyncScanCacheThr

• PPICasheThrP

See Utilities for information on shared and free memory and on how to reserve, increase, and monitor free memory.

Shared Memory Example

The following figure illustrates how FSG Cache is calculated for a Teradata Database 64-bit Linux system with 16 GB shared memory. This is an example only.

Performance Management 149

Page 150: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryShared Memory Example

When Linux boots, PDE:

• Determines how much memory the OS is already using.

In the 64-bit example, memory is shown to be 500 MB. This number may vary depending on the configuration of your 64-bit system.

• Leaves to the OS an amount of memory equal to the number of vprocs times the memory allocated to each vproc.

In the 64-bit example, this amount is 90 MB for each of the 28 vprocs.

Note: In the figure above, vprocs include AMPs, PEs, the Gateway, and the Node vproc.

• Calculates the FSG Cache by (1) subtracting the OS-managed memory from total memory size and then (2) applying a FSG Cache% to that amount.

Note: To calculate the FSG Cache for each AMP, divide the FSG Cache by the number of AMPs.

In the 64-bit example, the amount of OS-managed memory is 3020 MB and the FSG Cache, if the FSG Cache% is 100%, would be 13364 MB.

If the FSG Cache% were 65%, then the FSG Cache would be 13364 MB minus 8687 MB. This number is not shown in the figure.

If you intend to run additional applications (with memory requirements unknown to Teradata Database software), reduce the FSG Cache% to leave memory available for these applications.

1097A007

Linux - 500 MB

28 Vprocs @90 MB each

FSGCache

13364 MB65%

(8687 MB)

100%

3020 MBOS

Managed

FSGManaged

FSGCache%

Total Memory Size 16 GB (16384 MB)

150 Performance Management

Page 151: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryMonitoring Memory

Note: FSG Cache is managed in multiples of 128 KB. Thus, if the FSG Cache for one AMP were 260 KB, 4 KB would not be used.

Memory Size

The appropriate amount of memory in each system running Teradata Database depends upon the applications running. You can use benchmarks to determine the appropriate memory size for a specific system.

Reserving Free Memory

For example, to reserve 20% of the FSG Cache for applications over and above the 96 (for 64-bit systems) MB/vproc, go to the DBS screen and set FSG Cache percent to 80. The system assigns 80% of the FSG Cache to FSG and leaves the remaining 20% for other applications.

For information on ctl, see Utilities.

For information on free memory, see “Free Memory” on page 152.

For information on FSG Cache, see “FSG Cache” on page 154.

Monitoring Memory

Use the ResUsage tables to obtain records of free memory usage and FSG Cache.

To Reserve a Percentage of Shared Memory For the Following Applications... Use the Following Utility...

Linux or Windows ctl

Memory Type Managed by Monitor With Comments

Free Memory Linux tools sar (Linux) or ResUsage See Chapter 4: “Collecting Resource Usage Data” for more information on using ResUsage to monitor free memory.

Windows tools sar, ResUsage, Task Manager

FSG Cache Teradata Database file system

ResUsageSpma

ResUsageSvpr

See Chapter 4: “Collecting Resource Usage Data” for more information on ResUsage macros.

Performance Management 151

Page 152: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryFree Memory

Free Memory

Free memory is used by:

• Administrative programs, such as program text and data, message buffers, kernel resources, and other applications such as FastLoad that require memory use.

• Vprocs for non-file system activity, such as:

ResUsage and Available Free Memory

The ResNode macro displays limited information on free memory. The ResNode report includes the Free Mem% column, which is the percentage of unused memory.

Adjusting for Low Available Free Memory

When the amount of available free memory dips too far below 100 MB (25,000 pages), some sites have experienced issues. 40 MB of free memory is considered the low-end of acceptable. This is usually avoided if you configure your AMPs to have at least 135 MB of OS-managed memory per AMP for 64-bit systems respectively by adjusting the FSG Cache% down from 100%. You can adjust the amount of available free memory by performing the following:

• Use ctl (on Linux or Windows) to adjust the FSG Cache percent to make more memory available to free memory. If the system takes too much memory from FSG Cache, and the OS does not use that memory, the free memory is wasted.

• If available free memory goes below 100 MB during heavy periods of redistribution (as explained later in this section), lower the value of the RedistBufSize field in the DBS Control Record (see “RedistBufSize” in Utilities).

Assured Minimum Non-FSG Cache Size

Teradata Database supports the following configuration guidelines for minimum non-FSG Cache size per AMP.

• 135 MB per AMP when all nodes are up.

• 112 MP per AMP when 1 node is down in a clique.

Activity Description

AWT Pieces of AMP logic used for specific AMP tasks

Parser tasks Pieces of PE logic responsible for parsing SQL

Dispatcher tasks Pieces of PE logic responsible for dispatching work

Scratch segments Temporary work space

Messages Communication between vprocs

Dictionary cache Dictionary steps for parsing

Request cache Temporary space used when executing steps

152 Performance Management

Page 153: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryFree Memory

• 90 MB per AMP when the maximum number of nodes allowed down are down in a clique.

The above configuration guidelines help avoid performance issues with respect to memory swaps and paging, memory depletion, and CPU starvation when memory is stressed.

Performance Management Recommendations

Internal benchmarking and tactical experience in the field indicates that most sites require more free memory than the default calculated on “Shared Memory Example” on page 149. Teradata recommends that you provide additional memory per AMP to free memory by setting the FSG Cache percent to a value less than 100%.

Use the following calculation:

FSG Cache percent = (FSG Cache - 39 MB * # AMPs) / FSG Cache

For large configurations, consider one or more of the following options to resolve I/O bottlenecks or excessive memory swapping:

• Consider using aggregate join indexes to reduce aggregate calculations during query processing.

• Set RedistBufSize to an incremental value lower; for example, from 4 KB to 3 KB

• Set FSG Cache percent to less than 100% (the percent specified depends on total memory size). See “RedistBufSize” in Utilities.

• Consider modifying the application to reduce row redistribution.

• Ask your support representative to reduce the internal redistribution buffer size (for example, to 16 KB).

Note: This is an internal tuning parameter, not the user-tunable RedistBufSize.

You may need to perform further tuning depending on the load from applications and Teradata Database utility programs.

Potential Problems

A possible problem is when an application on a large configuration generates many messages over the BYNET with concurrent row redistributions involving all nodes (see the subsections below).

The following are not a problem:

• Row duplications

• Merging of answer sets

Memory Size (GB) for 64-bit Systems

Memory for Baseboard Drivers & 10 AMPs/2PE Vprocs FSG Cache (MB)

Less 58 MB per AMP Vprocs

FSG Cache Percent

6.0 1854 4290 3822 89%

8.0 1914 6278 5810 92%

Performance Management 153

Page 154: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryFSG Cache

Query Row Redistribution Memory Requirement

To avoid the overhead of sending lots of little messages across the BYNET, buffers are used to batch up individual rows during the row redistribution process. Both load utilities and queries involve such redistribution, but their approach to outbound buffering is different.

Row redistribution for query processing uses separate single buffers per AMP for each node in the system. This means that the amount of memory required for redistribution in a node grows as the system grows.

The DBC Control Record field, “RedistBufSize,” controls the size of redistribution buffers.

• Default query redistribution buffer size = 32 KB per target node

• Total memory for one sending AMP = 32 KB * number of nodes in system

• For eight AMPs per node, total memory required per node = 8 * 32 KB * number of nodes in system

Redistribution Processing

The following example provides the calculations for a configuration of 8 nodes at eight AMPs per node. (The system only reserves 32MB per AMP).

• Single node requirement (single user) = 32 KB * 8 = 256 KB

• Multi-user (for example, 20 concurrent users) = 20 * 256 KB = 5 MB (not a special problem)

The following example provides the calculations for a configuration of 96 nodes at 8 AMPs per node:

• Single node requirement (single user) = 32 KB * 96 = 3072 KB (3 MB)

• Multi-user (20 concurrent users) = 20 * 3072 KB = 64 MB (far exceeding 32 MB per AMP)

Symptoms of high-volume redistribution processing include:

• Excessive memory paging/swapping

• Possible I/O bottleneck on BYNET I/O

FSG Cache

Teradata Database file system manages FSG Cache, which is used by:

• AMPs on the node

• Backup activity for AMPs on other nodes

Teradata Database file system uses FSG Cache for file system segments such as:

• Permanent datablocks (includes fallback data and SIs)

• Cylinder Indexes for permanent datablocks

• Cylinder statistics for Cylinder Read

• Spool datablocks and Cylinder Indexes for spool

154 Performance Management

Page 155: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryFSG Cache

• WAL space, including Transient Journal (TJ) rows and WAL REDO records

• Recovery journal rows

Space in FSG Cache

Space in the FSG Cache is not necessarily evenly distributed among AMPs. It is more like a pool of memory; each AMP uses what it needs.

This cache contains as many of the most recently used database segments as will fit in it. When Teradata Database tries to read a database block, it checks the cache first. If the block is cached, Teradata Database avoids the overhead of rereading the block from disk.

The system performs optimally when FSG Cache is as large as possible, but not so large that not enough memory exists for the database programs, scratch segments, and other operating system programs that run on the node.

Calculating FSG Cache Size Requirements

The FSG Cache percent field controls the percentage of memory to be allocated to FSG Cache. You can change the value in FSG Cache percent using ctl (on Linux or Windows). See “DBSCacheThr” in Utilities.

As a priority, configure for sufficient operating system memory first, using the guidelines discussed in “Free Memory” on page 152. Then let the remaining memory be allocated to FSG Cache.

Cylinder Slots in FSG Cache

An FSG segment is the basic unit of memory buffer that the PDE provides for Teradata Database file system to manage and access data. When a task requires an FSG segment, the corresponding data is mapped into the FSG virtual address space.

With Cylinder Read, the FSG Cache can be viewed as consisting of two regions:

• Cylinder pool

• Individual segment

The cylinder pool occupies the high region and is cut into cylinder-sized memory slots.

Calculating FSG Cache Read Misses

To calculate if FSG Cache read misses have increased, use the following formulas:

• FSG Cache read miss = physical read I/O divided by logical read I/O

Physical read I/O counts can be obtained from ResUsageSpma table by adding FileAcqReads + FilePreReads.

Logical I/O counts can be obtained from ResUsageSpma table column FileAcqs.

• Increase in FSG Cache misses = FSGCacheReadMissAfter divided by FSGCacheReadMissBefore

Performance Management 155

Page 156: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryUsing Memory-Consuming Features

While Teradata Database cannot guarantee a particular improvement in system performance, experience has shown gains of 2-8% when adding 1GB of memory per node in such instances.

Using Memory-Consuming Features

Be aware that certain features may require more memory in order to show their optimal performance benefit. Of particular mention are:

• External Stored Procedures and table functions

• Large objects (LOBs) and user-defined functions (UDFs)

• PPI and compression

• Join index, hash-join, stored procedures and 128K datablocks

While each of the above features will function and even show performance gain in most instances without additional memory, the gain may be countered by the impact of working with a fixed-sized memory.

In turn, you may experience more segment swaps and incur additional swap physical disk I/O. To counter this, you can lower the FSG cache percent to assure that 135 MB per AMP is allocated in OS memory for 64-bit systems.

However, lowering the FSG cache percent may cause fewer cache hits on table data and instead cause a different type of additional physical disk I/O. In general, additional I/O on table data is not as severe a performance issue as swapping I/O, but it can still have a measurable impact on performance.

In a proactive mode prior to feature introduction, you can monitor the use of FSG cache memory to determine if you should add more memory to assure full performance.

To do this:

• Monitor your existing system during critical windows to understand the ratio of logical to physical I/Os.

• After the lowering of the FSG cache percent to provide more memory to the new feature, again monitor your existing system during critical windows to understand the ratio of logical to physical I/Os.

• If the amount of FSG cache misses increases by more than 20% and the system has become I/O-bound, then adding more memory, if possible, is recommended.

File System and ResUsage

Note the following:

• Cylinder Read counts ResUsage data in either the File system code or the PDE/FSG code, thereby identifying the code from which ResUsage data comes.

156 Performance Management

Page 157: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryManaging I/O with Cylinder Read

• There are ProcPend (Process Pending) and ProcWait (Process Waiting) ResUsage counters for SEG locks and FSG locks.

• There are ResUsage counters for TSKQNL, the service used to coordinate appending records to the WAL log.

• ResUsage counts are performed against the originating vproc.

• Certain ResUsage fields are renamed.

For the specific ResUsage fields in the ResUsageSvpr and ResUsageScpu tables that will be:

• Used by PDE/FSG only for Cylinder Read

• Used to distinguish SEG and FSG sleep locks

• Used to account for TSKQNL

• Renamed

see Resource Usage Macros and Tables.

Managing I/O with Cylinder Read

Cylinder Read is a method of loading datablocks in a single database I/O operation. Instead of block I/Os for operations such as table scans and joins that process most or all of the datablocks of a table, Cylinder Read issues an I/O of up to 2 MB.

Because Cylinder Read loads the desired datablocks in a single database I/O operation, Teradata Database incurs I/O overhead only once per cylinder rather than for each data block.

Cylinder Read Settings

The Cylinder Read setting and the number of slots/AMP value are interdependent, as follows.

IF the Cylinder Read field is set to … THEN …

DEFAULT the value for Cylinder Slot/AMP is calculated automatically.

If you set the slider to a value, the setting is ignored.

USER you can set the Cylinder Slots/AMP value yourself.

However, based on FSG Cache size, in rare cases FSG may have to change the number of slots per AMP.

Teradata recommends that as a general rule the default setting provides the best performance.

For an explanation and instructions on how to check the current allocation after a reset, see“Viewing the Cylinder Slot Configuration” on page 158.

Performance Management 157

Page 158: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryManaging I/O with Cylinder Read

Viewing the Cylinder Slot Configuration

During system reset, FSG recalculates the size of FSG Cache and determines if enough memory exists to allocate the number of slots per AMP that you selected.

If not, or if you did not select a number, FSG attempts to allocate the default. Otherwise, it allocates as many slots as it can. For example, only two slots can be configured when FSG Cache is down to 36 MB per AMP.

Therefore, it is possible, though not likely, that after a system reset the number of slots configured by FSG may be different from your selection. You can find the actual slot configuration using Database Window. For detailed information on Database Window, see Utilities.

Any time the system restarts, the number of slots being used may change due to memory availability. Typically this only happens when a node drops out of the configuration.

Since each surviving node is then responsible for more vprocs within the same clique, the number of cylinder slots used may need to be reduced. The Cylinder Read Ageing Threshold in the DBC Control (see Utilities) indicates the maximum % of FSG cache that should be used for cylinder reads.

Cylinder Read and I/O

With larger block sizes, it is possible that the frequency of Cylinder Reads will decrease, particularly for moderate or smaller tables. This may increase block-at-a-time I/O requests, driving up I/O demand.

To be a candidate for Cylinder Read on an AMP, the cylinder must contain 6 or more datablocks for the table being accessed. When you increase the block size drastically, more rows fit into each data block and fewer datablocks are needed to hold the rows of the table on each AMP. For large tables this may not be a problem, but you may end up doing more I/O operations on moderate or smaller tables.

One of the advantages of Cylinder Read is that it allows you to keep smaller block sizes, if you find advantages in doing so and still have the advantage of reading lots of rows in one I/O operation when scanning.

If most of your access is done by means of full table scans, and there is no tactical component to consider, increasing cylinder slots per AMP presents another opportunity for reducing overall I/O operations.

Using Cylinder Read for WAL Log Maintenance

Setting Cylinder Read to LogOnly using ctl limits cylinder scan to WAL log maintenance only. With LogOnly, the average size of the WAL log should be reduced. Table scan jobs may run slower than before because individual datablocks are read rather than whole cylinders. But there is improvement in WAL log maintenance because such maintenance no longer has to compete with scan jobs for the fixed number of available cylinder slots.

Setting Cylinder Read to LogOnly may reintroduce some Cylinder Read performance anomalies that may have caused you to disable Cylinder Read in the first place. But completely

158 Performance Management

Page 159: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryManaging I/O with Cylinder Read

disabling Cylinder Read can cause WAL log maintenance to fall behind, thereby causing a general system slowdown.

But by setting Cylinder Read to LogOnly, users only use Cylinder Read for WAL log maintenance and not for table scans, and because WAL log maintenance only runs in the background once a minute, the number of anomalies should be less than those you may have experienced when Cylinder Read was fully enabled.

Tracking Cylinder Read Resource Usage

To track Cylinder Read behavior if you enable resource usage logging, see the Cylinder Read Columns in the “ResUsageSvpr Table” in Resource Usage Macros and Tables.

Performance Management 159

Page 160: Teradata Database Performance Management

Chapter 11: Using, Adjusting, and Monitoring MemoryManaging I/O with Cylinder Read

160 Performance Management

Page 161: Teradata Database Performance Management

CHAPTER 12 Performance Tuning and DBSControl Utility

This chapter discusses performance implications of DBS Control utility fields. For more information on these fields, see Utilities.

DefragLowCylProd

A low value in this field reduces the performance impact of defragmentation. However, setting the value extremely low might cause cylinders to be consumed more quickly, which could cause more MiniCylPacks to run.

Set DefragLowCylProd higher than MiniCylPackLowCylProd because defragmentation has a smaller performance impact than cylinder pack.

DictionaryCacheSize

The default value allows more caching of table header information and reduces the number of I/Os required. It is especially effective for workloads that access many tables (more than 200) and for those that generate many dictionary seeks.

Increase the size of the dictionary cache to allow the parser to cache additional data dictionary and table header information.

For tactical and Online Complex Processing (OLCP) type workloads, maintaining a consistently short, few-second response time is important. These workloads may benefit from a larger dictionary cache, particularly when their query plans have not been cached in the request cache. A larger dictionary cache will allow more dictionary detail, needed for parsing and optimizing, to remain in memory for a longer period of time. For query workloads with a response time of more than one minute, there may be no measurable difference when this field is set to a higher value.

FreeSpacePercent

Reserved free space allows tables to expand within their currently allocated cylinders. This can prevent or delay the need for additional cylinders to be allocated, which incurs the overhead of moving data to the new cylinders. Avoiding data migrations can improve overall system performance.

Performance Management 161

Page 162: Teradata Database Performance Management

Chapter 12: Performance Tuning and DBS Control UtilityDisableMergeBlocks and MergeBlockRatio

Use the following guidelines to evaluate your free space needs:

• Evaluating a Higher Global FSP

Free space is allocated dynamically when a table expands as a result of modification operations. If some space is left free during the initial load, the need for subsequent migrations is reduced. If most of your tables will grow extensively, use a higher FSP value to achieve the best performance. However, if the percentage is too high, additional cylinders might be required to store the data.

For example, if you specify an FSP of 25%, a table that would otherwise fit on 100 cylinders would require 134 cylinders. If you specify an FSP of 75%, the same table would require 400 cylinders.

Make sure that the requisite cylinders are available. If they are not, MiniCylPacks might run, which can result in some performance degradation.

• Evaluating a Lower Global FSP

If little or no table expansion is expected, the lowest value (0%) should remain as the global default.

• Evaluating Global Versus Table-Level FSP

If many tables benefit from one FSP value and a few tables benefit from a different value, consider setting the FSP at the table level for the exceptions.

Carefully consider the performance impact of long-term growth against short-term needs before changing the global default value.

After setting a non-zero value for the free space percentage, all subsequent operations listed above respect that setting, and continue to reserve free space on the cylinder beyond what table data requires. To prevent free space from continuing to be reserved after data is loaded into tables, lower the FSP setting. That will make the free space available for data storage.

The newly available free space that had been reserved will be reclaimed when one of the following occurs:

• AutoCylPack background task runs to ensure cylinders are packed to FSP.

• The Ferret PACKDISK command is used.

• MiniCylPack runs due to a lack of available cylinders.

• Operations that do not honor the FreeSpacePercent setting are used to modify the table.

DisableMergeBlocks and MergeBlockRatio

During normal database operations, the data blocks that store table rows on cylinders can split and shrink, resulting in many blocks which are far smaller than the maximum allowed data block size. Full table modify operations for tables with several small data blocks require more disk I/O than would be required if the table rows were stored on fewer and larger data blocks. Teradata Database can merge the small data blocks of these tables automatically during full table modify operations, which can result in reduced I/O overhead and improved database performance.

162 Performance Management

Page 163: Teradata Database Performance Management

Chapter 12: Performance Tuning and DBS Control UtilityHTMemAlloc

HTMemAlloc

If your system is using large spool files, and the Optimizer is not using the hash join because of the HTMemAlloc limit, increase HTMemAlloc and see if performance improves.

For 32-bit systems:

• The only time that higher settings should be considered is when a system is always lightly loaded. This means that very few, if any, concurrent operations are performed using the system. In this case, you might want to increase HTMemAlloc to a value in the range of 3 through 5.

• Values in the 6 through10 range can improve hash join performance in some single-user situations but should not be specified on production systems. Do not use these values when more than one user is logged on. Most end users never have a need to use settings in these ranges.

IdCol Batch Size

The IdCol Batch Size setting involves a trade-off between insert performance and potential gaps in the numbering of rows inserted into tables that have identity columns.

A larger setting results in fewer updates to DBC.IdCol in reserving batches of numbers for a load. This can improve the performance of bulk inserts into an identity column table. However, because the reserved numbers are kept in memory, unused numbers will be lost if a database restart occurs, resulting in a gap in the numbering of identity columns.

PermDBAllocUnit

As tables are modified, rows are added, deleted, and changed. Data blocks grow and shrink dynamically to accommodate their current contents. However, data block sizes can change only in units of PermDBAllocUnit. This means there will nearly always be some unused space left at the end of the data block. If table modifications are relatively even, such incremental changes in data block size result in an average of approximately half an allocation unit of space wasted for every data block. (This is a rough approximation, and will depend on many factors that differ from database to database.)

In environments where new rows are added frequently to tables, or where tables with variable length rows are frequently growing, system performance might be improved slightly by increasing the allocation unit size. With a larger allocation unit, data blocks will not need to be enlarged as frequently, because there will already be room for additional changes. However, in environments where new rows are not added frequently, the additional space in each block can degrade performance by increasing the average I/O size.

Performance Management 163

Page 164: Teradata Database Performance Management

Chapter 12: Performance Tuning and DBS Control UtilityPermDBSize

Make only small changes to this setting at a time, and carefully evaluate the results before committing the change on a production system. Set the allocation unit to a multiple of the average row size of tables that change frequently, rounded up to the nearest sector.

Because the benefit of larger allocation units is often offset by the consequent increase in average wasted space, Teradata recommends that PermDBAllocUnit be left at the default setting.

PermDBSize

Database performance can be affected by the relationship of data block size to the type of work typically performed by the database:

• When database queries are tactical in nature, involving one or a few table rows, it is advantageous to have fewer rows stored per data block to speed data access. Online transaction processing (OLTP) is an example of this type of work.

• Alternatively, when database queries are strategic in nature, involving complex queries that involve many table rows per table, it is advantageous to have many rows stored in each data block, to minimize costly data I/O operations. Decision support software (DSS) and complex report generation are examples of this type of work.

PermDBSize sets the default maximum size used by the system for multirow data blocks in permanent tables. Use a larger value if the database is used primarily for strategic work, and a smaller value if the database is used primarily for tactical work.

In a mixed-work environment, determine a value for PermDBSize based on the kind of work typically performed by the database. For tables involved in other types of work, PermDBSize can be overridden on a table-by-table basis using the DATABLOCKSIZE option of the CREATE TABLE and ALTER TABLE SQL statements.

For example, if only a few tables are involved in decision support, overall throughput can be improved by up to 50% using the following strategy:

• Define a data block size of 254 sectors for the decision support tables.

• Set PermDBSize to 42, to serve as the default block size for the remaining tables, which are involved in tactical work.

• Use the smallest value for JournalDBSize, to expedite UPDATE operations.

PPICacheThrP

The current data block for the corresponding partition is associated with each context. The current set of data blocks (one for each context) are kept in memory, if possible, to improve the performance of processing the set of partitions at the same time. If there is a shortage of memory, these data blocks may need to be swapped to disk. Excessive swapping, however, can degrade system performance.

Larger values may improve the performance of PPI operations, as long as the following occur:

164 Performance Management

Page 165: Teradata Database Performance Management

Chapter 12: Performance Tuning and DBS Control UtilityPPICacheThrP

• Data blocks for each context can be kept in memory. When they can no longer be kept in memory and must be swapped to disk, performance may degrade.

• The number of contexts does not exceed the number of non-empty, non-eliminated partitions for PPI operations. (If they do, performance will not improve because each partition can have a context, and additional contexts would be unused.)

In some cases, increasing the value of PPICacheThrP above the default value can provide a performance improvement for individual queries that do these PPI operations. However, be aware of the potential for memory contention and running out of memory if too many of these queries are running at the same time.

The default setting of 10 is conservative, and intended to avoid such memory problems. With 80 AMP Worker Tasks (AWTs) on a system with the default setting of 10, the maximum amount of FSG cache memory per AMP that could be used for these PPI operations is 80% of FSG cache memory, if all AMPs are simultaneously executing PPI operations, such as sliding-window joins for 80 separate requests. For nonstandard configurations that have more than 80 AWTs defined as the maximum, the setting is scaled to the number AWTs. For example, at the default setting of 10, a cap of 80% of FSG cache memory per AMP would still be in effect on such systems.

For many sites, the default may be too conservative. All 80 AWTs might not be running PPI operations at the same time. If, at most, 60 PPI operations are expected to occur at the same time, the value of PPICacheThrP could possibly be raised to 15. If at most 40 are expected, the value could possibly be raised to 20, and so on. The best value for this parameter is dependent on the maximum concurrent users expected to be on the system and their workload. No one value is appropriate for all systems.

Also, consider that the number of concurrent PPI operations, such as sliding-window joins, may increase as PPI usage is increased. Increasing the value may increase performance now without memory contention or running out of memory but, in the future, as more PPI operations run concurrently, performance may decrease, or out of memory situations may occur.

If less than 80 concurrent PPI operations are expected for your site, and you think that better performance may be possible with an increased value for PPICacheThrP, you can experiment with PPICacheThrP settings to determine an increased PPICacheThrP setting that is optimal for your site and safe for your workloads. Measure pre- and post-change performance and degree of memory contention under expected current and future workloads to evaluate the effects of the change. If increasing the value to one that is reasonably safe for your site does not yield adequate performance for PPI operations such as sliding-window joins, consider defining partitions with larger granularity, so fewer partitions are involved in the sliding-window join.

Performance Management 165

Page 166: Teradata Database Performance Management

Chapter 12: Performance Tuning and DBS Control UtilityRedistBufSize

RedistBufSize

If a system has relatively few AMPs, a larger redistribution buffer size usually has a positive effect on load performance. However, on larger systems with many AMPs, a large buffer size can consume excessive memory, especially if many load jobs are run concurrently.

RollbackPriority

Because rollbacks can involve millions or billions of rows, competing for CPU and other system resources, rollbacks can impact system performance. Rollbacks can keep locks on affected tables for hours or days until the rollback is complete. During a rollback, a trade-off occurs between overall system performance and table availability.

How RollbackPriority affects performance is not always straightforward, and is related to the Priority Scheduler configuration, job mix, and other processing dynamics. The RollbackPriority setting should only be changed after full consideration of the performance consequences:

• When RollbackPriority is set to FALSE, rollbacks are performed at system priority, a special priority higher than any user-assigned priority, that is reserved for critical internal work. As a result, faster rollbacks occur at the expense of other online performance.

The default setting of FALSE is especially appropriate when rollbacks are large, occurring to critical tables that are accessed by many users. It is better to complete these rollbacks as quickly as possible to maximize table availability.

• When RollbackPriority is set to TRUE, rollbacks are executed within the aborted job's Performance Group or workload. This isolates the rollback processing to the aborted job's priority, and minimizes the effect on the performance of the rest of the system. However, if the rollback places locks on tables that other users are waiting for, this causes a greater performance impact for those users, especially if the rollback is running at a low priority.

A setting of TRUE is appropriate when rollbacks are typically smaller, occurring to smaller tables that are less critical, and less extensively used.

Regardless of the RollbackPriority setting, rollbacks are never subject to CPU limits:

• When RollbackPriority is FALSE, rollbacks run in the system performance group where jobs use as much CPU as they require.

• When RollbackPriority is TRUE, rollbacks are not constrained by CPU limits at the system, Resource Partition, or Allocation Group level.

SkewAllowance

If you know your data very well and do not expect skewing at this extreme, you can set this value to 50%, which still allows for a skew that is double the size the Optimizer uses in its estimates.

166 Performance Management

Page 167: Teradata Database Performance Management

Chapter 12: Performance Tuning and DBS Control UtilitySkewAllowance

Values 20 through 74% can improve hash join performance in some single-user situations but should not be used on production systems. Do not use these values whenever more than one user is logged on. Most end users never have a need to use settings in these ranges.

A setting higher than 75% might improve hash join performance when the data is so severely skewed that it degrades performance. In this case, turn SkewAllowance off or try increasing the value to 80%.

Performance Management 167

Page 168: Teradata Database Performance Management

Chapter 12: Performance Tuning and DBS Control UtilitySkewAllowance

168 Performance Management

Page 169: Teradata Database Performance Management

CHAPTER 13 Teradata Active SystemManagement

This chapter discusses workload management using Teradata ASM.

Overview

Teradata ASM is a set of products, including system tables and logs, that interact with each other and a common data source. It facilitates automation in the following four key areas of system management:

• Workload management

• Performance tuning

• Capacity planning

• Performance monitoring

Teradata ASM helps manage the system, thus reducing the effort required by DBAs, application developers, and support personnel.

With careful planning, Teradata ASM can improve your workload management and performance. It can also improve response times and ensure more consistent response times for critical work.

Architecture

The following products and components fall within Teradata ASM:

• Teradata Viewpoint Workload Designer portlet. It defines and manages workloads per directives you provide. Using the Workload Designer you can:

a Specify workload management rules (filter rules, throttle rules, and Workload Definitions [WDs]).

b Define events that activate various health condition events and planned operating events (these are operating windows or time periods).

c Create a state matrix that maps workload management rules and health condition and planned operating events to workloads.

Each mapping defines a “state” to which a working value set corresponds. The working value set includes all the thresholds that apply when that state is active.

Performance Management 169

Page 170: Teradata Database Performance Management

Chapter 13: Teradata Active System ManagementAreas of Management

• DBS-based Regulator is a database component of Teradata ASM that automatically manages job flow and priorities based on WDs and their operating rules.

• Teradata Workload Analyzer (Teradata WA)

Teradata WA recommends Workload Definition (WDs) and the rules according to which they operate. It also provides a tool for analyzing the characteristics of requests with a WD.

• Teradata Viewpoint Workload Health Portlet

The portlet lets you monitor workload performance against workload goals in real-time.

• Teradata Analyst Pack

• “Performance Tuning”

• “Capacity Planning” via third-party tools and professional services offerings.

Note: Performance Tuning and Capacity Planning are conceptual stages within Teradata ASM for altering application and database design in the interest of greater system performance and understanding resource usage and performance trends respectively.

Areas of Management

Teradata ASM establishes a framework to accommodate enhancements in the four key areas of system management.

• Workload Management: It entails imposing on Teradata Database workload management rules to improve workload processing. Workload management includes resource control, request monitoring, and automated request performance management.

• Performance Tuning: Entails altering application designs, physical database design, database or other tuning parameters, or system configuration balance in order to yield greater system performance.

• Performance Reporting and Monitoring: Entails real-time and historical monitoring of system performance in order to identify, eliminate or otherwise solve performance anomalies and provide views into system health.

• Capacity Planning: Entails understanding current and projecting future resource usage and performance trends in order to maintain an environment with sufficient performance and data capacity relative to growth.

Note: Automating or advising with respect to performance tuning and capacity planning requires DBA intervention through use of such tools as are found in Teradata Viewpoint, Teradata Analyst Pack (for example, Teradata Index Wizard, Teradata Visual EXPLAIN, Teradata SET), and 3rd-party capacity planning and performance monitoring offerings.

All components of Teradata ASM architecture draw data from a common Systems Management Database, providing a basic level of integration.

Conceptual Overview

The figure below provides a conceptual overview of Teradata ASM.

170 Performance Management

Page 171: Teradata Database Performance Management

Chapter 13: Teradata Active System ManagementTeradata ASM Flow

Teradata ASM Flow

The Teradata Viewpoint Workload Designer portlet can be considered the starting point of Teradata ASM flow. In that portlet the DBA:

1 Specifies workload management rules (filter, throttle, and WDs).

2 Defines events that activate various health condition events and planned operating events (these are operating windows or time periods).

3 Creates a state matrix that maps workload management rules and system health condition and planned operating events to workloads.

Each mapping defines a “state” to which a working value set corresponds. The working value set includes all the thresholds that apply when that state is active.

The Workload Analyzer, a tool, aids the Workload Designer portlet. The Workload Analyzer assists the DBA in defining WDs through mining the query log for patterns and merging that information with DBA-driven workload grouping desires. The Workload Analyzer can apply best practice standards to WDs, such as assistance in SLG definition, Priority Scheduler (PS) setting recommendations, and migration from PS to Teradata ASM definitions. The Workload

1097A004

MonitoringReporting

ThroughTeradata Viewpoint

Teradata ViewpointWorkload Designer

WorkloadAnalyzer

Wo

rklo

ad

CapacityPlanning

PerformanceTuning

Optimizer

Regulator

SysMgmtData

Wo

rklo

ad

Alter

Designs

Usage

Project

Actu

al

Go

als

vs.

Pro

file

Performance Management 171

Page 172: Teradata Database Performance Management

Chapter 13: Teradata Active System ManagementFollowing a Request in Teradata ASM

Analyzer can also be used as an independent analytic tool to understand work-load characteristics.

The Optimizer provides the Regulator with estimates that are used in analyzing filter and throttle rules and WDs.

The Regulator, a DBS-embedded component of Teradata ASM, provides dynamic management of workloads, guided by the rules provided through the Workload Designer portlet. By being integrated into the database, the Regulator is a proactive, not a reactive tool for managing workloads.

Reporting / Monitoring tools and applications in Teradata Viewpoint and the Workload Analyzer monitor the system. They provide various ad hoc and standard views with respect to workload behavior. This includes the ability to track workload performance against defined SLGs. Based on resulting behaviors, such as not meeting SLGs, the DBA can choose to find performance tuning opportunities, do capacity planning and/or workload management refinement.

Performance Tuning and Capacity Planning tools are loosely integrated tools.

Following a Request in Teradata ASM

The following figure illustrates how Teradata ASM handles a request.

172 Performance Management

Page 173: Teradata Database Performance Management

Chapter 13: Teradata Active System ManagementFollowing a Request in Teradata ASM

After the user submits the request, and as the request passes through the Teradata Dispatcher, the request is checked against filter and throttle rules and classified to execute under the rules of the appropriate WD. For specific information on WD classification criteria, see Workload Management API: PM/API and Open API.

If concurrency throttles exist, the request query is passed for concurrency management to the Query Delay Manager. It releases the requests for execution as concurrency levels reach acceptable thresholds.

Requests are then executed under the control of the Priority Scheduler.

Note: When Teradata ASM workload definition (Category 3) is enabled, the Priority Scheduler PG portion of the account string is ignored for purposes of priority assignment. Instead, workload classification determines the priority of the request.

Throughout execution of the request, the Exception Monitor monitors for exception criteria and automatically takes the designated action if the exception is detected.

Priority SchedulerPriority Scheduler

Requests filtered,

then classified into

a workload

TeradataDispatcher

Request

Query DelayManager

ExceptionMonitor

Preprocessing Processing

Requests throttled

to not exceed

workload

concurrency limits

Requests managed for

resource allowance and

exception actions

SystemRegulation

Operating Environment (Business) Events

System Performance& Availability Events

Apply new working value set as necessary

1097A006

Planned Environment

Events

Health Condition

Events

1097B006

Performance Management 173

Page 174: Teradata Database Performance Management

Chapter 13: Teradata Active System ManagementFollowing a Request in Teradata ASM

During request execution, the query log, the exception log and other logs keep track of the system demand from a workload-centric perspective. These logs can be accessed by the various Teradata ASM components (for example, the Workload Analyzer, Teradata Wizard) to monitor actual performance against SLGs or to show workload trends and for performance tuning opportunities.

174 Performance Management

Page 175: Teradata Database Performance Management

CHAPTER 14 Optimizing WorkloadManagement

This chapter discusses performance optimization through workload management.

Using Query Band

A query band is a set of name and value pairs that are defined by the user or middle-tier application.

You can set a query band for the transaction and session using the SQL statement, SET QUERY_BAND. For information on SET QUERY_BAND, see SQL Data Definition Language.

A query band can be:

• Logged by DBQL. Its reports are created using the query band name-values pairs to provide additional refinement for accounting and resource allocation purposes and to assist in troubleshooting performance problems.

• Used to determine the origin of a request that may be consuming system resources or blocking other requests.

For detailed information on Query Band APIs, including examples of Query Band APIs, see Workload Management API: PM/API and Open API.

Priority Scheduler

Priority Scheduler is a resource management tool that controls the dispersal of computer resources in Teradata Database. This resource management tool uses scheduler parameters that satisfy site-specific requirements and system parameters that depict the current activity level of Teradata Database. You can provide Priority Scheduler parameters to directly define a strategy for controlling computer resources.

The Priority Scheduler:

• Allows you to define a prioritized weighting system based on user logon characteristics or on Workload Definition (WD) rules.

• Prioritizes the workload in your data warehouse based on this weighting system.

• Offers utilities to define scheduling parameters and to monitor your current system activity.

Note: If Teradata ASM workload definition (Category 3) is active, no Priority Scheduler modifications are allowed through schmon.

Performance Management 175

Page 176: Teradata Database Performance Management

Chapter 14: Optimizing Workload ManagementPriority Scheduler

Design Goals

Priority Scheduler includes default parameters that provide four priority levels; all users are assigned to one level.

To take advantage of Priority Scheduler capabilities:

• Assign users to one of the several default priority levels based on a priority strategy.

• Define additional priority levels and assign users to them to provide a more sophisticated priority strategy.

• Assign user who execute very response-sensitive work to a very high priority level to support Teradata Database applications.

For a description of the structure of and relationships between the scheduling components and parameters of Priority Scheduler, see “Priority Scheduler” in Utilities.

When setting up for Priority Scheduler, Teradata recommends:

• A low number of active AGs are preferred. 5 or 6 is a good target.

• 1 or 2 Resource Partitions (RPs) to cover all user work.

• A reasonably higher weight assigned to tactical query components, compared to those supporting other user work.

• A demotion AG, if needed.

One possible RP setup is the following.

The recommended setup assumes that tactical queries are highly tuned and that they demonstrate the following characteristics: that all-AMP queries consume low levels of CPU per node.

Bringing together all PGs doing nontactical work into a single RP makes it:

• Easier to understand priority differences among AGs.

• Simpler to setup and tune.

• Less complex when faced with growth.

• Easier when several PGs to share a single AG.

RP Weight Description

Default 40 Console utilities

Tactical (optional) 40 Highly-tuned tactical queries only

Standard 20 All nontactical user-assigned work

176 Performance Management

Page 177: Teradata Database Performance Management

CHAPTER 15 Performance Reports and Alerts

This chapter describes performance reports and alerts.

Using Performance Reports

There are several kinds of system performance reports that are helpful in collecting performance data. These include reports that look at:

• Weekly trends, which establish ongoing visibility of system performance.

• Specific kinds of trends, including exception-based reporting

Standardized report and view help facilitate coordination among, for example, Teradata Support Center, Engineering, and Professional Services (PS).

Some Symptoms of Impeded System Performance

System Saturation and Resource Bottlenecks

All database systems, including Teradata Database, reach saturation from time to time, particularly in ad hoc query environments where end-users may saturate a system unless you control elements such as spool space or job entry.

System saturation and bottlenecks are interrelated. When the system is saturated, the bottleneck is usually some key resource, such as a CPU or disk. Looking at how often a resource or session is in use during a given period and asking such questions as the following, for example, help identity resource bottlenecks:

• How intensively was the AMP CPU used?

• Are all AMPs working equally hard?

• What are I/O counts and sizes for disks, BYNET, and client channel connections or Ethernet?

You can use the information obtained from resource usage, as well as system monitoring tools, to find where, and when, bottlenecks occur. Once you know which resource is frequently the bottleneck in certain applications, you can, for example, modify your job entry or scheduling strategies and justify system upgrades or expansions or tune your workloads for more efficient use of resources.

Performance Management 177

Page 178: Teradata Database Performance Management

Chapter 15: Performance Reports and AlertsMeasuring System Conditions

Processing Concurrency

Transaction locks are used to control processing concurrency. The type of lock (exclusive, write, read, or access) imposed by a transaction on an entity (database, table, rowhash rank, or rowhash) determines whether subsequent transactions can access the same entity.

A request is queued when a lock it needs cannot be granted because a conflicting lock is being held on the target entity. Such lock conflicts can hamper performance. For example, several jobs could be blocked behind a long-running insert into a popular table.

To resolve lock conflicts, you need to identify what entity is blocked and which job is causing the block. Then you may want to abort the session that is least important and later reschedule the long job to run in off hours.

Deadlocks

A deadlock can occur when two transactions each need the other to release a lock before continuing, with the result that neither can proceed. This occurrence is rare because Teradata Database uses a pseudo table locking mechanism at the AMP level, but it is possible. To handle the exceptions, the dispatcher periodically looks for deadlocks and aborts the longest-held session.

You can control the time it takes for a deadlock to detect and handle a deadlock automatically by shortening the cycle time of the Deadlock Detection mechanism. You can modify this value in the tunable Deadlock Timeout field of the DBS Control Record.

Measuring System Conditions

The following table summarizes recommended system conditions to measure.

178 Performance Management

Page 179: Teradata Database Performance Management

Chapter 15: Performance Reports and AlertsMeasuring System Conditions

System Conditions What to Use What to Collect How to Use

Response time Heartbeat queries

• Response-time samples

• System-level information

Saved samples can be used to:

• Track trends

• Monitor and alert

• Validate Priority Scheduler configuration

Baseline testing

• Response-time samples

• Execution plans

• Check for differences +/- in response time.

• Check EXPLAINs if degradation is present.

• Collect / keep information for comparison when there are changes to the system.

Database Query Log (DBQL)

Application response time patterns

Track trends:

• To identify anomalies

• To identify performance-tuning opportunities

• For capacity planning

Resource utilization

Resource Usage

• ResNode Macro set

• SPMA summarized to one row per node

Look for:

• Peaks

• Skewing, balance

AMPUsage CPU and I/O for each unique account

Use this to quantify heavy users.

DBQL CPU, I/O, spool used, rows returned

Data growth • Script row counts

• Permspace

Summary row per table once a month

Look at trends.

Changes in data access

DBQL Summary row per table and the number of accesses once a month

Look for increases in access, trends.

Increase in the number of active sessions

LogOnOffV, acctg

Monthly and summarized number of sessions

Look for increases in concurrency and active users.

Performance Management 179

Page 180: Teradata Database Performance Management

Chapter 15: Performance Reports and AlertsUsing Alerts to Monitor the System

Using Alerts to Monitor the System

The Teradata Viewpoint alerting monitors Teradata Database and automatically invokes actions when critical (or otherwise interesting) events occur.

• Teradata Viewpoint alerting can generate alerts based on the following events:

• System

• Node

• vproc

• Session

• SQL

• Manual

• Canary query

• System health

• Teradata Viewpoint alerting can:

• Send e-mail to an administrator

• Send an SNMP trap

• Run a program

• Run a BTEQ script

• Write to the alert log

Suggested Alerts and Thresholds

The following table lists key events and the values that constitute either warning or alerts.

Increase in system demand

DBQL Query counts and response times, plus other query information

Look for trends, including growth trends, and measure against goals.

Resource Usage

System demand

AMPUsage Queries with errors or queries that have been canceled

System Conditions What to Use What to Collect How to Use

Type Event Warning Critical

CPU saturation Average system CPU > x% (x = 95)

I/O saturation CPU+WIO > x% and WIO > y% (x = 90, y = 20)

180 Performance Management

Page 181: Teradata Database Performance Management

Chapter 15: Performance Reports and AlertsWeekly and/or Daily Reports

Weekly and/or Daily Reports

Weekly reports provide data on performance trends.

To make weekly and/or daily reporting effective, Teradata recommends that you establish threshold limits. Note that filtering on key windows having a 24-hour aggregate of weekly aggregates that includes low-used weekends and night hours distorts the report.

Furthermore, Teradata recommends that you analyze the longest period possible. This avoids misleading trend data by softening temporary highs and lows.

Below are examples of some of weekly reports and their sources:

• CPU AV & Max from ResUsage.

This report provides data on the relationship between resources and demand

• CPU by Workload (Account) from AMPUsage

• Throughput by Workload from DBQL

This report provides data on how much demand there is.

• General Response times from Heartbeat Query Log

This report provides data on the general responsiveness of the system.

• Response Time by Workload from DBQL

This report provides data on how fast individual queries were responded to.

• Active Query & Session Counts by Workload

This report provides data on how many users and queries were active and concurrent.

• CurrentPERM

Query blocked Query or Session blocked on a resource for longer than x minutes, and by whom

(x = 60)

Entire system blocked

Total number of blocked processes > x (x = 10)

User exceeding normal usage

Number of sessions per user >x (with an exclusion list, and custom code to roll up sessions by user)

(x = 4)

“Hot Node” or “Hot AMP” problem

Inter-node or inter-AMP parallelism is less than x% for more than 10 minutes

(x=10)

Disk space Disk Use% > x (vproc) (x=90) (x=95)

Product Join Average BYNET > x% (system) (x=50)

System restart Restart SNMP

Node down Node is down SNMP

Heartbeat query Timeout SNMP

Type Event Warning Critical

Performance Management 181

Page 182: Teradata Database Performance Management

Chapter 15: Performance Reports and AlertsDetecting Resource-Intensive Queries through Scripts

This report provides data on how data volume is or is not growing.

• Spool

This report provides data on how spool usage is or is not growing.

Note: The above reports can, of course, be run daily.

Detecting Resource-Intensive Queries through Scripts

If you are aware via user feedback or a canary query that the system is slowing down, you can use:

• Query Monitor to detect resource-intensive queries.

• Teradata Viewpoint to highlight queries that have exceeded defined thresholds for key resource metrics.

• Teradata Viewpoint alerting to detect automatically resource-intensive queries and either abort them or notify a DBA.

• Develop scripts to execute at regular intervals and tie them to an alert.

Sample Script: High CPU Use

/*==================================================== *//* The following query provides a list of likely candidates toinvestigate for over-consumption of CPU relative to disk I/O. In general, we are only concerned with multiple amp requests and requests of long duration (cpu time > 10). We are using the ratio of: disk IO / CPU time < 100. Altermatively, you can use the ratio: (cpu * 1000 ms / io) > 10. The cut-off or red flag point will be system dependent. For the 4700/5150, the ratio will be higher than for the 4800/5200 which in turn will be higherthan the 4850/5250. *//*==================================================== */.logon systemfe,service.export file=hicpu.outSELECT ST.UserName (FORMAT 'X(10)', TITLE 'UserName'),ST.AccountName (FORMAT 'X(10)', TITLE 'AccountName'),ST.SessionNo (FORMAT '9(10)', TITLE 'Session'),SUM(AC.CPU) (FORMAT 'ZZ,ZZZ,ZZZ,ZZ9.99', TITLE 'CPU//Seconds') as cput,SUM(AC.IO) (FORMAT 'ZZZ,ZZZ,ZZZ,ZZ9', TITLE 'Disk IO//Accesses') as dio,dio/(nullifzero(cput))(FORMAT 'ZZZ.99999',TITLE 'Disk to//CPU ratio', NAMED d2c)from DBC.SessionTbl ST,DBC.Acctg ACWHERE ST.UserName = AC.UserNameand ST.AccountName = AC.AccountNameGROUP BY 1,2,3HAVING d2c < 100 and cput > 10ORDER BY 6 asc;.export reset.quit;[end]

182 Performance Management

Page 183: Teradata Database Performance Management

Chapter 15: Performance Reports and AlertsDetecting Resource-Intensive Queries through Scripts

Sample Script: Active AMP Users with Bad CPU/IO Access Ratios

/*==================================================== *//* The following query, which uses DBC.AMPUsage, returns,for the active users, CPU Usage & Logical Disk I/Os & Skewby Users with more than 10,000 cpu seconds, cpu or disk ioskew greater than 100X the average. *//*==================================================== */.logon systemfe,service.export file activeampskew.out.set defaults.set width 130LOCK DBC.Acctg for ACCESSLOCK DBC.sessiontbl for ACCESSSELECT DATE, TIME, A.accountName (Format 'x(18)') (Title 'AMPusage//Acct Name'), A.username (Format 'x(12)') (Title 'User Name'), DT.sessionNo (Format '9(10)'), A.vproc (Format '99999') (Title 'Vproc'), A.CPUTime (Format 'zz,zzz,zz9') (Title 'CPUtime'), DT.AvgCPUTime (Format 'zz,zzz,zz9') (Title 'AvgCPUtime'), A.CPUTime/NULLIFZERO(DT.AvgCPUTime)(Format 'zzz9.99') (Title 'CPU//Skew')(Named CpuRatio), A.DiskIO (Format 'zzz,zzz,zzz,zz9') (Title 'DiskIO'), DT.avgDiskIO (Format 'zzz,zzz,zzz,zz9') (Title 'AvgDiskIO'), A.DiskIO /NULLIFZERO(DT.avgDiskIO)(Format 'zzz9.99') (Title 'Disk//Skew')(Named DISKRatio)FROMDBC.AMPUsage A,(SELECTB.accountName, C.sessionNo, B.username, AVG(B.CPUTime), SUM(B.CPUTime), AVG(B.DiskIO), SUM(B.DiskIO)FROMDBC.AMPUsage B, DBC.SessionInfoV CWHEREB.accountname = C.accountNameGROUP BY 1, 2, 3HAVING SUM(CPUTime) > 10000) DT (accountName, sessionno, username, avgCPUtime,sumCPUtime, avgDiskIO, sumDiskIO)WHERE A.username = DT.usernameAND A.accountname = DT.accountnameAND (CpuRatio > 100.00 OR DiskRatio > 100.00)/* Add the following to zero in on a given vproc.*//*and vproc in (243,244,251,252,259,260,267,268,275,276) */ORDER BY 7, 1, 2, 3, 4, 5 ;.export reset.quit;[end]

Performance Management 183

Page 184: Teradata Database Performance Management

Chapter 15: Performance Reports and AlertsDetecting Resource-Intensive Queries through Scripts

Sample Script: Hot / Skewed Spool in Use

/*==================================================== *//* The following query will sometimes identify hot AMP problems. It is dependent on the running job having spool (which may not always be true), and the user having select rights on DBC tables (you could convert this to DBC views). Its use is largely based on checking the count of vprocs that are in use and comparisons of the avg, max, and sum values of spool on the vprocs. Note the calculation of Standard Deviation (P) and the Number of Deviations. The count of vprocs is simple, if it is less than the number of vprocs on the system; then the query is not distributing to all VPROCS. If the MAX is twice the AVG, then you have an out of balance condition, since a single VPROC has twice the spool of the AVG vproc.*//*==================================================== */LOCK DBC.DataBaseSpace for access LOCK DBC.DBase for access SELECT DBase.databasename as UserDB, sqrt(((count(css) * sum(css*css))- (sum(css)*sum(css)))/ (count(css)*count(css)))

(format 'zzz,zzz,zzz,zz9.99') as STDevP,(maxspool - avgspool ) / nullifzero(stdevp)

(format 'zz9.99') as NumDev ,((maxspool - avgspool) / maxspool * 100)

(format 'zzz.9999') as PctMaxAvg,count(*) (format 'zz9') as VCnt,avg(css) (format 'zzz,zzz,zzz,zz9') as AvgSpool,max(css) (format 'zzz,zzz,zzz,zz9') as MaxSpool ,sum(css) (format 'zzz,zzz,zzz,zz9') as SumSpool from DBC.Dbase, (select DataBaseSpace.DatabaseId, DataBaseSpace.VProc, DataBaseSpace.CurrentSpoolSpace as css FROM DBC.DataBaseSpace WHERE DataBaseSpace.CurrentSpoolSpace <> 0) DBS WHERE DBase.DatabaseID = DBS.DatabaseID GROUP BY UserDB;.quit; [end]

184 Performance Management

Page 185: Teradata Database Performance Management

CHAPTER 16 Baseline Benchmark Testing

This chapter discusses Teradata Database performance optimization through baseline benchmark testing.

Benchmark Test Suite Defined

A benchmark test suite is really nothing more than a group of queries. The queries picked for this purpose are more accurate and useful if they come from actual production applications.

Test beds such as these can be used to:

• Validate hardware and software upgrade performance.

• Measure performance characteristics of database designs and SQL features such as join indexes, temporary tables, or analytical functions.

• Assess scalability and extensibility of your solutions architecture (process and data model).

• Distinguish between problems with the platform or database software versus problems introduced by applications.

Tips on Baseline Benchmarking Tests

The following are tips on baseline benchmarking tests:

• Tests should reflect characteristics of production job mix.

Choose a variety of queries (both complex and simpler ones that are run frequently by users).

Remember to include samples from applications that may only be run seasonally (end of year, end of month, and so on).

• Tests should be designed to scale with the system.

Do not expect tables with 200 rows to scale evenly across a system with over 1,000 AMPs.

• Run the benchmark directly before and after upgrades, deployment of new applications, and expansions.

Comparing a test run to a baseline taken months earlier is not an accurate comparison.

• Run the benchmark under the same conditions each time.

Response times are relative to the amount of work being executed on the system at any particular time.

If the system was otherwise idle when the first test was run, it should also be idle when executing subsequent runs as well.

Note: Running the benchmark with the system ideal is best.

Performance Management 185

Page 186: Teradata Database Performance Management

Chapter 16: Baseline Benchmark TestingBaseline Profiling

Baseline Profiling

Baseline profiles provide information on typical resource usage. You can build baseline resource usage profiles for single operations (such as FastLoad, full table scans, PI INSERT . . . SELECTs, SELECT joins) and also for multiple, concurrently run jobs.

Maintaining profiles of resource usage by user and knowing how patterns of resource usage fluctuate at the site simplifies performance evaluation.

Performance Metrics for Baseline Profile

Teradata recommends the following types of performance metrics.

Metric Description

Elapsed time Time for a job or transaction to run from beginning to end, either in actual seconds, or within a set of specified time intervals, or below a specified time limit.

I/O rate Average number of I/O operations per transaction.

Throughput rate Any of the following:

• Transaction: total number of transactions in a job divided by job elapsed time.

• Rows: total number of rows in a table divided by elapsed time of an all-rows transaction.

• Parallel processing: rows per second per AMP or PE.

Resource utilization Percentage of time a resource (for example, CPU, disk, or BYNET) is busy processing a job.

For example, for a full table scan, CPU usage may be 30% busy and disk usage may be 70% busy.

Path time Time a resource spends per transaction or row, which you can calculate as resource utilization divided by throughput rate.

For example, a CPU utilization of 70% means the CPU is busy 70% of one second, or 0.7 of a second, or 700 milliseconds.

If the processing throughput rate is 10 transactions per AMP per second, calculate the path time by dividing 700 milliseconds by 10 transactions. The result is 70 milliseconds per transaction.

186 Performance Management

Page 187: Teradata Database Performance Management

CHAPTER 17 Real-Time Tools for MonitoringSystem Performance

This chapter provides information on real-time tools that monitor Teradata Database performance.

Teradata Viewpoint

Teradata Viewpoint enables database and system administrators and business users to monitor and manage Teradata Database systems from anywhere using a standard web browser.

Teradata Viewpoint allows users to view system information, such as query progress, performance data, and system saturation and health through preconfigured portlets displayed from within the Teradata Viewpoint portal. Portlets can also be customized to suit individual user needs. User access to portlets is managed on a per-role basis.

Database administrators can use Teradata Viewpoint to determine system status, trends, and individual query status. By observing trends in system usage, system administrators are better able to plan project implementations, batch jobs, and maintenance to avoid peak periods of use. Business users can use Teradata Viewpoint to quickly access the status of reports and queries and drill down into details. Teradata Viewpoint includes:

Component Description

Viewpoint Server The hardware on which the other components are installed. It is an Ethernet-connected server housed in the Teradata rack so it can be managed by server management software.

Viewpoint Portal A Java-based portal that resides on the Viewpoint Server and is the framework for delivering the Teradata Viewpoint portlets. The portal includes built-in security, user management, and role-based permissions for defining which users can perform specific actions and for granting or limiting system access.

Data Collection Service A Java process that tracks daily activities on Teradata Database and maintains data history for comparison and reporting purposes. Because it maintains a local cache database, users can access the system even during critical events, such as outages.

Portlets Preconfigured and customizable content that provides a current view of the workload and throughput of Teradata systems and a historical view of system capacity and use.

Performance Management 187

Page 188: Teradata Database Performance Management

Chapter 17: Real-Time Tools for Monitoring System PerformanceTeradata Utilities and System Performance

For more information on Teradata Viewpoint, contact your Teradata representative.

Teradata Utilities and System Performance

The following utilities have functions that allow you to monitor Teradata Database system performance.

ctl

The ctl utility, the Control GDO Editor, sometimes called the PDE Control program, lets you display and modify PDE configuration settings. These settings affect how the PDE handles startup, crashes, and the normal operation of Teradata.

ctl is available on Windows and Linux systems.

For information on ctl, see Utilities.

Query Session

The Query Session utility, qrysessn, provides information about active Teradata Database sessions. It allows you to monitor the state of all or selected database sessions on all or selected logical host IDs attached to the Teradata Database. Query Session is also known as Session States.

For detailed information on Query Session, see Utilities.

Show Locks

The Show Locks utility provides information about Host Utility (HUT) locks placed on databases and tables by the Archive/Recovery (ARC) utility during database operations.

These locks may interfere with application processing and should be released after utility processing is complete. Show Locks also displays information about locks placed during a Teradata Database system failure.

For detailed information on Show Locks, see Utilities.

Query Configuration

The Query Configuration utility, qryconfig, reports the current Teradata Database system configuration from a Teradata Database system console running the Database Window or from a host terminal. This configuration is defined using the Configuration and Reconfiguration utilities.

For detailed information on Query Configuration, as well as the Configuration and Reconfiguration, see Utilities.

188 Performance Management

Page 189: Teradata Database Performance Management

Chapter 17: Real-Time Tools for Monitoring System PerformanceTDP Monitoring Commands

Gateway Control

The Gateway Control utility, gtwcontrol, modifies the default values in the fields of the gateway control Globally Distributed Object (GDO).

There can be multiple gateways per node if the gateways belong to different host groups and listen on different IP addresses.

For detailed information on Gateway Control, see Utilities.

SHOWSPACE Command

The SHOWSPACE command in the Ferret utility displays storage space utilization for permanent, spool, WAL log, temporary, and journal data, including the amount of free space remaining.

Use the SHOWSPACE command in the Ferret utility to determine whether your disks require compression or your system requires expansion.

For more information on the SHOWSPACE command, see Ferret utility in Utilities.

TDP Monitoring Commands

You can use the following commands to investigate TDP activity.

These TDP commands display information about all sessions that are active on a specific TDP For more information on the TDP commands, see Teradata Director Program Reference.

System Activity Reporter

The System Activity Reporter (sar) is a command-line utility that allows you to monitor hardware and software information for a system running Linux or Windows. It is a node-local, generic tool, providing reports that are applicable to Teradata Database.

sar can produce snapshots of CPU activity that are useful for quick troubleshooting. Output is logged and can be displayed or printed. Use sar to obtain one or more of the following:

• Real-time snapshots at user-defined intervals

This TDP Command… Displays…

DISPLAY CELLS cell availability and usage information the TDP internal memory management system generates.

DISPLAY IFP status of PEs allocated to the TDP.

DISPLAY POOL information and statistics about one or more established session pools.

DISPLAY SESSIONS status of sessions communicating with Teradata through the TDP.

Performance Management 189

Page 190: Teradata Database Performance Management

Chapter 17: Real-Time Tools for Monitoring System PerformanceTOP

• Historical data stored as a binary flat file

Because you can set log times for intervals as short as five seconds, you can obtain snapshots in a stream of near-real-time information.

TOP

TOP, a standard tool for Linux, lists the tasks that used the most CPU time.

On Linux, the top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel.

The types of system summary information shown and the types, order and size of information displayed for tasks are all user configured and that configuration can be made persistent across restarts.

BYNET Link Manager Status

You can use the BYNET Link Manager Status (blmstat) utility to troubleshoot BYNET problems.

Enter the following command to start the blmstat utility:

blmstat -qv | grep BrdActive

Interpreting the Results

If the output is approaching or exceeding 40%, the BYNET is saturated due to broadcast messages. Saturation due to broadcast messages causes slow point-to-point messages, which causes the entire system to slow down.

Example Output

Following is example output of the blmstat utility:

BrdActive% 10 BrdActive% 10

AWT Monitor

The AWT Monitor utility (awtmon) collects and displays a user-friendly summary of the AMP Worker Task (AWT) in-use count snapshots for the local node or all nodes in the Teradata Database system. This information is useful for debugging purposes, including hot-AMP conditions.

For output examples and descriptions of available awtmon options, see Utilities.

190 Performance Management

Page 191: Teradata Database Performance Management

Chapter 17: Real-Time Tools for Monitoring System Performanceampload

Performance Benefit

awtmon enables efficient debugging of hot AMPs by reducing the manual effort associated with collecting and sorting puma -c command output.

Developers and support personnel can quickly view the snapshot of AWTs in-use to troubleshoot any performance problem.

For more information on awtmon, see Utilities.

DBC.ResUsageSAWT Table

The system collects ResUsage data for the DBC.ResUsageSAWT table if logging is enabled. The table holds information about the mailbox for each AMP, whether or not the system is in flow control, and the maximum number of AWTs for a vproc on a node. For more information, see Resource Usage Macros and Tables.

ampload

Because there are a limited number of AWTs available, the system cannot do additional work if all AWTs are in use.

The ampload utility enables you to view the following information for each AMP:

• The number of AWTs available to the AMP vproc

• The number of messages waiting (message queue length) on the AMP vproc.

For more information on how to use the ampload utility, see Utilities.

Differences Between ampload and awtmon Utilities

The differences between ampload and awtmon are described in the following table.

awtmon ampload utility

awtmon prints the break down of the AWT INUSE count by work type, that is, WORKNEW, WORKONE, etc.

Breakdown of work type for active AWTs is useful for troubleshooting performance problems, and more specifically, finding hot-amps per node.

ampload only prints AWT INUSE count.

awtmon prints local nodes AWT INUSE count information by default and [-s] option for system-wide information via pcl.

When invoked from a command prompt window, ampload displays only local node info. When invoked from the CNS/DBW supervisor window, ampload prints only system-wide info.

Performance Management 191

Page 192: Teradata Database Performance Management

Chapter 17: Real-Time Tools for Monitoring System PerformanceResource Check Tools

Resource Check Tools

Resource Check Tools (RCT) is a suite of usage sampling tools and utilities designed to assist you in:

• Identifying a slow down or hang of Teradata Database.

• Providing system statistics to help you determine the cause of the slowdown or hang.

RCT includes the following.

awtmon provides [t [n]] options for a loop mode

ampload does not provide options for a loop mode. You must manually submit the ampload command repeatedly if you wish to collect several reports for comparison.

awtmon provides [-t count] option to filter out and reduce the output to report any vproc where the AWT in-use count is less than a number you specify.

ampload always reports the vproc number, node ID number, message count, and AWT availability count for each vproc across the entire system.

awtmon attempts to reduce the number of outputs by default. This helps reduce the output on a large MPP system. Also, no line is printed if a vproc's AWT in-use count is ZERO.

ampload prints a line for each vproc even when AWT in-use count is ZERO.

awtmon does not report the number of messages waiting on the AMP.

ampload allows you to see how many messages are waiting on the AMP.

awtmon ampload utility

192 Performance Management

Page 193: Teradata Database Performance Management

Chapter 17: Real-Time Tools for Monitoring System PerformanceResource Check Tools

For details on how to run these tools, see Utilities.

Tool Description

dbschk Identifies if the system is hung or congested.

By default, when the PDE reaches the TPA/TPA ready state, dbschk is started to run as a background task.

dbschk normally runs in batch mode, but it can an also be run interactively against a functional Teradata system.

Multiple instances can run simultaneously. The results from all instances are logged to the same log file unless you specify a different filename to each instance.

RCT option, dbschk, captures data during a performance problem. A user-specified trigger in dbschk calls a script (like perflook.sh) or another tool (like syscheck, another RCT option) to determine the system state at the time of the slowdown.

Such scripts / tools run in the background, invisible to the user, creating logs that assist in determining the root cause of a slowdown and revealing system performance issues. The script or tool fires as soon as dbschk logs a performance event to the streams log. Moreover, it does not wait for the script or tool to complete, but continues monitoring.

dbschk determines if the performance of the system is slowing down or if the system is throttled because it has too much work. dbschkrc provides, for example, information about timeout, rate of collection, job, debug, and delay.

nodecheck Provides local, node-level resources values, such as free memory, free swap space, and available AMP worker task information, on the local node.

Also provides summary data to syscheck for analysis.

Notifies you of resources that have reached WARN or ALERT levels. You can modify threshold values to make a customized syscheckrc file.

Collected information is reported when you run syscheck. The node-level resource information is located in the node only section of the syscheckrc configuration file.

syscheck This system-wide tool (as compared to nodecheck, which is node-only tool):

• spawns an instance of nodecheck on all live TPA nodes. nodecheck gathers data from live components unless you invoke syscheck with the -t option. With -t , nodecheck reads the data from its log file.

• compares the nodecheck results from each node against threshold values defined in the local syscheckrc file or files.

• displays the current resource values on the local node.

• displays current resource status or if they have reached WARN or ALERT levels.

syscheckrc configuration file

A file containing user-defined parameters that syscheck and nodecheck employ as criteria to determine when certain statistics of the system have reached alert or warning levels.

Performance Management 193

Page 194: Teradata Database Performance Management

Chapter 17: Real-Time Tools for Monitoring System PerformanceCheckTable

CheckTable

Index Performance

Level-two checking supports CheckTable for large tables defined with SIs (tables with more than 20 millions rows).

Compressed Value Validation

CheckTable supports an option, compresscheck, that provides improved checking and validation of compressed values in the table header.

Having CheckTable check the compressed multivalue helps avoid delays during migration for tables containing compressed values.

For specific information, see Utilities.

Performance on Large Systems

Dictionary Check always attempts to use fast mode. In that mode, CheckTable collects no error information. It reports only success or failure.

If an error is found, Dictionary Check collects and reports detailed error information such as missing or extra table headers.

Dictionary Check refers to verifying that each table in the dictionary (in DBC.TVM and DBC.TEMPTABLES) also exists on all AMPs.

For more information, see Utilities.

Workload Management APIs and Performance

Workload Management API consists of interfaces to PM/APIs and open APIs. These interfaces are used to:

• Monitor system and session-level activities.

• Manage Teradata Active System Management (ASM) rules.

• Update components stored in a custom database called TDWM.

• Track system usage and manage task priorities.

For specific information on PM/APIs and open APIs, see Workload Management API: PM/API and Open API.

194 Performance Management

Page 195: Teradata Database Performance Management

Chapter 17: Real-Time Tools for Monitoring System PerformanceWorkload Management APIs and Performance

PM/APIs

What PM/APIs Collect

PM/APIs collect performance data and set data sampling and logging rates. Collected data is stored in memory buffers and is available to PM/APIs with little or no performance impact on the Teradata Database.

Because PM/APIs collect data in memory and not in a spool file on disk, PM/API queries cannot be blocked. They thus incur low overhead.

Note: The exception to this rule is IDENTIFY, which obtains the username of a session, database, user or table name from database, user and table IDs. IDENTIFY uses access locks on the system tables DBC.SessionTbl, DBC.DBase, DBC.TVM.

PM/APIs store node and vproc ResUsage data and session-level usage data in separate collection areas. Data is updated once during each sampling period. All users share the collected data.

PM/API data may be used to show how efficiently Teradata Database is using its resources, to identify problem sessions and users, and to abort sessions and users having a negative impact on system performance.

Collecting and Reporting Processor Data

PM/APIs report processor (node and vproc) usage data only for the most recent sampling period.

The data from each subsequent sampling period overwrites the data collected during any preceding sampling period.

Collecting and Reporting Session-Level Usage Data

PM/APIs report cumulative results of session-level usage data.

Session-level usage data is collected cumulatively. The sampling period is used to limit the frequency at which cumulative data is updated. For example, if you set the session usage sampling interval to 60 seconds and issue a MONITOR SESSION request, session-level usage data is cumulatively totaled and updated every 60 seconds. The session-level data reported includes data for the beginning 60 seconds as well as for subsequent intervals.

Note: Other data, such as locking information and AMP state, is collected at the AMP level and is not stored cumulatively.

Open APIs

The open API provides an SQL interface to System PMPC through user-defined functions (UDFs) and external stored procedures.

The following are examples of some of the UDFs and external stored procedures used:

Performance Management 195

Page 196: Teradata Database Performance Management

Chapter 17: Real-Time Tools for Monitoring System PerformanceWorkload Management APIs and Performance

Most of the SQL interfaces available to System PMPC provide similar functionality to the CLIv2 interfaces.

Note: Most open APIs do not follow transaction rules. If, within a transaction, a UDF or external stored procedure is called that performs an action (for example, setting a session account string) and the transaction rolls back, the action of the UDF or external stored procedure is not rolled back.

However, those interfaces that update the Teradata Dynamic Workload Management database, such as the TDWMRuleControl, TDWMObjectAssn, and TDWMSetLimits procedures, must follow transaction rules. If one of these interfaces is called within a transaction, the update will be rolled back if the transaction is aborted.

Use the ... To ...

AbortSessions function abort queries submitted by a set of users that have been running longer than 10 minutes and have been skewed by more than 10% for 5 minutes.

TDWMRuleControl function temporarily enable a rule to block an application from accessing a database while it is synchronized between two active systems.

GetQueryBandValue procedure query the DBQLogTbl based on specified names and values in the QueryBand field.

196 Performance Management

Page 197: Teradata Database Performance Management

CHAPTER 18 Troubleshooting TeradataDatabase Performance

This chapter describes troubleshooting issues and provides tips and techniques to help handle performance problems.

Busy Systems

CPU Saturation

Systems that run frequently at or near capacity are often the subject of assessments in which attempts are made to determine the extent of resource exhaustion.

When CPU is the binding factor and the scarce resource on a system, it is useful to determine the highest level of saturation of the system, that is, the point to which you can drive the system before it becomes so overloaded that it appears to be hanging.

Once a system reaches this level of saturation, the system should still be able to work itself out, but that may require extra time to do so.

Without appropriate performance monitoring, users may start, once the system appears to be hanging, to:

• Abort jobs that could cause rollbacks, or

• Submit duplicate queries that create more work on an already-exhausted system.

While ResUsage data provides the bulk of the information needed to know that a system is at 100% busy with respect to CPU or CPU+I/O Wait, other information may also be needed in order to examine the extent of CPU saturation and system congestion. In other words, one can know that the system is running at capacity, but not the extent of the overload.

When the system becomes so busy that logons become slow or hung, performance monitoring is not able to determine whether the system is actually hung or simply overloaded without using other tools.

Suggested Monitoring Techniques

Since the goal is to be able to drive the system as hard as possible without overloading it, some techniques for assessing the level of busy can be used:

When CPU usage is high:

1 Check AWT utilization. If the number is constantly at or near maximum, then

2 Check the message flow control. If there are tasks apparently in flow control, then

Performance Management 197

Page 198: Teradata Database Performance Management

Chapter 18: Troubleshooting Teradata Database PerformanceBusy Systems

3 Check run queue (sar –q) or ResUsageSawt MailBoxDepth data. If the run queue is growing longer and longer, the system is too busy and will slow down dramatically.

See “System Activity Reporter” on page 189.

While it is not unusual to see a busy system with high AWT counts, the presence of flow control means that some tasks are currently being blocked from sending more work to AMPs that are very busy.

If the run queue grows longer and longer, more work is in the CPU run queue, and each process will have to wait longer in turn before it is again switched in.

Finding a Saturated Resource

Use RCT to check for saturated resources.

High I/O Wait

Teradata recommends configurations with CPU to I/O bandwidth ratios according to typical Teradata Database workload demands in order to avoid CPU starvation. If these guidelines are not followed, or customer workload is unusually heavy with respect to I/O, it is possible that CPU starvation may still occur, as reflected by high I/O WAIT on the system.

If I/O wait is rising while CPU busy is falling, as shown by ResUsage data, this is an indication that the system is not able to use as much CPU as before because I/O is becoming the bottleneck.

If the onset of high I/O wait is sudden:

1 Determine if the high I/O wait is due to disk I/O, to waiting for AMPs from other nodes (because of skewing or coexistence imbalance), to low system demand, or (least likely) to BYNET I/O.

CPU+WIO less than 90% may suggest low system demand without a true I/O bottleneck. Look at node efficiency to determine if the I/O wait is due to node waiting.

2 Look at the actual disk I/O wait queue using sar –d, and examine:

• avwait

Average time in milliseconds that transfer requests wait idly on queue for response (in the FibreChannel driver queue or in the disk's queue).

• avserv

IF … THEN …

dbschk is not already running as a background task

run dbschk interactively to check current Teradata Database response time.

the dbschk log, or current display, shows a slow response or time-out

run syscheck to obtain a report showing any attribute that falls below the specified danger level.

no attribute is reported as being at the WARN level

check disk and AMP CPU usage.

198 Performance Management

Page 199: Teradata Database Performance Management

Chapter 18: Troubleshooting Teradata Database PerformanceLooking for the Bottleneck in Peak Utilization Periods

Average time to be serviced (which for disks includes seek, rotational latency and data transfer times).

If I/O Wait and the disk I/O queue length are both increasing, it indicates a true I/O bottleneck.

Looking for the Bottleneck in Peak Utilization Periods

Once you have identified the peak period, you can look for the bottleneck. Examining ResUsage data is one technique helpful in finding bottlenecks.

After you locate the bottleneck, you can make informed decisions on how best to alleviate the problem.

Some possible bottlenecks are:

• CPU saturation

• Disk saturation

• Free and/or FSG cache memory

• BYNET

• vprocs (hot AMPs, coexistence, load balancing)

• Channel (number of sessions or channel speed)

• LAN/gateway (number of sessions or network connections)

• Lock contention

Job Scheduling Around Peak Utilization

Rescheduling Jobs

Once you determine your peak system utilization times, you can recommend that some jobs be moved to other time slots.

For example, if peak periods are 9 A.M. to 5 P.M., you might want to schedule batch and load jobs overnight to reserve peak daytime hours for DSS queries or HATP (High Availability Transaction Processing).

Bound Jobs

Since some jobs tend to be CPU-bound and others I/O- bound, it is a good idea to determine which jobs fit into which category. You can determine this by means of AMPUsage data analysis.

Knowing which jobs are CPU-bound and which are I/O-bound enables you to recommended scheduling a CPU-bound job with an I/O bound job so that the resource under utilized by one job can be used more fully by the other.

Performance Management 199

Page 200: Teradata Database Performance Management

Chapter 18: Troubleshooting Teradata Database PerformancePreventing Slow or Hang Events

Teradata ASM and Concurrency

Teradata ASM throttle rules can be useful in controlling the concurrency of certain types of queries during peak utilization times.

In addition, Teradata ASM filter rules can prevent queries with certain characteristics from even starting to execute during specific windows of time. This can help keep utilization levels under control at times of high contention.

Managing I/O-Intensive Workloads

Below are suggestions for balancing resource usage when the system is I/O-bound:

• Identify I/O-intensive portions of the total work using AMPUsage reports and DBQL.

• Balance query activity during the day if I/O-Intensive work can be re-scheduled to off-hours.

• Look for query or database tuning opportunities, including:

• Collecting / refreshing statistics on all join and selection columns

• Adding indexes, join indexes, or sparse indexes

• Using MVC to reduce row size and in this way get more rows per block

• Increasing the number of Cylinder Read slots at least as high as the default setting for the platform.

• Using PP)

• Increasing block sizes

• Using 3NF data model to obtain narrower rows, more rows / block, fewer I/Os, and then denormalizing as needed

• Increasing node memory, in order to expand the size of the FSG cache

Preventing Slow or Hang Events

Problem detection includes monitoring processing as well as looking at how often a resource is in use during a given period. This could include asking questions such as:

• Is one query consistently blocking others?

• Are there many transaction deadlocks during peak workloads?

• Are all AMPs working equally hard?

• What are the disk loads and I/O counts?

• Is traffic on the BYNET moving normally?

• the hardware status of your system to determine whether the BYNET or an AMP or PE is down,

which could cause your job to appear hung or to run slowly.

Handling Blocked Internal DBQL Requests

DBQL uses express requests to do internal work on DBQL log tables. If you are performing work on the same log tables, such as deleting rows, this may block DBQL internal requests

200 Performance Management

Page 201: Teradata Database Performance Management

Chapter 18: Troubleshooting Teradata Database PerformancePreventing Slow or Hang Events

because you have a lock on the table. The internal requests will then consume AMP worker tasks until your work finishes or until you disable the DBQL logging.

Note: You cannot abort or otherwise do anything to this internal session record. It exists only to show that internal DBQL requests are blocked.

If DBQL internal requests remain blocked, the system deadlock detection may abort them. If the system aborts these internal sessions, the logging data will be lost. You must submit the appropriate END QUERY LOGGING statement to end logging before you do any kind of maintenance on DBQL tables.

If you notice frequently blocked DBQL internal requests reported by the internal session, consider scheduling your queries (such as table maintenance) on DBQL tables when there is no logging activity. Or use locks that hold the logging tables for a minimal amount of time.

Canceling Rollbacks

When a transaction fails to complete because the database restarts or the job of the transaction has failed, the system must remove any partially completed database updates to assure data integrity.

However, rollbacks can involve a very large number of rows and can also lock affected tables until the rollback completes, which may take a long time.

If you decide that restoring specific tables would be quicker than waiting for a rollback to complete, cancel the rollback using the Recovery Manager utility.

Note: You must first list rollback tables using the LIST ROLLBACK TABLES command and then cancel rollback from that list within the same Recovery Manager session.

For more information Recovery Manage, see Utilities.

Noticing a Slow Rollback of an ALTER TABLE Statement

Because Teradata ensures that all transactions must succeed or fail in their entirety, the system does not allow partial results. In most cases, if a transaction fails, Teradata rolls back all the previous actions of a transaction. This requires that the system logs the before state of every action so that the action can, if necessary, be undone later. For instance, when you delete a row, Teradata must first log a copy of the row, so that it can be restored if the transaction fails.

Sometimes, however, logging is extremely expensive compared to the cost of the action: dropping a large table, for example. If the system does not have to roll the table back later, the system may simply release all table data entirely, deallocating whole cylinders with a single action. In this case, Teradata delays performing the action until it is certain that the transaction will succeed. In effect, Teradata commits the transaction before executing the deferred action.

During this special post-commit phase, the transaction is still working and holding locks, but it cannot be aborted (since it is already committed). In most cases, this post-commit phase finishes quickly and is unnoticeable.

The exception is the ALTER TABLE statement. It may take longer when used to add or remove columns of a table. ALTER TABLE requires that every row in the table be rebuilt and is,

Performance Management 201

Page 202: Teradata Database Performance Management

Chapter 18: Troubleshooting Teradata Database PerformanceSkewed Queries

therefore, an expensive operation. It can take hours on a large table, even with the highly optimized, block-oriented method that Teradata uses.

When you notice that a rollback of an ALTER TABLE statement is taking a very long time but you cannot abort it, the system is deferring an abort and will need to complete it.

Skewed Queries

A skewed query is one that does not take full advantage of the parallelism of Teradata Database. Instead, it concentrates work on a small number of AMPs. Because these AMPs are busy working on the skewed query, they do not have as much time to work on other more parallel queries, causing the remaining AMPs to be under utilized while they wait for the hot AMPs to finish. By locating and correcting the source of skewed resource utilization, you can improve the overall system performance.

Tables Prone to Skewing

The following tables should be carefully managed because they are prone to skewing.

• Journal tables. Journal space (spool and other transient storage) typically comes from free space. A common practice is to allocate a database with a certain amount of PERM, but never use it. This ensures that there is free space for journals and so on. When your journal tables are skewed, you get an error when running queries, letting you know the system needs the journal space but cannot obtain it.

• The DBC.AccessRights table is notorious for becoming skewed. If it grows too large, it will affect the space limits dramatically. To reduce this skewing, either clean up DBC.AccessRights by removing redundant rows or convert to using roles and then remove the now redundant rows.

• DBC.AccLogTbl table. Teradata Database generates at least one entry every time a user defined for logging attempts to access an object. This table can grow very large and consume a lot of disk space.

• Resource usage logging tables or query logging tables. Consider daily offloading data from the actual DBC tables into temporary staging tables and ultimately history tables for analysis or archive.

Note: Perform maintenance when the system is most quiescent to avoid performance impact. Cleaning certain tables, such as AccLogTbl would lock the table and prevent access to the system. You may need to set up a maintenance window to clean up AccLogTbl.

Every table in the system, however, is prone to becoming skewed if the index is poorly designed or if statistics have not been collected and the Optimizer is using extremely stale data. A usual tell-tale sign is if one or more queries are doing a lot of maintenance on typically a single AMP. If this happens, you need to track that query down and handle it like any other skewed processing.

202 Performance Management

Page 203: Teradata Database Performance Management

Chapter 18: Troubleshooting Teradata Database PerformanceControlling Session Elements

Possible Causes of Skewing

The following table provides suggestions to help you determine possible causes of skewing.

Controlling Session Elements

All database systems reach saturation from time to time, particularly in ad hoc query environments. On Teradata Database, however, you can control session elements such as user spool space and job entry, and thus you can minimize how often end-users might saturate the database capabilities.

Teradata Database provides several tools with which you can control these elements. The most commonly used are introduced in the following table.

Problem Action

If the system is CPU-bound or disk-bound, or the processing is skewed.

• Look at physical resource utilization:

• Use Teradata Viewpoint. Monitor physical resources to view CPU/disk usage by node. Monitor virtual resources to view CPU/disk usage by vproc.

• Use sar -u to view CPU usage.

• Use sar -d to view disk usage.

• Look at DBC.ResusageSpma to view CPU, disk, and BYNET usage.

• Look at workload utilization:

Use DBQL to find the user with the skewed processing.

• Look at data distribution:

Use DBC.TableSizeV to identify tables with skewed data distribution.

If the processing is skewed, identify the user.

• Use Teradata Viewpoint.

• Check DBC.SessionInfoV or run Query Session to identify the current user.

• Check DBC.LogOnOffV to view historical users.

• Check DBC.AMPUsage to identify the heavy resource user.

• Check DBC.DiskSpaceV to identify large spool user.

If the processing is skewed, get information on the problematic query.

• To identify the query, ask the user or check query log.

• Run EXPLAIN on the query to determine if a bad join plan exists. Look for unexpected product joins. Verify that large tables are not joined before small tables. Look for estimates that are too high or too low.

Performance Management 203

Page 204: Teradata Database Performance Management

Chapter 18: Troubleshooting Teradata Database PerformanceIdentifying and Handling Resource-Intensive Queries in Real Time

Identifying and Handling Resource-Intensive Queries in Real Time

Characteristics of a Resource-Intensive Query

The following are characteristics of a resource-intensive query (sometimes called a “killer” query):

• High CPU

IF you want to … THEN use one or more of these tools … For details and instructions, see …

control logon access • Teradata Viewpoint to control access to objects as well as active sessions by user, account, and WD

• Logon controls such as passwords, settings in profiles, or assignments to specific roles

• Host group IDs, to authorize logons from specific client platforms with GRANT/REVOKE LOGON ... host_groupid

• Teradata Viewpoint HELP

• Database Administration

• “LOGON Controls” in Security Administration

• “GRANT LOGON” in SQL Data Control Language

• Teradata Director Program Reference

control object access • Teradata Viewpoint to:

• Control access to database objects.

• Limit parameters (such as response rows) based on query type.

• Limit the number of active queries by user, account and WD.

• User spool space, to limit response sizes

• User, role, and/or object access privileges with GRANT/REVOKE

• Implement operations so that users access portions of data through views, macros, and stored procedures

• Teradata Viewpoint HELP

• Database Administration

• “GRANT” statement in SQL Data Control Language

• Security Administration

• Teradata Director Program Reference

manage access to resources

• Teradata Viewpoint, based on concurrent sessions, query type, WD, quantity of response rows, and/or workload flow.

• Priority Scheduler to schedule priority of account access to resources such as CPU and memory.

• Teradata Viewpoint HELP

• “Priority Scheduler (schmon)” in Utilities

• ACCOUNT keyword under “CREATE USER” in SQL Data Definition Language

determine if an upgrade or expansion of your Teradata Database would help avoid future resource saturation

• Baseline profiling comparisons

• RCT

• ResUsage reports

• Chapter 16: “Baseline Benchmark Testing”

• “Solving Bottlenecks by Expanding Your Teradata Database Configuration” on page 211

• Resource Usage Macros and Tables

204 Performance Management

Page 205: Teradata Database Performance Management

Chapter 18: Troubleshooting Teradata Database PerformanceIdentifying and Handling Resource-Intensive Queries in Real Time

In general, you can identify the profile for a bad query by looking at the ratio between high CPU use relative to I/O access. If for a query the ratio of CPU milliseconds consumed divided by the number of I/Os performed is > 100, then the query is a resource-intensive query.

A product join, for example, with high numbers of actual rows in both the left and right tables might become a resource-intensive query.

• Hot / Skewed Spool

Queries that require lots of spool space have a tendency to run out of spool on one AMP before they can finish (or run forever). If users have very high individual spool space limits, this can cause all free spool on a single AMP to be exhausted for everyone else as well.

In a case where CPU busy is very high or at maximum capacity, hot spool queries will also cause overall degradation because the AMP is likely hot also.

Execution Plans for Resource-Intensive Queries

Resource-intensive queries can have a bad execution plan for a number of reasons, such as:

• Stale statistics

• No statistics on a skewed, highly nonunique index

• Poor choice of a PI

• Poorly written SQL, particularly with missing join constraints

Ensuring Parallel Node Efficiency

Seeing poor parallel node efficiency is not a matter of having a heavy workload. Parallel node efficiency is a matter of how evenly the workload is shared among nodes. The more evenly the nodes are shared, the higher the parallel efficiency.

Parallel node efficiency is calculated by dividing average node utilization by maximum node utilization. The result illustrates your node workload distribution, as follows:

Possible causes of poor parallel node efficiency include:

• Down node

• Non-Teradata Database application running on a TPA node

• Co-existence system where the number of AMPs per node is not optimally balanced with different node powers.

• Poor AMP efficiency

IF parallel node efficiency … THEN …

is nearly 100% the better the nodes are working together.

falls significantly below 100% in that time period, one or a few nodes are working harder than the others.

Performance Management 205

Page 206: Teradata Database Performance Management

Chapter 18: Troubleshooting Teradata Database PerformanceIdentifying and Handling Resource-Intensive Queries in Real Time

AMP Efficiency and Balanced Data

AMP vprocs always run in parallel, but the way data rows or column values are striped across the disks affect the parallel operation of AMP step processing.

Unbalanced, or skewed, or spiked, disk loads can cause one or a few AMPs to be doing most of the I/Os. For example, when a numeric column allows zeros and/or nulls, the majority of rows might hash to the same AMP.

If your disk loads are poorly balanced, discuss with operations ways to correct the situation. For example:

• Perhaps queries or views against a column with zero/null values could use “WHERE NOT NULL” or “NOT= 0” qualifiers.

• If the cause is a NUPI, consider redefining the index, especially for a very large table, to achieve a higher percentage of uniqueness.

Tools for Analyzing Lock Problems

The following table provides suggestions for analyzing and solving lock problems. For additional information, see Chapter 8: “Database Locks and Performance.”

falls below 60% more often than a couple of sampling periods

your installation is not taking advantage of the parallel architecture.

IF parallel node efficiency … THEN …

Tool Analysis Solution

Lock Display Transaction locks Determine which session is holding the lock that blocks others.

Query Session Blocked session Abort the session causing blocked transactions.

Show Locks Host utility (HUT) locks; that is, locks placed by a client-based utility, such as Archive and Recovery

Submit RELEASE LOCKS as either an Archive and Recovery command or an SQL statement.

206 Performance Management

Page 207: Teradata Database Performance Management

APPENDIX A Performance and DatabaseRedesign

This appendix discusses revising database design to improve performance.

Database Design Phases

The following diagram illustrates the phases of database design.

KY01A003

Planning

ApplicationTransaction

Modeling

DatabaseModeling

PhysicalDatabase

Design

OperationalAnalysis

Build

Implementation

Performance Management 207

Page 208: Teradata Database Performance Management

Appendix A: Performance and Database RedesignDatabase Design Phases

After Initial Design Implementation

After initial database design implementation, you may experience:

• Expanded or additional applications

• Different data demographics

• Different processing windows

• Different transaction frequencies

• Different access paths

Adding New Applications

Many database designs are denormalized to support one or two particular applications. When you add new applications, the denormalization that helped the performance of earlier applications may actually hinder the performance of later applications.

If you add new applications, or if you gather any other information during the design phase changes, you may need to revert to the Application Transaction Modeling phase of database design and proceed from there. Reiteration is a necessary fact of life in system design and development.

Shortcut Rules

At times you may need to take shortcuts. Either you have inherited a poor design, or the requirements have changed and you have only a short time frame in which to make performance improvements. In these cases, the following basic rules may help:

• You can improve Decision Support System (DSS) performance by reconsidering PIs to improve join performance for large tables. Tactical query performance may also improve if PI selection is redefined in a way that supports single-AMP joins between two tables.

• Because common join columns usually make good PIs, making it possible for the system to perform PI-based merge joins, choose common join column as PIs. For multicolumn joins, a good PI may be all join columns or some subset of the join columns.

• To minimize searching through nonqualifying columns, choose PIs through the most frequent table access path.

• Single table join indexes may be useful for tactical queries because such structures offer an alternative PI access to the base table data.

• Common selection columns make good PIs, especially if one column also serves as a join column.

Common selection columns usually make good SIs for DSS performance. For multicolumn accesses, a good SI may be all access columns or some subset of the access columns.

208 Performance Management

Page 209: Teradata Database Performance Management

Appendix A: Performance and Database RedesignDatabase Design Phases

Caveats

Be aware of the following caveats if you decide to change your PIs based on join requirements:

• If you include too many columns in the PI, queries that do not specify ALL columns will not access the table via the PI but rather via a scan, eliminating PI access or merge join from the Optimizer plan. Alternative access or joins for big tables are expensive.

• If you include too few columns in the PI, each PI value may correspond to thousands of rows. This may cause data skewing, as well as performance degradation, on operations (such as data maintenance) that must touch all rows in a row hash.

In order to avoid the overhead of probing each partition when doing PI access to PPI tables:

• Consider including the partitioning column(s) in the PI definition

• If that is not possible then

• Define a USI or NUSI on the PI column

• Build a single-table join index with the PI column(s) as the PPI table

• Consider fewer partitions (hundreds or less, not thousands) in the PPI table

Performance Management 209

Page 210: Teradata Database Performance Management

Appendix A: Performance and Database RedesignDatabase Design Phases

210 Performance Management

Page 211: Teradata Database Performance Management

APPENDIX B Performance and CapacityPlanning

This appendix discusses system performance and capacity planning

Solving Bottlenecks by Expanding Your Teradata Database Configuration

System saturation and bottleneck identification are interrelated. When your Teradata Database is saturated, the bottleneck is usually some key resource, such as a CPU or disk. Use the information obtained from performance monitoring, resource usage, and query capture and process tracking tools to find the cause of repeated bottlenecks.

If a resource has been a bottleneck consistently during peak utilization periods and you have determined that your database design, data modeling, and query structures are efficient, consider expanding your Teradata Database configuration to improve performance.

Expansion involves adding any combination of disk arrays, memory, vprocs, or nodes (with BYNETs), and then running the Parallel Upgrade Tool (PUT) and Configuration and Reconfiguration utilities.

The Reconfiguration Estimator utility can provide an estimate of the duration of outage based on parameters you supply interactively.

Determining Resource Needs

When planning expansion, you need to determine whether your configuration needs more memory, a more powerful or additional processors, more nodes, more disks, more disk array controllers, or additional options for OLTP environments.

To do this, analyze the performance information on AMPs and PEs, including:

• System usage

• Resource usage across AMPs and PEs

• The amount of disk I/O, BYNET traffic, and client I/O that is occurring

• Whether congestion or excessive swapping is a problem on any AMP or PE

Often you can satisfy the need for increased capacity or throughput by adding disks, memory, or vprocs. If that does not suffice, you can add nodes. In particular, you must add nodes when your system is CPU bound.

Performance Management 211

Page 212: Teradata Database Performance Management

Appendix B: Performance and Capacity PlanningSolving Bottlenecks by Expanding Your Teradata Database Configuration

The Reconfiguration utility can provide an estimate of the duration of outage based on parameters you supply interactively. For information on the Reconfiguration Estimator, see Utilities.

Note: Make sure your applications are scalable and can take best advantage of the expansion. For guidelines, see “Scaling Your Applications” on page 214.

Adding Disk Arrays

When you add disk arrays, you increase the capacity for disk I/O.

This is helpful both for a DSS environment with a growing database, and for an OLTP environment with increasing concurrent disk I/O.

To determine if the system needs more storage, look at the ResUsageSvpr table for unusual disk activity, such as frequent:

• MiniCylPacks

• Defrags

• Packdisks

You may need to add more storage capacity to existing nodes when:

• Excessive disk activity is impacting performance

• Application changes require additional spool space

• Database growth requires additional storage

When you add disks, you run utilities to slice and assign them automatically.

Note: Teradata recommends that you increase memory when you add AMPs.

Adding Vprocs

The following table lists reasons for when to add vprocs.

IF you want to increase … THEN add …

storage capacity disks, probably with AMPs, and perhaps nodes. When you add storage, you normally add AMPs.

The number of AMPs you add depends on the number of ranks assigned to the existing AMPs. For example, if you add two disk cabinets to each of 20 ranks, you would add 10-20 more AMPs to your configuration. If you add AMPs, be sure your memory is adequate. Normally, you should add memory when you add AMPs.

If your existing CPUs cannot handle the load caused by the additional AMPs, you also need to consider adding nodes.

212 Performance Management

Page 213: Teradata Database Performance Management

Appendix B: Performance and Capacity PlanningSolving Bottlenecks by Expanding Your Teradata Database Configuration

Adding Memory

When you add memory, you increase the cache to maximize the capability of the CPUs. This is helpful when CPUs are processing faster than the disk contents can be read into memory (that is, the system is I/O-bound).

The following table lists conditions for when to add more memory.

single-user response time

AMPs.

• Optimization of single-user throughput or response time is especially significant when there are fewer AMPs than CPUs per node.

• In a concurrent workload, you might be able to achieve the desired throughput by adding more AMPs to existing nodes.

• Be sure your memory is adequate for the additional AMP workload.

capacity for concurrent sessions

PEs, perhaps nodes, and perhaps a channel connection.

• For network sessions, add PEs if you have fewer than 10 PEs per node. If you already have 10 PEs per node, add another node.

Note: The session limit is 120 for each PE and 1200 for each gateway. Because each node runs just one instance of the gateway, more than 10 PEs per node does not provide increased network sessions.

• The channel driver does not have a session limit, but normally one PE is configured for each channel connection.

If your configuration has less than 4 channel connections, you can add another channel connection and another PE.

• Verify that there is enough CPU capacity to handle more PEs. If there is not, add nodes.

IF you want to increase … THEN add …

Condition Description

Add vprocs to existing nodes

Each vproc consumes 32 MB of memory. When you add vprocs to existing nodes, you probably should add memory.

Additional vprocs can substantially reduce free memory, which can cause moreI/Os because the system can cache fewer datablocks.

Excessive paging/swapping (thrashing)

More memory means that more code and data can be cached, achieving less I/O for paging and swapping.

Tables in memory

Increased memory may reduce I/O by accommodating:

• Tables that are currently too large to remain in memory during processing

• More small tables concurrently residing in memory during processing

I/Os can be affected by the size of table datablocks. For information, see the DATABLOCKSIZE option of the CREATE TABLE statement in SQL Data Definition Language.

Performance Management 213

Page 214: Teradata Database Performance Management

Appendix B: Performance and Capacity PlanningSolving Bottlenecks by Expanding Your Teradata Database Configuration

Adding Nodes

Often you can satisfy the need for increased capacity or throughput by adding disk arrays, memory, or vprocs. If that does not suffice, you can add one or more nodes.

Although adding a new node costs more than adding disks and memory to an existing node, you must add nodes when your system is CPU bound.

For example, if the configuration is adequate from an I/O perspective but the CPU is at maximum capacity, adding a node tends to alleviate the problem.

If you need more nodes but not storage space, determine if you should add nodes:

• To existing cliques to share existing disk arrays.

• Plus disk arrays to maintain the current ratio.

• Plus AMPs and redistribute data to reduce the amount of storage managed by each AMP.

Reconfiguring Your Teradata Database

When you add nodes, AMPs, or disks to your configuration, you must reconfigure the TPA. The Reconfiguration utility can do most things automatically, such as partitioning the disks and redistributing your data rows.

However, it is good practice to always first archive your data. Also, some types of expansion, such as adding a disk array, still require row redistribution. So if your tables are very large, it may be faster to archive and restore.

You can use the Reconfiguration Estimator utility to obtain an estimate of elapsed time needed for reconfiguration, based on the number and size of the data tables on your current system.

The Reconfiguration Estimator prompts you for information about the planned upgrade and provides estimates for the following phases:

• Redistribution

• Deletion

• Rebuilding SIs

If you have questions about possible procedures or the reported time estimates, contact the Teradata Support Center. For an overview of the utilities, issues, and procedures involved in a TPA reconfiguration, see Utilities.

Scaling Your Applications

Your user applications should be scalable.

For example, assume that table BigT is a very large table but its rows are hash distributed based on only 16 unique values. Applications using BigT perform well and with high parallelism on a system with 1 or 2 nodes and 8 to 16 AMPs.

If you then expand the system to 128 AMPs, the rows of BigT still hash to only 16 unique values, and so are still distributed among only 16 AMPs. Thus, applications do not perform any better, and perhaps not as well.

214 Performance Management

Page 215: Teradata Database Performance Management

Appendix B: Performance and Capacity PlanningPerformance Considerations When Upgrading

To ensure scalability of your applications, try to make your PI a unique or nearly unique index. If a single column does not provide uniqueness, combine two or more columns to make up the index. You can define up to 64 columns per PI.

The result is many more unique hash combinations, and your applications should continue to perform well as your system expands.

Performance Considerations When Upgrading

At some point, your system will exhaust all its resources, and performance tuning and workload management can no longer respond to the impacts of growth. At that point, you will need to upgrade your system.

Recommendations

When upgrading, it is best to monitor your system to make sure the upgrade is delivering as expected. In order to achieve that, Teradata recommends the following tasks be performed on the production system

• Recollect statistics where possible.

• Create a baseline test suite.

• Run the baseline on the production system before the upgrade.

• Save the EXPLAINs for the base queries.

• Run the test bed again after the upgrade, and get new EXPLAINs.

• Compare the two EXPLAINS.

• Check for faster/slower response on any of the test queries. If slower, look at the before and after EXPLAIN.

• Check statistics.

• If stale, recollect.

• If not, report this as a performance regression problem to Teradata Support immediately.

Performing these simple activities, even with a small group of queries (5-10 minimum are recommended), will help validate your performance results and expectations following any sort of important system change.

Performance Management 215

Page 216: Teradata Database Performance Management

Appendix B: Performance and Capacity PlanningPerformance Considerations When Upgrading

216 Performance Management

Page 217: Teradata Database Performance Management

APPENDIX C Performance Tools and Resources

This appendix lists performance monitoring tools and resources. It also explains the place of system components in tuning and monitoring performance.

Performance Monitoring Tools and Resources

You can use the following Teradata Database performance monitoring tools and resources to analyze and improve system performance.

Tool Description For more details, see...

Access controls • Use the REVOKE statement to revoke an explicitly granted privilege from the specified user.

• Use the REVOKE LOGON statement to revoke access from one or more specified client platforms.

SQL Data Control Language

Security Administration

Access Logging Suite of tools enabling C2 security that log the results of checks for access privileges.

Log information can include user or session, privilege checked, and database entity.

Note: This feature incurs processing overhead but can be useful for identifying bottlenecks or lock conflicts.

Security Administration

AMP Load (ampload)

Displays the load on all AMPs in a system, including the number of AMP worker tasks (AWTs) available for each AMP and the number of messages waiting (message queue length) on each AMP.

Utilities

AWT Monitor (awtmon)

Collects and displays a user-friendly summary of the AWT in-use count snapshots for the local node and all nodes in Teradata Database.

Utilities

CheckTable (checktable)

Checks for inconsistencies between internal data structures such as table headers, row identifiers, and second indexes,

Utilities

Control GDO Editor (ctl))

Displays the fields of the PDE Control Parameters GDO and allows modification of the settings.

Note: Runs only under Windows and Linux.

Utilities

Ctl Utility (ctl) Displays the fields of the PDE Control Parameters GDO and allows modification of the settlings.

Note: Runs only under Linux and Windows

Utilities

Performance Management 217

Page 218: Teradata Database Performance Management

Appendix C: Performance Tools and ResourcesPerformance Monitoring Tools and Resources

Database Window (DBW)

GUI (Graphical User Interface) that runs on the AWS (Administration Workstation) available with MPP platforms.

With DBW, you can execute commands via the Console Subsystem, view the status of AMPs and disks, and otherwise run other administrative tools for your Teradata Database.

Utilities

DBC.AMPUsage System view that shows AMP usage by user and account.

Use this view to determine which users are submitting jobs that are CPU- or I/O intensive; for example:

.set retlimit 10select username,processor,sum

(cputime),sum(diskio)from dbc.ampusagewhere processor ='1-0'order by 2,3 descgroup by 1,2;

Data Dictionary

DBC.DiskSpaceV System view that provides information by AMP on disk space usage, including permanent and spool data by database or account for each AMP. Use this view to track large spool usage.

Data Dictionary

DBC.LogOnOffV System view that provides historical information about logon and logoff activity. Use this view to identify users logged on at some time in the past.

Data Dictionary

DBC.SessionInfoV System view that provides information about users who are currently logged on. Use this view to determine the logon times of current sessions.

Data Dictionary

DBC.Software_Event_Log

System view that provides information about hardware and software failures and severity, diagnostic information, and so on. Use this view to identify errors or system failures.

Data Dictionary

DBC.TableSizeV System view that provides information about disk space usage (excluding spool), by database and table, for each AMP.

You can query this view to identify tables with skewed data distribution; for example:

select AMP, CurrentPermfrom DBC.TableSizeVwhere DatabaseName =

'databasename'and TableName = 'tablename' order by 1;

Data Dictionary

DBS Control (dbscontrol)

Displays and modifies the tunable parameters in the DBS Control Record.

Utilities

DEFRAGMENT Ferret command that combines free sectors on a cylinder. Utilities

Database Initialization Program (DIP)

Executes one or more of the standard DIP scripts packaged with Teradata Database.

These scripts create a variety of database objects that can extend the functionality of Teradata Database with addition, optional features.

Utilities

Tool Description For more details, see...

218 Performance Management

Page 219: Teradata Database Performance Management

Appendix C: Performance Tools and ResourcesPerformance Monitoring Tools and Resources

DIPVIEWS DIPVIEWS script file that contains the SQL definitions for the system views.

Execute this script to create the views that provide access to commonly referenced information in the underlying system tables.

Data Dictionary

EXPLAIN SQL modifier that allows you to see the steps and access path that would be used to process an SQL statement.

EXPLAIN is useful for constructing complex queries, especially those planned for repetition and for evaluating the syntax of slow or problem queries.

SQL Data Manipulation Language

Lock Display Displays active transaction locks that control concurrent processing and that can be applied to the rowhash level. You can specify the display by:

• A specific AMP

• A group of AMPs

• All AMPs

Useful for finding a session that is blocking others, particularly when you need to break a deadlock.

Utilities

PACKDISK Ferret command that moves datablocks to create free space for cylinders.

Utilities

Priority Scheduler (schmon)

Utility based on weighted priorities that lets you fine-tune the allocation of CPU resources among concurrent sessions.

Utilities

Query Session (qrysessn)

Utility that provides information on the processing state for both SQL and utility sessions.

Utilities

Recovery Manager (rcvmanager)

Displays information used to monitor progress of a Teradata Database recovery.

Utilities

Resource Check Tools (dbschk, nodecheck, syscheck)

Tools used to detect systems slowdowns or hangs. Utilities

ResUsage Macros, tables, and views that monitor, record, and report the use of Teradata Database resources, including CPU and I/O consumption, during transaction processing.

Resource Usage Macros and Tables

sar Tool that monitors and records the use of system resources. Resource Usage Macros and Tables

SCANDISK Ferret command that checks File System integrity (integrity of internal file system data structures, such as the master index, the cylinder index, and datablocks, including those associated with the WAL log).

Utilities

Database Administration

Show Locks (showlocks)

Displays locks placed by Archive and Recovery and Table Rebuild operations on databases and tables.

Teradata Archive/Recovery Utility Reference

Utilities

Tool Description For more details, see...

Performance Management 219

Page 220: Teradata Database Performance Management

Appendix C: Performance Tools and ResourcesSystem Components and Performance Monitoring

System Components and Performance Monitoring

The following table explains the place of system components in monitoring and tuning performance.

SHOWSPACE Ferret command that displays disk utilization and available free cylinders.

Utilities

stune file File in which you can modify the LOTSFREE, DESFREE, and MINFREE parameters.

Database Administration

TOP Node-level performance tool that displays the top 15 processes by percentage of CPU usage.

“TOP” on page 190

TPCCONS (Two-Phase Commit Console Interface)

Console interface you can use to investigate and resolve Two-Phase Commit (2PC) in-doubt transactions.

Utilities

Teradata Viewpoint

Teradata Viewpoint, a web-based application, enables you to monitor and manage Teradata Database system performance.

Viewpoint provides both historical views of system capacity and use and current views of work and system throughput

Viewpoint HELP

Vproc Manager (vprocmanager)

Manages the vprocs.

For example, obtains status of specific vprocs, initializes vprocs, forces a vproc to restart, and forces a Teradata Database restart.

Utilities

Tool Description For more details, see...

System Component Performance Considerations

AMPs An AMP vproc, which executes I/O tasks, should be balanced to have the same processing power, memory allotment, and I/O capacity.

Banyan Network (BYNET)

hardware/software

The BYNET, a high-speed node interconnect designed for optimum Massively Parallel Processing (MPP) operation, increases Teradata Database availability and performance with:

• Wide bandwidth and quad redundancy

• Dynamic, automatic load balancing of node traffic and vproc reconfiguration

• Broadcast or point-to-point communication, as appropriate for fastest query performance

Channel connection

A channel connects a Teradata Database platform to a mainframe host. A separate channel is needed for each host.

You should have multiple connections per channel for redundancy.

Channel driver Each node can accommodate one channel driver. The driver has no session limit.

220 Performance Management

Page 221: Teradata Database Performance Management

Appendix C: Performance Tools and ResourcesSystem Components and Performance Monitoring

CPUs Because all servers used with Teradata Database have multiple Central Processing Units (CPUs), you should know the number of CPUs and their speeds.

Gateway Because the gateway has a limit of 1200 sessions per node, be sure that the session limit can accommodate the number of PEs on that node.

LAN connection

You must have a separate Local Area Network (LAN) connection for each logical LAN.

A group of LAN PEs service requests for the LANs connected to your database.

Memory Increased memory often improves performance for certain applications.

Always ensure that memory is adequate when increasing the number of vprocs. Prepare for such contingencies as vprocs migrating when a node is down in a clique.

PEs 120 concurrent sessions can be supported per PE.

You can, therefore, increase the number of sessions that run concurrently by adding PEs to a single node. Be sure, however, that the gateway on each node affected can support the total number of sessions.

storage If your database is out of space, you need more storage. You can archive and delete data from existing storage or add disk arrays.

System disks In addition to the operating system and your applications, system disks store Teradata Database executables. Teradata Database also obtains dump and swap space from the system disks. Be sure to plan for this consumption.

Vprocs Virtual Processors (vprocs), which include Access Module Processor (AMP) and Parsing Engine (PE) vprocs, should be balanced to have the same processing power, memory allotment, and I/O capacity.

System Component Performance Considerations

Performance Management 221

Page 222: Teradata Database Performance Management

Appendix C: Performance Tools and ResourcesSystem Components and Performance Monitoring

222 Performance Management

Page 223: Teradata Database Performance Management

Glossary

2PC Two-Phase Commit

ACP AutoCylinderPack

AG Allocation Group

ALC Algorithmic Level Compression

AMP Access Module Process

API Application Programming Interface

ANSI American National Standards Institute

ARC Archive Recovery

ASE Account String Expansion

AWS Administration Workstation

AWT AMP Worker Task

BLC Block Level Compression

BLMSTAT BYNET Link Manager Status

BT Begin Transaction

BTEQ Basic Teradata Query

BTEQWIN BTEQ Window

BYNET Banyan Network

CI Cylinder Index

CLI Call-Level Interface

CPU Central Processing Unit

DBA Database Administrator

DBQL Database Query Log

DBW Database Window

DDL Data Definition Language

DIP Database Initialization Program

DML Data Manipulation Language

Performance Management 223

Page 224: Teradata Database Performance Management

Glossary

DSS Decision Support Systems

ET End Transaction

FIB File Information Block

FK Foreign Key

FSG File System Segment

FSP FreeSpacePercent

GDO Globally Distributed Object

GMT Greenwich Mean Time

GSS Global Sales Support

GUI Graphical User Interface

HATP High Availability Transaction Processing

HI Hash Index

HSI Host System Interface

HUT Host Utility

I/O Input/Output

JDBC Java Database Connectivity

JI Join Index

LAN Local Area Network

LDV Logical Device

LOB Large Object

LSN Logon Sequence Number

LT/ST Large Table / Small Table

MI Master Index

MLPPI Multilevel Partitioned Primary Index

MPP Massively Parallel Processing

MVC Multi-Value Compression

NoPI No Primary Index

NUPI Nonunique Primary Index

NUSI Nonnique Secondary Index

224 Performance Management

Page 225: Teradata Database Performance Management

Glossary

ODBC Open Database Connectivity

PDE Parallel Database Extensions

PDT Period Data Type

PE Parsing Engine

PG Performance Group

PI Primary Index

PJ Permanent Journal

PK Primary Key

PM/API Performance Monitor/Application Programming Interface

PMPC Performance Monitor and Production Control

PPI Partitioned Primary Index

PS Priority Scheduler

PTP Point-to-point

PUT Parallel Upgrade Tool

QCD Query Capture Database

QCF Query Capture Facility

RAID Redundant Array of Independent Disks

RAM Random Access Memory

Resource Check Tools RCT

Resource Partition RP

ResUsage Resource Usage

RI Referential Integrity

SAR System Activity Reporter

SI Secondary Index

SMF System Management Facility

SMP Symmetric Multiprocessing

SNMP Simple Network Management Protocol

TDP Teradata Director Program

TDPUTCE TDP User Transaction Collection Exit

Performance Management 225

Page 226: Teradata Database Performance Management

Glossary

Teradata Active EDW Teradata Active Enterprise Data Warehouse

Teradata ASM Teradata Active System Management

Teradata SET Teradata System Emulation Test

TJ Transient Journal

TLE Target Level Emulation

TMSM Teradata Multisystem Management

TPA Trusted Parallel Application

TPCCONS Two-Phase Commit Console Interface

UDF User-Defined Function

UPI Unique Primary Index

USI Unique Secondary Index

VECOMP Visual Explain and Compare

vdisk Collection of cylinders currently allocated to an AMP

vproc Virtual Processor

WIO I/O Wait

XML Extensible Markup Language

z/OS IBM System z Operating System

226 Performance Management

Page 227: Teradata Database Performance Management

Index

Numerics2 PC protocol 114

AAccess controls 217Access Logging 217Access objects, tools for controlling 204Account string

ASE and 31performance groups and 32

Account String Expansion. See ASEAggregate cache size, increase in 75Aggregates on views 95ALTER TABLE statement

column compression 61data retrieval 59

AMP efficiency, balanced data and 206AMP sampling, kinds of 103ampload utility 191AMPs, adding 213AMPUsage

ASE, parameters 32data collection and 49ResUsage and 49

Analytical functions, performance and 68Array support 74ASE

account string and 31AMP performance and 31parameters, AMPUsage and 32PE performance and 31

AutoCylPack 132AWT Monitor utility 190

BBaseline profile, performance metrics for 186Baseline test suite, uses of 185Basic system management practices 21

activities supporting 23data monitoring and collection 23performance expectations 27performance reports 177teradata support center, accessibility to 28

BETWEEN clause, performance and 78Block

journal size 146maximum size 143minimum size 145splitting 144

Bottlenecks, solving chronic 211BSMP. See Basic system management practicesBTNET Link Manager Status utility 190BYNET data macro, ResUsage and 48

CCache hit rates, ResUsage and 42Canary query 50

production 51system 50

Capacity planning, ResUsage and 43CASE expression, performance and 65Channel connections, adding 213CheckTable utility 194CPU utilization, ResUsage and 44CREATE TABLE statement, data retrieval and 59ctl utility 188, 217Cylinder Read, FSG Cache, viewing 155Cylinder splits, FreeSpacePercent and 141Cylinders

data block allocation unit, minimum 145data block size, maximum 143data compression 141defragmentation 135disk space, adding 146free, running out of 132freeing

AutoCylPack 132MiniCylPack 134

freeing space on 136journal data block size 146

DDaily reports 181Data

parallel efficiency 126skewing 125, 126, 127uneven

Hash functions and 124identifying 123

Data compression 141Data distribution

Performance Management 227

Page 228: Teradata Database Performance Management

Index

aggregation and 126issues with respect to 123join processing and 126primary index and 127referential integrity and 127

Data space, data collecting 131Database design, performance and 207Database Initialization Program 218Database Query Log. See DBQLDatabase Window 218DATABLOCKSIZE, performance and 61DBC Control utility 218DBC.AMPUsage view 49, 218

ASE and 49ResUsage and 43

DBC.DiskSpace view 218DBC.LogOnOff view 218DBC.SessionInfo view 218DBC.Software_Event_Log view 218DBC.TableSize view 218DBQL 37

best practices, logging 37overview 37user types 37

Deadlocks, preventing excessive 118DefragLowCylProd 161DEFRAGMENT utility 218Defragmentation 135DictionaryCacheSize 161DIP. See Database Initialization ProgramDIPVIEWS 219Disk arrays,adding 212Disk space

adding 212running out of 131

Disk utilization and 47

EError logging 72Expanding system, determining needs for 211EXPLAIN statement 219

Optimizer and 108

FFerret utility, SHOWSPACE command 189FREESPACE, performance and 63FreeSpacePercent 161

cylinder splits and 141determining value for 137operations disregarding 139operations honoring 139PACKDISK utility 140

FSG Cache 154

calculating FSG Cache misses 155Cylinder Read and 155space in 155

FSG Cache percent 153, 155FSP. See FreeSpacePercent

GGateway Control utility 189GROUP BY, join optimization and 97

HHang events, preventing 200Hash bucked expansion 128Hash joins

costing 80dynamic 80performance and 80

Heartbeat query. See Canary queryHost trafic, ResUsage and 44HTMemAlloc 163

II/O bottlenecks, solutions to 153IdCol Batch Size 163Identity columns, performance and 113Index Wizard. See Teradata Index WizardIN-List value limit 77INSERT. . . SELECT statement

error logging and 72performance and 72

Iterated requests, support for 74

JJob scheduling

peak utilization and 199tools for setting up automatically 204

Jobs, bound, performance and 199Join index

NUSI and 93performance and 87

Joins on view 95

LLarge table/small table joins, performance and 97Lock Display utility 219Locking

deadlocks 118mechanisms for 117overview 117

Locking Logger utility, performance and 118Locks, tools for analyzing 206

228 Performance Management

Page 229: Teradata Database Performance Management

Index

MMemory

adding 213adjusting for low available free 152efficient use of 149free 152memory-consuming features and 156monitoring 151ResUsage and 152row redistribution and 154swapping, solutions to 153

Merge joins, performance and 79MERGE statement, error logging and 72MiniCylPack 134Multilevel partitioned primary index 101

NNode efficiency, ensuring 205Nodes

adding 214adding, determining configuration for 214

Non-FSG Cache size, minimum 152Nonunique Secondary Index. See NUSINoPI tables, support for 74, 128Normalized view for coexistence 42NUSI

join index and 93performance and 84rollback performance 71

NUSI. See also Secondary index

OOptimized DROP features, performance and 76Optimizer, EXPLAIN and 108

PPACKDISK utility 219

FreeSpacePercent and 140others utilities and 140

Parameterized statement caching 76Partitioned primary index, performance and 99Partitioned primary index. See also Multilevel partitioned

primary indexPartition-level backup and restore 101Path enhancements, non-key access 115Performance

CPU saturation and 197database design and 207deadlocks and 178I/O wait and 198processing concurrency and 178resource bottlenecks and 177

system components and 220system saturation and 177system upgrade and 215

Performance management, benefits of 21Performance reports 177Performance resources

Database Initialization Program 218DBC.AMPUsage view 218DBC.DiskSpace view 218DBC.LogOnOff view 218DBC.SessionInfo view 218DBC.Software_Event_Log view 218DBC.TableSize view 218DIPVIEWS 219EXPLAIN statement 219stune file 220TPCCONS 220

Performance toolsAccess controls 217Access Logging facility 217ctl utility 217Database Window 218DBC Control utility 218DEFRAGMENT utility 218Lock Display utility 219PACKDISK utility 219Priority Scheduler 219Query Session utility 219Recovery Manager 219ResUsage 219sar utility 219Scan Disk 219Show Locks utility 219SHOWSPACE command 220TOP utility 220vproc manager 220

Permanent space, spool space and 146PermDBAllocUnit 163PermDBSize 164PEs, adding 213Phantom spool, spool space accounting 147PPICacheThrP 164Priority Scheduler 219

overview 175performance groups and 32

Priority Scheduler data, ResUsage and 41

QQCF 53Query analysis

resources 53tools 53

Query band

Performance Management 229

Page 230: Teradata Database Performance Management

Index

SET QUERY_BAND 175using 175

Query Capture Facility. See QCFQuery Configuration utility 188Query Session utility 188, 219

RReconfiguration utility, using after expansion 214Recovery Manager 219Recursive query, performance impact of 64RedistBufSize 166Redistribution row 154Referential constraints, performance and 83Referential Integrity, performance and 80Request cache entries 75Resource Check Tools 192Resource usage. See ResUsageResource-intensive queries, deteching 182ResUsage 219

AMPUsage and 49available free memory and 152Bynet data macro and 48cache hit rates 42capacity planning and 43CPU utilization and 44DBC.AMPUsage view and 43disk utilization and 47host traffic and 44priority scheduler data and 41value of 41

RollbackPriority 166Rollbacks

canceling 201noticing slow rollback on ALTER TABLE 202

Row redistribution, reducing 77

Ssar utility 219Saturated resources, finding 198Scan Disk 219Secondary index

index access and 85performance and 84

Session elements, controlling 203SET QUERY_BAND, SQL statement 175Shared memory 149Show Locks utility 188, 219SHOWSPACE command, Ferret utility 220SkewAllowance 166Skewing 125, 126, 127, 203Slowdown, preventing 200Sparse indexes, performance and 95Spool space

accounting 147as trip wire 147increasing 147managing 146permanent space and 146

SQL and performanceaggregate cache size 75aggregates on view 95ALTER TABLE statement

column compression 61data retrieval 59

AMP sampling 103analytical functions 68BETWEEN clause 78bulk SQL error logging 72CASE expression 65collecting statistics 101collecting statistics guidelines 103CREATE TABLE and data retrieval 59dynamic hash joins 80EXPLAIN and the Optimizer 108GROUP BY and join optimization 97hash join costing 80hash joins 80identity column 113index access 85IN-List value limit 77INSERT. . . SELECT statement 72join index 87join index and NUSI, compared 93joins on views 95large table/small table joins 97merge joins 79multilevel partitioned primary index 101NUSI

rollback performance 71using 84

optimized DROP features 76parameterized statement caching 76PARTITION statistics 105partitioned primary index 99partition-level backup and restore 101recursive query 64reducing row redistribution 77referential constraints 83referential integrity 80request cache entries 75secondary index 84sparse indexes 95star join processing 99state statistics 106TOP N row option 63updatable cursors 114USI maintenance performance 71

230 Performance Management

Page 231: Teradata Database Performance Management

Index

USI rollback performance 71SQL statements, SET QUERY_BAND 175Stale statistics 106Star join processing, improvements to 99Statistics

guidelines for collecting 103PARTITION 105performance and 101stale 106

Statistics Wizard. See Teradata Statistics Wizardstune file 220System

bottlenecks, identifying 199expanding the Teradata Database 211expansion

determining requirements for 211tools for estimating need for 204

peak periods, ob scheduling and 199upgrade, performance and 215

System Activity Report utility 189System alerts, thresholds for 180System components, performance monitoring and 220System conditions, measuring

changes in data access 179data growth 179increase in active sessions 179increase in system demand 180resource utilization 179response time 179

System Emulation Tool. See Teradata SETSystem performance, impeded 177System reconfiguration, after expansion 214

TTarget Level Emulation. See TLETDP monitoring commands 189Teradata Active System Management. See Teradata ASMTeradata ASM

areas of management 170defined 169example 172flow 171overview 170

Teradata Index Wizard 55Teradata SET 55Teradata Statistics Wizard 57Teradata System Emulation Tool. See Teradata SETTeradata utilities

ctl 188Ferret, SHOWSPACE command 189Gateway Control 189Query Configuration 188Query Session 188

Reconfiguration 214Show Locks 188system performance and 188

Teradata Viewpointoverview 187performance and 24

Teradata Visual EXPLAIN 54TLE 54TOP N row option 63TOP utility 190, 220TPCCONS 220Transaction rollback, and performance 119

UUnique Secondary Index. See USIUpdatable cursors, performance and 114Upgrade. See System upgradeUSI maintenance performance 71USI rollback performance 71

VViewpoint. See Teradata ViewpointVisual EXPLAIN. See Teradata Visual EXPLAINvproc manager 220Vprocs, adding 212

WWeekly reports 181

Performance Management 231

Page 232: Teradata Database Performance Management

Index

232 Performance Management