TERADATA PPT

TERADATA TERADATA OVERVIEW & UTILITIES OVERVIEW & UTILITIES

KNOWLEDGE SHARING ON TERADATAKNOWLEDGE SHARING ON TERADATA

Introduction to TERADATAIntroduction to TERADATA Flow through TERADATA UTILITIES with Flow through TERADATA UTILITIES with

Lab ExamplesLab Examples Bteq Bteq Fast LoadFast Load Fast ExportFast Export Muilti LoadMuilti Load TpumpTpump

Comparative study of the Teradata Comparative study of the Teradata loading utilities.loading utilities.

AgendaAgenda

Teradata is a Relational Database Management System Teradata is a Relational Database Management System (RDBMS) that drives a company’s data warehouse.(RDBMS) that drives a company’s data warehouse.

Teradata is an open system, compliant with industry Teradata is an open system, compliant with industry ANSI standards. ANSI standards.

It is currently available for the following operating It is currently available for the following operating systems:systems:

· UNIX MP-RAS· UNIX MP-RAS

· Windows 2000· Windows 2000 The ability to manage terabytes of data is The ability to manage terabytes of data is

accomplished using the concept of parallelism, wherein accomplished using the concept of parallelism, wherein many individual processors perform smaller tasks many individual processors perform smaller tasks concurrently to accomplish an operation against a huge concurrently to accomplish an operation against a huge repository of data. repository of data.

To date, only parallel architectures can handle To date, only parallel architectures can handle databases of this size.databases of this size.

What is Teradata?What is Teradata?

There are many reasons to choose Teradata as the There are many reasons to choose Teradata as the preferred platform for enterprise data warehousing:preferred platform for enterprise data warehousing:

Supports easy scalability from a small (10 GB) to a Supports easy scalability from a small (10 GB) to a massive (100+TB) database.massive (100+TB) database.

Automatic and even data distribution eliminates Automatic and even data distribution eliminates complex indexing schemes or time-consuming complex indexing schemes or time-consuming reorganizations.reorganizations.

Designed and built with parallelism from day one .Designed and built with parallelism from day one . Single operational view of the entire MPP system and Single operational view of the entire MPP system and

single point of control for the DBA (Teradata Manager).single point of control for the DBA (Teradata Manager). Teradata has been doing data warehousing longer than Teradata has been doing data warehousing longer than

any other vendor.any other vendor.

Why Teradata?Why Teradata?

Start Smaller and Grow: One ExperienceStart Smaller and Grow: One Experience

200-300 users 200-300 users Over 7500 usersOver 7500 users

30 concurrent users Over 2000 concurrent 30 concurrent users Over 2000 concurrent usersusers

300 GB disk space Over 50 TB user data300 GB disk space Over 50 TB user data

1.7 billion-row table 1.7 billion-row table Over 7.5 billion -row tableOver 7.5 billion -row table

200 queries per day200 queries per day Over 20,000 queries per Over 20,000 queries per dayday

30M-row batch per night 30M-row batch per night Over 500M-row batch per Over 500M-row batch per nightnight

1 main application1 main application Over 30 applications Over 30 applications

BUT ONE REMAINS CONSTANSTBUT ONE REMAINS CONSTANST

Scalability in a Production EnvironmentScalability in a Production Environment

i . e, Through Put.i . e, Through Put.

ADVANTAGE of TERADATAADVANTAGE of TERADATAEase of setup and maintenanceEase of setup and maintenanceNo reorganization of data neededNo reorganization of data neededMost robust utilities in the industryMost robust utilities in the industryLow cost of disk to data ratioLow cost of disk to data ratioEase in expanding the systemEase in expanding the system

ADVANTAGE of TERADATAADVANTAGE of TERADATA

SMP & MPP platformsSMP & MPP platforms BYNETBYNET Disk ArraysDisk Arrays CliquesCliques Hot Standby nodesHot Standby nodes Virtual processorsVirtual processors Request processingRequest processing Teradata Database RASUITeradata Database RASUI

TERADATA Architecture(Shared Nothing)TERADATA Architecture(Shared Nothing)

Each Teradata Node is made up of hardware and Each Teradata Node is made up of hardware and softwaresoftwareEach node has CPUs, system disk, memory and Each node has CPUs, system disk, memory and adaptersadaptersEach node runs copy of OS and database SWEach node runs copy of OS and database SW

Node Architecture(Shared Nothing)Node Architecture(Shared Nothing)

BYNETBYNETThe BYNET performs the internal communication of the The BYNET performs the internal communication of the Teradata RDBMSTeradata RDBMSAll communication between PEs and AMPs is done via All communication between PEs and AMPs is done via the BYNETthe BYNET

BYNET in TERADATABYNET in TERADATA

Boardless BYNETBoardless BYNETSingle-node SMP systems use Board less Bynet ( or virtual Single-node SMP systems use Board less Bynet ( or virtual BYNET software to simulate the Bynet hardware driver.BYNET software to simulate the Bynet hardware driver.

Disk ArraysDisk Arrays A Disk Array is a configuration of disk drives that A Disk Array is a configuration of disk drives that

utilizes specialized controllers to manage and utilizes specialized controllers to manage and distribute data and parity across the Disks while distribute data and parity across the Disks while providing fast access and data integrity.providing fast access and data integrity.

CliqueClique A Clique is a set of Teradata nodes that share a A Clique is a set of Teradata nodes that share a

common set of disk arrays.common set of disk arrays. In the event of failure, all virtual processors can In the event of failure, all virtual processors can

migrate to another available node in the clique.migrate to another available node in the clique. All nodes in the clique must have access to the same All nodes in the clique must have access to the same

disk arrays.disk arrays.

Disk Arrays & CliqueDisk Arrays & Clique

Hot Standby NodesHot Standby Nodes

The hot standby Node feature allows spare nodes to be The hot standby Node feature allows spare nodes to be incorporated into the production environment so that incorporated into the production environment so that Teradata Database can take advantage of the presence of Teradata Database can take advantage of the presence of the spare nodes to improve availability and maintain the spare nodes to improve availability and maintain performance levels.performance levels.

Is Member of a CliqueIs Member of a Clique

Does not normally participate in the trusted parallel Does not normally participate in the trusted parallel application(TPA).application(TPA).

Can be brought into the TPA to compensate for the loss Can be brought into the TPA to compensate for the loss of a node in the Cliqueof a node in the Clique

Virtual ProcessorsVirtual Processors

The versatility of Teradata Database is based on virtual The versatility of Teradata Database is based on virtual processors(vprocs) that eliminate dependency on processors(vprocs) that eliminate dependency on specialized physical processors. Vprocs are a set of specialized physical processors. Vprocs are a set of software processes that run on a node under Teradata software processes that run on a node under Teradata Parallel Database Extensions(PDE) within the multitasking Parallel Database Extensions(PDE) within the multitasking environment of the operating system.environment of the operating system.

The two types of vprocsThe two types of vprocsPE PE Parsing Engine) Parsing Engine)AMP AMP Access module processor Access module processor

AMPAMPThe AMP is a virtual processor designed for and dedicated to The AMP is a virtual processor designed for and dedicated to managing a portion of the entire database.managing a portion of the entire database.An AMP will control some portion of each table on the system.An AMP will control some portion of each table on the system.It performs all database management functions such as sorting, It performs all database management functions such as sorting, aggregating and formatting data.aggregating and formatting data.The AMP receives data from the PE, formats rows and distributes The AMP receives data from the PE, formats rows and distributes them to the disk storage units it controls.them to the disk storage units it controls.The AMP also retrieves the rows requested by the PE.The AMP also retrieves the rows requested by the PE.

VprocVproc

PEPEA Parsing Engine (PE) is a virtual processor that manages the A Parsing Engine (PE) is a virtual processor that manages the dialogue between the client application and the RDBMS.dialogue between the client application and the RDBMS.It interprets the SQL requests, receives input records and passes It interprets the SQL requests, receives input records and passes data.data.It is made of the following software components: Session It is made of the following software components: Session Control, the Parser, the Optimizer and the DispatcherControl, the Parser, the Optimizer and the Dispatcher

Table A rowsTable A rowsTable B rowsTable B rows

AMPAMP AMPAMP AMPAMP AMPAMP

• The rows of The rows of everyevery table are distributed among all AMPs table are distributed among all AMPs

• Each AMP is responsible for a Each AMP is responsible for a subsetsubset of the rows of each table. of the rows of each table.

• Ideally, each table will be evenly distributed among all AMPs.Ideally, each table will be evenly distributed among all AMPs.

• Evenly distributed tables result in evenly distributed workloads.Evenly distributed tables result in evenly distributed workloads.

• The uniformity of distribution of the rows of a table depends on the choice The uniformity of distribution of the rows of a table depends on the choice of the of the Primary Index.Primary Index.

Data Store on DisksData Store on Disks

Request processingRequest processing

NODE

Parsing Engine Parsing Engine

BYNET

AMP AMP AMP AMP

Disk Storage Disk Storage Disk Storage Disk Storage

SQL Request Answer Set Response

The major Teradata utility that assists in data The major Teradata utility that assists in data warehousing management and maintenance warehousing management and maintenance along with the Teradata RDBMS are along with the Teradata RDBMS are

BTEQ BTEQ FASTLOAD FASTLOAD FAST EXPORT FAST EXPORT MULTILOAD MULTILOAD TPUMPTPUMP

TERADATA UTILITIESTERADATA UTILITIES INTRODUCTION INTRODUCTION

General-purpose, command-based program that General-purpose, command-based program that

allows users on a workstation to communicate with one allows users on a workstation to communicate with one

or more Teradata Database systems.or more Teradata Database systems. A set of SQL statements used to inserts updates or A set of SQL statements used to inserts updates or

deletes in Teradata tables.deletes in Teradata tables.Imports data to Teradata database from a file. Imports data to Teradata database from a file. Exports data from table and formats the results and Exports data from table and formats the results and

returns them to the screen, a file, or to a designated returns them to the screen, a file, or to a designated

printer.printer. Do report the error occurs but will not capture it as Do report the error occurs but will not capture it as

log. log.

BTEQ - BTEQ - BBasic asic TTeradata eradata QQueryuery

Enter Teradata SQL statements to view, add, modify, Enter Teradata SQL statements to view, add, modify, and delete data.and delete data.

Enter operating system commands. Enter operating system commands. CreateCreate and use Teradata stored procedures and use Teradata stored procedures BTEQ supports Teradata-specific SQL functions for BTEQ supports Teradata-specific SQL functions for

doing complex analytical querying and data miningdoing complex analytical querying and data mining All database requests in BTEQ are expressed in All database requests in BTEQ are expressed in

Teradata SQL. Teradata SQL. BTEQ also supports the conditional logic (i.e., BTEQ also supports the conditional logic (i.e.,

"IF.THEN...") based on activity count or error code. It is "IF.THEN...") based on activity count or error code. It is useful for batch mode export / import processing. useful for batch mode export / import processing.

Error handling is applicable in BTEQ. We can assign Error handling is applicable in BTEQ. We can assign error level for each error code and make decisions error level for each error code and make decisions based on the level assigned. based on the level assigned.

Capabilities in BTEQ Capabilities in BTEQ

Interactive modeInteractive mode you start a BTEQ session by entering BTEQyou start a BTEQ session by entering BTEQ logon at the logon at the

system prompt on your terminal and submit SQL system prompt on your terminal and submit SQL commands to the database as needed. commands to the database as needed.

Format of logon cmd: bteq .logon server Format of logon cmd: bteq .logon server name/user_name, passwordname/user_name, password

Batch modeBatch mode In batch mode, you prepare BTEQ scripts or macros, In batch mode, you prepare BTEQ scripts or macros,

and then submit them to BTEQ from a scheduler or and then submit them to BTEQ from a scheduler or manually for processing.manually for processing.

A BTEQ script is a set of SQL statements and BTEQ A BTEQ script is a set of SQL statements and BTEQ commands saved in a file with the extension ".bteq". commands saved in a file with the extension ".bteq".

The BTEQ script can be run using the following The BTEQ script can be run using the following command (in UNIX or Windows)command (in UNIX or Windows)

OPERATING MODES in BTEQOPERATING MODES in BTEQ

Export BTEQ by default delivers a response to all SQL Export BTEQ by default delivers a response to all SQL queries that includes a helpful message along with queries that includes a helpful message along with helpful diagnostic information about the time taken to helpful diagnostic information about the time taken to perform the query. perform the query.

If all of this information is captured in a single output file, If all of this information is captured in a single output file, this mixed output typically renders the data unsuitable this mixed output typically renders the data unsuitable for some other purposes. for some other purposes. So the .EXPORT feature provides the ability to separate So the .EXPORT feature provides the ability to separate

the report or output data to a separate file.the report or output data to a separate file. The output file of this script will contain only the The output file of this script will contain only the

messages and not the data. It is exported to a file which messages and not the data. It is exported to a file which can be used for some other purposes also.can be used for some other purposes also.

Export types are Export types are export record , export data, export export record , export data, export reset , export indicdata, export difreset , export indicdata, export dif

EXPORT in BTEQEXPORT in BTEQ

Import data from host to Teradata as a series of inserts Import data from host to Teradata as a series of inserts updates and deletes.updates and deletes.

Import types supported are Import types supported are import dataimport data import recordimport record import indicdata.import indicdata.

IMPORT in BTEQIMPORT in BTEQ

All the BTEQ commands must be preceded by a dot ‘All the BTEQ commands must be preceded by a dot ‘..’ ’ character and also BTEQ commands may or may not character and also BTEQ commands may or may not end with a semicolon ‘;’.end with a semicolon ‘;’.

They are of four types asThey are of four types as Session controlSession control File controlFile control Format controlFormat control Sequence control commandsSequence control commands

BTEQ COMMANDSBTEQ COMMANDS

Report formatting. Report formatting.

Ad hoc query tool .Ad hoc query tool .

Database administration .Database administration .

Best for small data volumes.Best for small data volumes.

BTEQ AdvantagesBTEQ Advantages

Lab.shLab.sh

#! /bin/sh#! /bin/sh

.logon tdprd/username, pwd; .logon tdprd/username, pwd;

.Export report File=lab.txt .Export report File=lab.txt

.set record vartext "|";.set record vartext "|";

.BEGIN LOADING emp ERRORFILES Error_1, Error_2; .BEGIN LOADING emp ERRORFILES Error_1, Error_2;

DEFINE DEFINE

empnoempno (VARCHAR (50)),(VARCHAR (50)),

empname empname (VARCHAR (50)),(VARCHAR (50)),

dojdoj (VARCHAR (30)) (VARCHAR (30))

FILE = /ngs/app/asrdedwp/SCRIPTS/emp.txt;FILE = /ngs/app/asrdedwp/SCRIPTS/emp.txt;

.Set Underline Off;.Set Underline Off;

.Set Titledashes Off;.Set Titledashes Off;

.Set Errorout Stdout;.Set Errorout Stdout;

.Set Width 4000;.Set Width 4000;

select * From table_name where …;select * From table_name where …;

Delete from table_name where…….;Delete from table_name where…….;

Insert into table name values(….);Insert into table name values(….);

Update table name set where…..;Update table name set where…..;

Call macro, procedure etc… Call macro, procedure etc…

.if errorcode <> 0 then .exit 2.if errorcode <> 0 then .exit 2

.export reset.export reset

.logoff.logoff

.quit.quit

Bteq Lab ExerciseBteq Lab Exercise

FastLoad- Fload or FL is a multi - sessioned parallel load FastLoad- Fload or FL is a multi - sessioned parallel load utility for initial table load in bulk mode on a Teradata utility for initial table load in bulk mode on a Teradata Database.Database.

It is a command-driven utility to load large data into an It is a command-driven utility to load large data into an empty table on a Teradata RDBMS with no secondary empty table on a Teradata RDBMS with no secondary indexes.indexes.

It uses multiple database sessions to load data.It uses multiple database sessions to load data.

FL-FASTLOADFL-FASTLOAD

Full Restart capability.Full Restart capability. Checkpoints provided for restart.Checkpoints provided for restart. Checkpoints slow fast load processing. Set the Checkpoints slow fast load processing. Set the

checkpoint large enough to be taken every 10 to 15 checkpoint large enough to be taken every 10 to 15 minutes.minutes.

Two Error tables and Error Limits, accessible using SQL.Two Error tables and Error Limits, accessible using SQL. In one Error table, rows which failed due to constraints In one Error table, rows which failed due to constraints

or translation errors are loaded. In another table or translation errors are loaded. In another table duplicate rows for UPIs are captured.duplicate rows for UPIs are captured.

Error table is loaded with one row at a time, so errors Error table is loaded with one row at a time, so errors slow down the performance of fastload.slow down the performance of fastload.

FASTLOAD CapabilityFASTLOAD Capability

Phase 1Phase 1 FastLoad uses one SQL session to define AMP stepsFastLoad uses one SQL session to define AMP steps The PE sends a block to each AMPThe PE sends a block to each AMP Amps hash each record and redistribute them to the AMP Amps hash each record and redistribute them to the AMP

responsible for the hash valueresponsible for the hash value Records are written to the target table in unsorted blocksRecords are written to the target table in unsorted blocks

Phase 2Phase 2 starts after .end loading command. So if this command not starts after .end loading command. So if this command not

specified fast load will be paused and not terminated.specified fast load will be paused and not terminated. When loading completes, each AMP sorts the target table, puts When loading completes, each AMP sorts the target table, puts

the rows into blocks, and writes the blocks to the diskthe rows into blocks, and writes the blocks to the disk Fall back rows are then generated if requiredFall back rows are then generated if required

Fast Load Operates in two phases Fast Load Operates in two phases

Interactive modeInteractive mode In In interactive modeinteractive mode, Teradata FastLoad uses , Teradata FastLoad uses

terminal screen and keyboard as the standard terminal screen and keyboard as the standard output and input streams. output and input streams.

For Interactive mode, fastloadFor Interactive mode, fastload .logon .logon tdprd/user_id, pwdtdprd/user_id, pwd

Batch modeBatch mode In In batch modebatch mode, FastLoad uses , FastLoad uses >> and and << to to

redirect the standard output / input streams. redirect the standard output / input streams. For Batch mode, fastload [options] < For Batch mode, fastload [options] < infileinfile > > outfileoutfile

Here, the Here, the infileinfile is a Teradata FastLoad job script is a Teradata FastLoad job script file and the file and the outfileoutfile is the FastLoad output stream is the FastLoad output stream file. file.

OPERATING MODES in Fast LoadOPERATING MODES in Fast Load

CREATE TABLE CREATE TABLE

Defines the columns, index and other qualities of Defines the columns, index and other qualities of a tablea table

DATABASE DATABASE

Changes the default databaseChanges the default database DELETE DELETE

Deletes rows from a tableDeletes rows from a table DROP TABLE DROP TABLE

Removes a table and all of its rows from a Removes a table and all of its rows from a databasedatabase

INSERTINSERT

Inserts rows into a tableInserts rows into a table

SQL Statements Supported in FastLoadSQL Statements Supported in FastLoad

.LOGON TDP/username,pwd .LOGON TDP/username,pwd;.LOGON TDP/username,pwd .LOGON TDP/username,pwd;

errlimit 1;errlimit 1;

tenacity 4; tenacity 4;

sleep 6;sleep 6;

DROP TABLE ;DROP TABLE ;

SET RECORD UNFORMATTED;SET RECORD UNFORMATTED;

.begin loading filename errorfiles filename_ref1, filename_ref2;.begin loading filename errorfiles filename_ref1, filename_ref2;

DefineDefine

feildsfeilds

file=stg_sref_service_price.out;file=stg_sref_service_price.out;

show;show;

checkpoint 0 ;checkpoint 0 ;

INSERT INTO table nameINSERT INTO table name

( (

) )

VALUESVALUES

( (

) ; ) ;

end loading;end loading;

logoff;;logoff;;

Lab Excercise FastLoadLab Excercise FastLoad

Teradata FastExport, also called "FastExport" or "FE," is Teradata FastExport, also called "FastExport" or "FE," is a multi-sessioned command-driven utility for export in a multi-sessioned command-driven utility for export in bulk mode from tables and views of the Teradata bulk mode from tables and views of the Teradata Database to a client-based application.Database to a client-based application.

It is the reverse of the Teradata FastLoad utility. It is the reverse of the Teradata FastLoad utility. Teradata FastExport processes a series of FastExport Teradata FastExport processes a series of FastExport

commands and Teradata SQL statements written in a commands and Teradata SQL statements written in a batch mode job script or interactively entered. batch mode job script or interactively entered.

The FastExport commands provide the session control The FastExport commands provide the session control and data handling specifications for the data transfer and data handling specifications for the data transfer operations, and the Teradata SQL statements perform operations, and the Teradata SQL statements perform the actual data export functions on the Teradata the actual data export functions on the Teradata RDBMS tables and viewsRDBMS tables and views

FE—Fast ExportFE—Fast Export

Fully automated Restart.Fully automated Restart.

Export from multiple tablesExport from multiple tables

There are two techniques to provide variable inputs to There are two techniques to provide variable inputs to fastexport for selection controls.fastexport for selection controls.

They are They are

ACCEPT from a parameter file; only accept from a ACCEPT from a parameter file; only accept from a

single recordsingle record IMPORT from a data file; each import record is applied IMPORT from a data file; each import record is applied

to every select.to every select.

Capability of Fast ExportCapability of Fast Export

Interactive mode; Interactive mode; In interactive mode, Teradata FastLoad uses terminal screen and In interactive mode, Teradata FastLoad uses terminal screen and

keyboard as the standard output and input streams.keyboard as the standard output and input streams.

Interactive mode for Microsoft Windows: c:\ncr\fexq Interactive mode for Microsoft Windows: c:\ncr\fexq

Batch mode; Batch mode;

Batch mode for Microsoft Windows: c:\ncr\fexq [options] < Batch mode for Microsoft Windows: c:\ncr\fexq [options] < infileinfile > > outfileoutfile

In batch mode, FastExport uses In batch mode, FastExport uses >> and and << to redirect the standard to redirect the standard output / input streams. output / input streams.

Operating Modes in Fast ExportOperating Modes in Fast Export

CREATE TABLE CREATE TABLE

Defines the columns, index and other qualities of Defines the columns, index and other qualities of a tablea table

DATABASE DATABASE

Changes the default databaseChanges the default database DELETE DELETE

Deletes rows from a tableDeletes rows from a table DROP TABLE DROP TABLE

Removes a table and all of its rows from a Removes a table and all of its rows from a databasedatabase

INSERTINSERT

Inserts rows into a tableInserts rows into a table

SQL Statements Supported in Fast ExportSQL Statements Supported in Fast Export

ALTER TABLEALTER TABLEChanges the column configuration or options of an existing Changes the column configuration or options of an existing tabletable

COLLECT STATISTICS COLLECT STATISTICS Collects statistical data for one or more columns of a tableCollects statistical data for one or more columns of a table

COMMENTCOMMENTStores or retrieves comment string associated with a database Stores or retrieves comment string associated with a database objectobject

CREATE DATABASE,MACRO,TABLE,VIEWCREATE DATABASE,MACRO,TABLE,VIEWCreates a new database, macro, table, or viewCreates a new database, macro, table, or view

DATABASEDATABASESpecifies a new default database for the current sessionSpecifies a new default database for the current session

DELETEDELETERemoves rows from a tableRemoves rows from a table

SQL Statements Supported in Fast ExportSQL Statements Supported in Fast Export

DELETE DATABASEDELETE DATABASE

Removes all tables, views, and macros from a databaseRemoves all tables, views, and macros from a database DROP DATABASEDROP DATABASE

Drops the definition for an empty database from the Data Drops the definition for an empty database from the Data DictionaryDictionary

DROP TABLEDROP TABLE

Removes a table from the databaseRemoves a table from the database GIVEGIVE

Transfers ownership of a database to another userTransfers ownership of a database to another user GRANTGRANT

Grants access privileges to a database objectGrants access privileges to a database object INSERTINSERT

Inserts new rows to a tableInserts new rows to a table

SQL Statements Supported in FastExportSQL Statements Supported in FastExport

RENAMERENAME Changes the name of an existing table, view, or macro Changes the name of an existing table, view, or macro

REPLACE MACRO,VIEWREPLACE MACRO,VIEW Redefines an existing macro or viewRedefines an existing macro or view

REVOKEREVOKE Rescinds access privileges to a database objectRescinds access privileges to a database object

UPDATEUPDATE Changes the column values of an existing row in a tableChanges the column values of an existing row in a table

SQL Statements Supported in FastExportSQL Statements Supported in FastExport

..LOGTABLE utillog; LOGTABLE utillog;

.LOGON tdpz/user,pswd; .LOGON tdpz/user,pswd;

.BEGIN EXPORT .BEGIN EXPORT

SESSIONS 20; SESSIONS 20;

.LAYOUT UsingData; .LAYOUT UsingData;

.FIELD ProjId * Char(8); .FIELD ProjId * Char(8);

.FIELD WkEnd * Date; .FIELD WkEnd * Date;

.IMPORT INFILE ddname1 .IMPORT INFILE ddname1

LAYOUT UsingData; LAYOUT UsingData;

.EXPORT OUTFILE ddname2; .EXPORT OUTFILE ddname2;

SELECT SELECT

EmpNo, EmpNo,

HoursHours

FROM CHARGES FROM CHARGES WHERE WkEnd = :WkEnd /* these input variables are refered from imported WHERE WkEnd = :WkEnd /* these input variables are refered from imported

AND Proj_ID = :ProjId input file */AND Proj_ID = :ProjId input file */

ORDER BY EmpNo; ORDER BY EmpNo;

.END EXPORT; .END EXPORT;

.LOGOFF;.LOGOFF;

Lab Exercise Fast ExportLab Exercise Fast Export

MultiLoad - MLoad or ML is a command-driven MultiLoad - MLoad or ML is a command-driven parallel load utility for high-volume batch parallel load utility for high-volume batch maintenance on multiple tables and views of maintenance on multiple tables and views of the Teradata Databasethe Teradata Database

Multi LoadMulti Load

Teradata MultiLoad executes a series of MultiLoad Teradata MultiLoad executes a series of MultiLoad commands and Teradata SQL statements written in a commands and Teradata SQL statements written in a batch mode job script or interactively enteredbatch mode job script or interactively entered

Supports up to five populated tablesSupports up to five populated tables Fastload like technology – Tpump like functionalityFastload like technology – Tpump like functionality Multiple operations with one pass of input filesMultiple operations with one pass of input files Conditional logic for applying changes Conditional logic for applying changes Supports INSERTs, UPDATEs, DELETEs and UPSERTsSupports INSERTs, UPDATEs, DELETEs and UPSERTs Full restart capabilityFull restart capability Error reporting via error tablesError reporting via error tables Support for INMODsSupport for INMODs

Features of MultiLoadFeatures of MultiLoad

ALTER TABLEALTER TABLE

Changes the column configuration or options of an existing tableChanges the column configuration or options of an existing table COLLECT STATISTICS COLLECT STATISTICS

Collects statistical data for one or more columns of a tableCollects statistical data for one or more columns of a table COMMENTCOMMENT

Stores or retrieves comment string associated with a database objectStores or retrieves comment string associated with a database object CREATE DATABASE,MACRO,TABLE,VIEWCREATE DATABASE,MACRO,TABLE,VIEW

Creates a new database, macro, table, or viewCreates a new database, macro, table, or view DATABASEDATABASE

Specifies a new default database for the current sessionSpecifies a new default database for the current session DELETEDELETE

Removes rows from a tableRemoves rows from a table DELETE DATABASEDELETE DATABASE

Removes all tables, views, and macros from a databaseRemoves all tables, views, and macros from a database

SQL Statements Supported in MultiLoad SQL Statements Supported in MultiLoad

DROP DATABASEDROP DATABASEDrops the definition for an empty database from the Data DictionaryDrops the definition for an empty database from the Data Dictionary

DROP TABLEDROP TABLERemoves a table from the databaseRemoves a table from the database

GIVEGIVETransfers ownership of a database to another userTransfers ownership of a database to another user

GRANTGRANTGrants access privileges to a database objectGrants access privileges to a database object

INSERTINSERTInserts new rows to a tableInserts new rows to a table

RENAMERENAMEChanges the name of an existing table, view, or macro Changes the name of an existing table, view, or macro

REPLACE MACRO,VIEWREPLACE MACRO,VIEWRedefines an existing macro or viewRedefines an existing macro or view

REVOKEREVOKERescinds access privileges to a database objectRescinds access privileges to a database object

UPDATEUPDATEChanges the column values of an existing row in a tableChanges the column values of an existing row in a table


Interactive modeInteractive mode In Interactive mode, Teradata MultiLoad uses terminal screen and In Interactive mode, Teradata MultiLoad uses terminal screen and

keyboard as the standard output and input streams. keyboard as the standard output and input streams. Interactive mode for Microsoft Windows: c:\ncr\bin\MultiLoad Interactive mode for Microsoft Windows: c:\ncr\bin\MultiLoad

Batch modeBatch mode In batch mode MultiLoad uses In batch mode MultiLoad uses >> and and << to redirect the standard to redirect the standard

output / input streams. output / input streams.

Batch mode for Microsoft Windows : c:\ncr\bin\MultiLoad Batch mode for Microsoft Windows : c:\ncr\bin\MultiLoad [options] < [options] < infileinfile > > outfileoutfile

infileinfile is a Teradata MultiLoad job script file and the is a Teradata MultiLoad job script file and the outfileoutfile is the is the output stream file. output stream file.


IMPORT task IMPORT task These are the tasks which intermix a number of different SQL/DML These are the tasks which intermix a number of different SQL/DML

statements and apply them to up to five different tables depending statements and apply them to up to five different tables depending on the APPLY conditionson the APPLY conditions

Import tasks are always primary index operations, but not allowed Import tasks are always primary index operations, but not allowed to change the value of table’s primary index.to change the value of table’s primary index.

Allows restart and checkpoint during each operating phase.Allows restart and checkpoint during each operating phase. Import tasks cannot be done on tables with USI’s, Referential Import tasks cannot be done on tables with USI’s, Referential

Integrity, Join Indexes, Hash Indexes, and Triggers.Integrity, Join Indexes, Hash Indexes, and Triggers. Phases involved in this task arePhases involved in this task are

Preliminary – Basic set upPreliminary – Basic set up DML phase – Get DML steps down on AmpsDML phase – Get DML steps down on Amps Acquisition phase – Send the input data to Amps and sort itAcquisition phase – Send the input data to Amps and sort it Application phase – Send the input data to target tablesApplication phase – Send the input data to target tables End phase – Basic clean upEnd phase – Basic clean up

MULTILOAD TASKSMULTILOAD TASKS

Basic set up involves validate all sql, starts all sessions, Basic set up involves validate all sql, starts all sessions, create work tables (one per target), error tables (two create work tables (one per target), error tables (two per target), restart log table (one per table), apply locks per target), restart log table (one per table), apply locks to target tables (to prevent access to target while to target tables (to prevent access to target while loading).loading).

Basic clean up involves session logoff, dropping error Basic clean up involves session logoff, dropping error and work tables, releasing table locks.and work tables, releasing table locks.

DELETE taskDELETE task

These are tasks which execute a single DELETE These are tasks which execute a single DELETE

statement on a single tablestatement on a single table..


Each MultiLoad import task can do multiple data insert, Each MultiLoad import task can do multiple data insert, update, and delete functions on up to five different update, and delete functions on up to five different tables or views; tables or views;

Each MultiLoad import task can have up to 100 DML Each MultiLoad import task can have up to 100 DML steps; steps;

Each MultiLoad delete task can remove large numbers Each MultiLoad delete task can remove large numbers of rows from a single table. of rows from a single table.

Multi Load AdvantageMulti Load Advantage

.LOGTABLE dwlogtable;.LOGTABLE dwlogtable;

.logon TDPROD/username,pwd;.logon TDPROD/username,pwd;

.begin import mload tables FRAUD_CHECK;.begin import mload tables FRAUD_CHECK;

.layout UsingData; ;.layout UsingData; ;

.field Claimed_Fg_Serial_Nr.field Claimed_Fg_Serial_Nr * VARCHAR(50);* VARCHAR(50);

.field Claimed_Module_Serial_Nr.field Claimed_Module_Serial_Nr * VARCHAR(50);* VARCHAR(50);

.field Request_Ts .field Request_Ts * VARCHAR(30);* VARCHAR(30);

.field Notif_Id.field Notif_Id * VARCHAR(30);* VARCHAR(30);

.field Claimed_Module_Part_Nr.field Claimed_Module_Part_Nr * VARCHAR(50);* VARCHAR(50);

.field Fraud_Request_Source_Id.field Fraud_Request_Source_Id * VARCHAR(30);* VARCHAR(30);

.Field Claimed_Fg_Warranty_Cd .Field Claimed_Fg_Warranty_Cd * VARCHAR(30);* VARCHAR(30);

.Field Fraud_Mode_Cd .Field Fraud_Mode_Cd * VARCHAR(30);* VARCHAR(30);

.Field Commodity_Cd .Field Commodity_Cd * VARCHAR(10);* VARCHAR(10);

.Field Dispatch_Id .Field Dispatch_Id * VARCHAR(20);* VARCHAR(20);

.Field Create_Ts .Field Create_Ts * VARCHAR(30);* VARCHAR(30);

.dml label FRAUD_CHECKdml;.dml label FRAUD_CHECKdml;

insert into FRAUD_CHECK.*;insert into FRAUD_CHECK.*;

.import inFILE FRAUD_CHECK_Request_LOG.txt.import inFILE FRAUD_CHECK_Request_LOG.txt

format VARtext ';'format VARtext ';'

layout UsingData layout UsingData

apply FRAUD_CHECKdml;apply FRAUD_CHECKdml;

.end mload;.end mload;

.logoff;.logoff;


Teradata TPump, short for " TPump, short for "TTeradata Parallel Data eradata Parallel Data PumpPump," is a ," is a continuous data-loading utility used to move data into the continuous data-loading utility used to move data into the Teradata Database without locking the affected tables. Teradata Database without locking the affected tables.

Instead of updating Teradata Databases overnight, or in batches Instead of updating Teradata Databases overnight, or in batches throughout the day, TPump updates information in near real-time throughout the day, TPump updates information in near real-time or real time, acquiring data from the client system with low or real time, acquiring data from the client system with low processor utilization.processor utilization.

This parallel utility is featured by stream-mode loading its SQL-This parallel utility is featured by stream-mode loading its SQL-based, but not block-based, protocol. based, but not block-based, protocol.

TPUMPTPUMP

DATABASEDATABASE

Changes the default database qualification for all DML Changes the default database qualification for all DML statements.statements.

DELETEDELETE

Removes specified rows from a tableRemoves specified rows from a table EXECUTEEXECUTE

Specifies a user-created (predefined) macro for Specifies a user-created (predefined) macro for execution.The macro named in this statement resides in the execution.The macro named in this statement resides in the Teradata Database and Teradata Database and specifies the type of DML specifies the type of DML statement (INSERT, UPDATE, or DELETE) being handled by the statement (INSERT, UPDATE, or DELETE) being handled by the macro. macro.

INSERTINSERT

Adds new rows to a table by directly specifying the row Adds new rows to a table by directly specifying the row data to be inserteddata to be inserted

UPDATEUPDATE

Changes field values in existing rows of a table.Changes field values in existing rows of a table.

SQL Statements Supported in TPumpSQL Statements Supported in TPump

Interactive mode;Interactive mode;

In interactive mode, Teradata Tpump uses terminal screen and In interactive mode, Teradata Tpump uses terminal screen and keyboard as the standard output and input streams, involving the keyboard as the standard output and input streams, involving the more or less continuous participation of the user. more or less continuous participation of the user.

Batch mode. Batch mode.

In batch mode, Teradata Tpump processes data in discrete groups In batch mode, Teradata Tpump processes data in discrete groups of previously scheduled operations, typically in a separate of previously scheduled operations, typically in a separate operation, rather than interactively or in real time.operation, rather than interactively or in real time.

Operation Modes in TPumpOperation Modes in TPump

Its setup does not require staging of data, intermediary files, or Its setup does not require staging of data, intermediary files, or special hardware; special hardware;

Its operation is not affected by database restarts, dirty data, and Its operation is not affected by database restarts, dirty data, and network slowdowns. network slowdowns.

Its jobs restart without intervention; Its jobs restart without intervention; Fast, scalable continuous data loads Fast, scalable continuous data loads Row hash lock enables concurrent queries Row hash lock enables concurrent queries Dynamic throttling feature Dynamic throttling feature Best for small data volumes Best for small data volumes Multiple sessions and multistatement request are typically used to Multiple sessions and multistatement request are typically used to

increase throughput.increase throughput. TPump also provides a dynamic throttling feature that enables it to TPump also provides a dynamic throttling feature that enables it to

run “all out” during batch windows, but within limits when it may run “all out” during batch windows, but within limits when it may impact other business uses of the Teradata RDBMS. Operators can impact other business uses of the Teradata RDBMS. Operators can specify the number of statements run per minute, or may alter specify the number of statements run per minute, or may alter throttling minute-by-minute, if necessary.throttling minute-by-minute, if necessary.

TPUMP AdvantagesTPUMP Advantages

.LOGON TDMNT/username,pwd;.LOGON TDMNT/username,pwd;

.LOGTABLE cs_irepair_log;.LOGTABLE cs_irepair_log;

.ROUTE MESSAGES WITH ECHO TO FILE ‘cs_irepair.out.ldrlog' ;.ROUTE MESSAGES WITH ECHO TO FILE ‘cs_irepair.out.ldrlog' ;

DROP TABLE ET_cs_irepair;DROP TABLE ET_cs_irepair;

.BEGIN LOAD.BEGIN LOAD

ERRORTABLE ET_cs_irepairERRORTABLE ET_cs_irepair

ERRLIMIT 0ERRLIMIT 0

CHECKPOINT 15CHECKPOINT 15

TENACITY 4TENACITY 4

SESSIONS 1SESSIONS 1

SLEEP 6SLEEP 6

SERIALIZE OFFSERIALIZE OFF

PACK 20PACK 20

ROBUST OFFROBUST OFF

NOMONITOR;NOMONITOR;

/* Begin Layout Section *//* Begin Layout Section */

.Layout InputFileLayout;.Layout InputFileLayout;

.Field Warr_Code * varchar(6) ;.Field Warr_Code * varchar(6) ;

.Field Prod_hier * varchar(1) ;.Field Prod_hier * varchar(1) ;

.Field Descript * varchar(50) ;.Field Descript * varchar(50) ;

.Field Serv_Matl * varchar(50) ;.Field Serv_Matl * varchar(50) ;

.Field Reference * varchar(20) ;.Field Reference * varchar(20) ;

/* End Layout Section *//* End Layout Section */

/* begin DML Section *//* begin DML Section */

.DML Label tagDML;.DML Label tagDML;

INSERT INTO cs_irepair (INSERT INTO cs_irepair (

Warr_Code ,Warr_Code ,

Prod_hier ,Prod_hier ,

Descript ,Descript ,

Serv_Matl ,Serv_Matl ,

Reference )Reference )

VALUES (VALUES (

:Warr_Code ,:Warr_Code ,

:Prod_hier ,:Prod_hier ,

:Descript ,:Descript ,

:Serv_Matl ,:Serv_Matl ,

:Reference :Reference

) ;) ;

/* end DML Section *//* end DML Section */

/* Begin Import Section *//* Begin Import Section */

.Import Infile 'cs.txt'.Import Infile 'cs.txt'

Layout InputFileLayoutLayout InputFileLayout

Format vartext'|'Format vartext'|'

Apply tagDML;Apply tagDML;

/* End Import Section *//* End Import Section */

.END LOAD;.END LOAD;

.LOGOFF.LOGOFF;;

TPUMP Lab exerciseTPUMP Lab exercise

Comparative study of the Teradata loading utilitiesComparative study of the Teradata loading utilities

TERADATA PPT

Documents

Transcript of TERADATA PPT