A high

A high-availability system must have the ability to identify and correct errors, exceptions and failures in a timely and

reliable manner to meet challenging service level objectives. The Teradata database and the utilities and components

(used to both load and access data) provide capabilities to implement reliable error and exception handling functionality.

These capabilities combined with a well designed high availability architecture allow a Teradata Active Enterprise

Intelligence (AEI) system to meet the service level objectives required to support mission critical business processes.

This series of articles focuses on the error handling functionality and restart capabilities of the Teradata database and the

Teradata data load utilities.

Examples of error and restart handling for the following utilities are included in this series:

Fastload

Multiload

Tpump

Teradata Parallel Transporter (TPT)

o Load operator

o Update operator

o Stream operator

In most cases a load utility job for each of these utilities can be re-started after a database failure and reset, a load process

error or failure or a load utility client platform failure after the underlying problem has been corrected.

Active Enterprise Intelligence Ecosystem

An Active Enterprise Intelligence Ecosystem consists of not only the Teradata database subsystem but all of the

surrounding subsystems that require service from the database subsystem and provide services to the database

subsystem. The following diagram illustrates the major subsystems in an AEI Ecosystem.

Each subsystem must have error and failure handling capabilities. This series covers the capabilities and approaches to

handle database, data integration and application integration errors and failures. This series will focus on handling errors,

exceptions and failures in a Data Integration subsystem that is implemented with Teradata load utilitities.

Errors, Exceptions and Failures

A system can encounter multiple types of errors and failures. Error and failure handling processes must execute the

following steps:

1. Identify error or failure condition

2. Classify error or failure

3. Log error or failure

4. Notify appropriate personnel

5. Fix error or failure condition

6. Return system to service

7. Restart or re-execute operation that was executing when error or exception occurred.

The first step is to identify that an error or failure has occurred. In most cases a failure of the database will be detected by

either a data load client or a data access client during a database operation. In this case the error must be classified and

reported by the database client. In other cases the client may fail during a database operation. In each case the error must

be classified.

Errors in the database or a database client can be classified as:

1. System Error – System errors are an error or failure in the processing platform, storage or system software. For example,

database amp failures, node failures, disk failures or storage cabinet failures would be system failures. In most cases the

Teradata database will isolate the failed components and can be restarted. In many cases a database operation can be

restarted where it was when the error occurred.

2. Configuration Error – A configuration error is caused by a system configuration setting and can be corrected by changing or

adjusting the configuration. For example, the database may report an out of space error when the perm space or spool

space does not provide sufficient storage for a database operation. In many cases this type of error can be corrected and

the database operation can be restarted.

3. Programming Error – Programming errors represent errors in data load scripts or programs that are executing data access

logic. In most cases the error is detected before a database operation has interacted with the database. In this case the

programming error can be corrected and the operation can be re-run.

4. Data Error – A data error or exception represents an error in the structure or value of data that is used to query or update a

database. For example an error may be reported when a query to return a row based on a primary key can not find the

associated row or an insert attempts to insert a duplicate row. In this case, the data values must be corrected and the

database operation re-executed.

5. Application Logic Error – An application logic error represents a violation of application logic during a database operation. In

most cases the conditions that cause the error must be corrected and the operation must be re-executed. For example, a

database interaction may attempt to increase a salary attribute by 25%. The application logic may restrict increases to

20%. This would cause an application logic error.

The ability to restart Teradata load utility operations that are interrupted by an error or exception is a key feature of these

utilities. Care must be taken to identify the error properly to determine whether a restart is possible and to configure the

utility scripts properly for restarts.

It is critical that errors and exceptions be identified and classified. This is necessary to determine whether the problem can

be isolated and the system returned to service automatically using Teradata high availability features or if the error

requires intervention. In the case of intervention the type of error will determine whether the intervention is by a DBA,

programmer or Teradata support personnel.

Teradata load utilities have robust error handling and restart capabilities. This allows a database operation that is

interrupted by a system error or by excessive data errors to be restarted and not have to be completely re-executed. A

data integration operation may be interrupted by database errors, script errors, data errors or failures of a client process or

platform. The Teradata load utilities provide the capability to restart a load operation in all of these cases.

Data integration operations are usually implemented as jobs that contain a series of steps. The steps may be defined in a

single Teradata load utility script or as series of steps that execute load utilities or SQL scripts. In each case the job should

be step restartable. That means that after an error the job should be able to be restarted at the point or step that was

executing when the error occurred. In addition, each step should be able to be restarted or re-run. A restarted step will not

re-execute the database interactions that occurred before the error. A step that is re-run will re-execute all database

interactions in that step and continue with succeeding steps.

Jobs can be defined and executed using job scheduling and control systems (eg. Control-M, Tivoli, AutoSys), data

integration systems (eg. Informatica, Oracle ODI, DataStage), or with scripts (eg. shell scripts, perl). In each case the job

scheduling and control system should allow a job to be started at a step that was executing when an error occurred and it

should allow a step to either be restarted or re-run.

Teradata utilities allow steps defined within the utility scripts to start at the step where an error occurred or to be rerun

completely. They also allow a job interrupted by a database reset to continue automatically once the database is back in

service.

The utilities allow checkpoints to be established to prevent the re-execution of database operations that have completed

before an error caused an interruption.

Each of the load utilities allows parameters to be set that control:

1. Tenacity – Tenacity define the length time an execution of a utility will attempt to establish a database connection

2. Checkpoint interval – A checkpoint interval defines the frequency (either in time or records processed) when actions by the

utility are marked as successful. This allows a restart to continue after the last successfully action

3. Error Limit – The Error Limit parameter defines the maximum number of errors (usually data errors) that may occur before

the load utility terminates due to errors.

All of the Teradata load utilities provide a return or completion code to the operation system or program that executed the

utility. All job control definitions should check the return codes and take the appropriate action based on the return code.

The Teradata utilities return the following completion codes:

0 – Normal completion. All steps executed successfully.

4 – Warning. A warning condition occurred; for example, a job deviated from normal or from the specified plan, but still

completed successfully. A warning may indicate deviation from the plan; for example, the number of sessions specified

were not actually used, or a part of the job did not run. Warning conditions do not terminate the job.

8 – User error. A user error is usually a programming or configuration error in the load utility script. User error conditions

terminate the job. In most cases the programming or script error can be corrected and the load utility job can be re-started.

12 - Severe error. A fatal error terminated the job. A fatal error is any error other than a user error. Severe errors terminate

a load utility job. In most case the error can be corrected and the load utility restarted at the point where the failure

occurred.

In each case the specific database error must be found in the output log of the load utility to determine the next course of

action (fix, re-start, re-run). A job control system can parse the output logs to determine and classify the database errors

(beyond the scope of this series of articles). In all cases the underlying error must be identified, classified, fixed and then

the job restarted at the appropriate step.

In the following series of articles we will describe how to use the features of Teradata load utilities to handle database and

client errors and failures.

The Teradata Multiload utility provides the capability to perform batch maintenance on tables (insert, update, delete and upsert). It loads data from external sources and provides the capability to restart jobs interrupted by errors, exceptions and failures.

A Multiload job executes in 5 phases:

1. Preliminary

Parses and validates all of the MultiLoad commands and Tera-data SQL statements in your MultiLoad job

Establishes sessions and process control with the Teradata Database

Submits special Teradata SQL requests to the Teradata Data-base

Creates and protects temporary work tables and error tables in the Teradata Database

2. DML Transaction

Submits the DML statements specifying the insert, update, and delete tasks to the Teradata Database

3. Acquisition

Imports data from the specified input data source

Evaluates each record according to specified application con-ditions

Loads the selected records into the worktables in the Tera-data Database

There is no acquisition phase activity for a MultiLoad delete task.

4. Application

Acquires locks on the specified target tables and views in the Teradata Database

For an import task, inserts the data from the temporary work tables into the target tables or views in the Teradata Data-base

For a delete task, deletes the specified rows from the target table in the Teradata Database

Updates the error tables associated with each MultiLoad task

5. Cleanup

Forces an automatic restart/rebuild if an AMP went offline and came back online during the application phase

Releases all locks on the target tables and views

Drops the temporary work tables and all empty error tables from the Teradata Database

Reports the transaction statistics associated with the import and delete tasks.

Multiload returns the returns codes described previously (0, 4, 8, 12) to the operating system or calling program.

Restarts

It is possible to restart a Multiload job in any of the phases after an er-ror or failure has interrupted an executing Multiload job. There are differences in the restart procedures in some of the phases that will be described. It is also possible to rerun an interrupted Multiload job in some of the phases which will also be discussed.

A Multiload job creates a restart log table that must be present for a job to be restarted and must be dropped before a job can be rerun. It creates a work table for each input source. These tables must be pre-sent for a job to restart and must be dropped to rerun a job. Multiload also creates two error tables for each target table that must be present for a restart and that must be dropped to rerun a Multiload job.

A Multiload job can not be restarted or rerun if these tables are not present for an interruption that occurs during the APPLICATION PHASE.

Checkpoint frequency can be set for checkpoint to occur after a num-ber of records have been read from an input source. The checkpoints are taken during the acquisition phase. A restart after an interruption in the acquisition phase will start reading the input source at the point following the last checkpoint executed before the interruption.

Interruptions that occur during the application phase will be restarted based on internal checkpoints created by the database.

The following is a simple Multiload script that will be use to illustrate the Multiload restart capabilities.

?

code

1

2

3

4

.logtable test.tranmllog;

.LOGON test/testuser,test;

.begin import

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

mload tables test.tran

checkpoint 20000;

.layout tranlayout;

.FIELD STORE * varchar(10);

.field TRAN_NO * varchar (10);

.field TRAN_DATE * varchar (10);

.field TRAN_TIME * varchar (10);

.field TRAN_TYPE * varchar (1);

.field TRAN_TERMINAL * varchar (3);

.field CLERK * varchar (10);

.field CUST_NO * varchar (10);

.field NAME * varchar (30);

.field TOTAL_SALE * varchar (14);

.field SALES_TAX * varchar (14);

.field DISCOUNT * varchar (14);

.field DISC_PCT * varchar (10);

.field DISC_TAKEN_FLAG * varchar (1);

.field CASH_TENDERED * varchar (14);

.field CHANGE_GIVEN * varchar (14);

.field CHARGE_TENDERED* varchar (14);

.field CHECK_NUMBER * varchar (20);

.field CHECK_TENDERED * varchar (14);

.field BANKCARD_TENDERED * varchar (14);

.field PURCHASE_ORDER_NUM * varchar (20);

30

31

32

33

34

35

36

37

38

39

.dml label insdml;

insert into test.tran.*;

.import infile C:\DataIntegration\Data\TRAN2.DAT

format vartext '|'

layout tranlayout

apply insdml;

.end mload;

.logoff;

This script inserts rows into one table, “TRAN”. It set a checkpoint every 20000 input records.

Restart after database reset

The following is an example of an automatic restart that occurs for an interruption and reset that occurred during the acquisition phase.

?

code

1

2

3

4

5

6

7

8

9

C:\DataIntegration\Scripts>mload -b < tran.ml

**** 22:11:09 UTY2414 BRIEF option is enabled.

========================================================================

= MultiLoad Utility Release MLOD.13.00.00.000 =

= Platform WIN32 =

========================================================================

= Copyright 1990-2009, Teradata Corporation. ALL RIGHTS RESERVED. =

========================================================================

**** 22:11:09 UTY2411 Processing start date: SUN AUG 09, 2009

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

========================================================================

= Logon/Connection =

========================================================================

0001 .logtable test.tranmllog;

0002 .LOGON test/testuser,;

**** 22:11:11 UTY8400 Teradata Database Release: 12.00.01.24

**** 22:11:11 UTY8400 Teradata Database Version: 12.00.01.24

**** 22:11:11 UTY8400 Default character set: ASCII

**** 22:11:11 UTY8400 Current RDBMS has interval support

**** 22:11:11 UTY8400 Current RDBMS has UDT support

**** 22:11:11 UTY8400 Maximum supported buffer size: 1M

**** 22:11:11 UTY8400 Data Encryption supported by RDBMS server

**** 22:11:12 UTY6211 A successful connect was made to the RDBMS.

**** 22:11:12 UTY6217 Logtable 'test.tranmllog' has been created.

0003 .begin import


checkpoint 20000;

========================================================================

= Processing MultiLoad Statements =

========================================================================

0004 .layout tranlayout;

0005 .FIELD STORE * varchar(10);

0006 .field TRAN_NO * varchar (10);

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

0007 .field TRAN_DATE * varchar (10);

0008 .field TRAN_TIME * varchar (10);

0009 .field TRAN_TYPE * varchar (1);

0010 .field TRAN_TERMINAL * varchar (3);

0011 .field CLERK * varchar (10);

0012 .field CUST_NO * varchar (10);

0013 .field NAME * varchar (30);

0014 .field TOTAL_SALE * varchar (14);

0015 .field SALES_TAX * varchar (14);

0016 .field DISCOUNT * varchar (14);

0017 .field DISC_PCT * varchar (10);

0018 .field DISC_TAKEN_FLAG * varchar (1);

0019 .field CASH_TENDERED * varchar (14);

0020 .field CHANGE_GIVEN * varchar (14);

0021 .field CHARGE_TENDERED* varchar (14);

0022 .field CHECK_NUMBER * varchar (20);

0023 .field CHECK_TENDERED * varchar (14);

0024 .field BANKCARD_TENDERED * varchar (14);

0025 .field PURCHASE_ORDER_NUM * varchar (20);

0026 .dml label insdml;

0027 insert into test.tran.*;

0028 .import infile C:\DataIntegration\Data\TRAN2.DAT

format vartext '|'

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

layout tranlayout

apply insdml;

0029 .end mload;

========================================================================

= MultiLoad Initial Phase =

========================================================================

**** 22:11:12 UTY0829 Options in effect for this MultiLoad import task:

. Sessions: One session per available amp.

. Checkpoint: 20000 records.

. Tenacity: 4 hour limit to successfully connect load sessions.

. Errlimit: No limit in effect.

. AmpCheck: In effect for apply phase transitions.

**** 22:11:13 UTY0815 MLOAD session(s) connected: 2.

========================================================================

= MultiLoad Acquisition Phase =

========================================================================

**** 22:11:16 UTY1812 A checkpoint is being initiated because 20000 output recor

ds have been

sent to the RDBMS.

**** 22:11:17 UTY0827 A checkpoint has been taken, recording that input record 2

0000 has been

processed for IMPORT 1 of this MultiLoad Import task.


ds have been

sent to the RDBMS.

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

10

**** 22:11:43 UTY0817 MultiLoad submitting the following request:

CHECKPOINT LOADING INTERVAL 2;

**** 22:11:43 UTY0806 RDBMS error, 2825: NO RECORD OF THE LAST REQUEST WAS FOUND

AFTER DBC

RESTART

**** 22:11:43 UTY1804 Re-starting this MultiLoad task because The RDBMS crashed.

========================================================================


========================================================================








========================================================================


========================================================================

**** 22:11:46 UTY1806 This MultiLoad Import task is re-starting from checkpoint

taken at

record 20000 of IMPORT 1.


ds have been

6

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

sent to the RDBMS.


0000 has been



ds have been

sent to the RDBMS.


0000 has been


**** 22:11:51 UTY0826 A checkpoint has been taken, recording that end of file ha

s been reached

for IMPORT 1 of this MultiLoad Import task.

**** 22:11:51 UTY1803 Import processing statistics

. IMPORT 1 Total thus far

. ========= ==============

Candidate records considered:........ 67817....... 67817

Apply conditions satisfied:.......... 67817....... 67817

Candidate records not applied:....... 0....... 0

Candidate records rejected:.......... 0....... 0

========================================================================

= MultiLoad Application Phase =

========================================================================

**** 22:12:10 UTY0818 Statistics for table test.tran:

Inserts: 67817

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

Updates: 0

Deletes: 0

========================================================================

= MultiLoad Task Cleanup =

========================================================================

**** 22:12:10 UTY0821 Error table test.ET_tran is EMPTY, dropping table.

**** 22:12:11 UTY0821 Error table test.UV_tran is EMPTY, dropping table.

**** 22:12:11 UTY0825 Error table statistics for:

Target table 1: test.tran

Number of Rows Error Table Name

============== ========================================================

0 test.ET_tran

0 test.UV_tran

**** 22:12:12 UTY0822 MultiLoad processing complete for this MultiLoad import ta

sk.

========================================================================

= MultiLoad Task Complete =

========================================================================

0030 .logoff;

========================================================================

= Logoff/Disconnect =

========================================================================

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

15

**** 22:12:12 UTY6216 The restart log table has been dropped.

**** 22:12:12 UTY6212 A successful disconnect was made from the RDBMS.

**** 22:12:12 UTY2410 Total processor time used = '1.625 Seconds'

. Start : 22:11:09 - SUN AUG 09, 2009

. End : 22:12:12 - SUN AUG 09, 2009

. Highest return code encountered = '0'.

3

154

155

156

157

158

159

160

161

162

163

164

165

An automatic restart may also occur if an interruption occurs during the application phase. The following output illustrates this type of re-start.

?

code

1 C:\DataIntegration\Scripts>mload -b < tran.ml

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26


========================================================================


= Platform WIN32 =

========================================================================


========================================================================


========================================================================


========================================================================












0003 .begin import


27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

checkpoint 20000;

========================================================================


========================================================================





















52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76






format vartext '|'

layout tranlayout

apply insdml;

0029 .end mload;

========================================================================


========================================================================








========================================================================


========================================================================


77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

10

ds have been

sent to the RDBMS.


0000 has been



ds have been

sent to the RDBMS.


0000 has been



ds have been

sent to the RDBMS.


0000 has been



s been reached




. ========= ==============



1

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116



========================================================================


========================================================================

**** 22:24:19 UTY0817 MultiLoad submitting the following request:

EXEC MLOAD test.tran;

**** 22:24:19 UTY0805 RDBMS failure, 2828: Request was rolled back during system

recovery.

**** 22:24:19 UTY1804 Re-starting this MultiLoad task because The RDBMS crashed.

========================================================================


========================================================================








========================================================================


========================================================================


117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

Inserts: 67817

Updates: 0

Deletes: 0

========================================================================


========================================================================






============== ========================================================

0 test.ET_tran

0 test.UV_tran


sk.

========================================================================


========================================================================

0030 .logoff;

========================================================================


133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

14

========================================================================




. Start : 22:22:57 - SUN AUG 09, 2009

. End : 22:24:29 - SUN AUG 09, 2009


8

149

150

151

152

153

154

155

156

157

158

Restarts after client failure

A Multiload job may be restarted after it is interrupted by a failure of the Multiload process or the client processing platform. This type of er-ror may be a system error or can be a programming error or data er-ror. The system can be restarted by correcting the problem and re-running the Multiload job. The following output is an example of a re-start for a failure of a Multiload process during the acquisition phase.

?

code

1 C:\DataIntegration\Scripts>mload -b < tran.ml

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26


========================================================================


= Platform WIN32 =

========================================================================


========================================================================


========================================================================


========================================================================












0003 .begin import


27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

checkpoint 20000;

========================================================================


========================================================================





















52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76






format vartext '|'

layout tranlayout

apply insdml;

0029 .end mload;

========================================================================


========================================================================








========================================================================


========================================================================

**** 22:35:57 UTY1812 A checkpoint is being initiated because 20000 output reco

77

78

79

80

81

82

83

84

85

86

87

88

89

90

ds have been

sent to the RDBMS.

**** 22:35:58 UTY0827 A checkpoint has been taken, recording that input record

0000 has been



ds have been

sent to the RDBMS.

**** 22:36:00 UTY0827 A checkpoint has been taken, recording that input record

0000 has been



ds have been

sent to the RDBMS.

The gateway sessions associated with the Multiload job will remain ac-tive for the gateway timeout time (20 minutes default). These sessions can be killed using the gtwglobal utility in the same manner that was described for the Fastload utility.

The interrupted Multiload job can be restarted by re-executing the job. The following output is from a re-execution of the example Multiload job.

?

code

1

2

3

4

5



========================================================================


= Platform WIN32 =

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

========================================================================


========================================================================


========================================================================


========================================================================











**** 22:42:38 UTY6210 Logtable 'test.tranmllog' indicates that a restart is in p

rogress.

0003 .begin import


checkpoint 20000;

========================================================================


31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

========================================================================























56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80




format vartext '|'

layout tranlayout

apply insdml;

0029 .end mload;

========================================================================


========================================================================








========================================================================


========================================================================

**** 22:42:40 UTY1806 This MultiLoad Import task is re-starting from checkpoint

taken at

record 40000 of IMPORT 1.


81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

ds have been

sent to the RDBMS.


0000 has been



s been reached




. ========= ==============





========================================================================


========================================================================


Inserts: 67817

Updates: 0

Deletes: 0

========================================================================


========================================================================

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119






============== ========================================================

0 test.ET_tran

0 test.UV_tran


sk.

========================================================================


========================================================================

0030 .logoff;

========================================================================


========================================================================




. Start : 22:42:35 - SUN AUG 09, 2009

. End : 22:42:59 - SUN AUG 09, 2009

120

121

122

123

124

125

126

127

128

129

130

131


A Multiload job may also be restarted if interrupted during the application phase.

It is important to note that the restart log and error tables must be available. If they are not the Multiload job can not be restarted or rerun. In this case the table must be reloaded from backup or source because they are unusable.

The following output is an example of a Multiload job restarted after an interruption during the application phase.

?

code

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24



========================================================================


= Platform WIN32 =

========================================================================


========================================================================


========================================================================


========================================================================











**** 22:53:19 UTY6210 Logtable 'test.tranmllog' indicates that a restart is in p

rogress.

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

0003 .begin import


checkpoint 20000;

========================================================================


========================================================================


















50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74









format vartext '|'

layout tranlayout

apply insdml;

0029 .end mload;

========================================================================


========================================================================








========================================================================

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99


========================================================================


Inserts: 67817

Updates: 0

Deletes: 0

========================================================================


========================================================================






============== ========================================================

0 test.ET_tran

0 test.UV_tran


sk.

========================================================================


========================================================================

100

101

102

103

104

105

106

107

108

109

0030 .logoff;

========================================================================


========================================================================




. Start : 22:53:17 - SUN AUG 09, 2009

. End : 22:53:27 - SUN AUG 09, 2009


Rerun

Multiload jobs that are interrupted before the application phase starts or after the application phase ends may be rerun. Jobs that are inter-rupted during the application phase must be restarted.

To rerun an entire Multiload job the following actions must be taken before the job is rerun:

Use BTEQ to execute RELEASE MLOAD command for each target table in Multiload script

Drop work table for each target table

Drop acquisition error table for each target table

Drop application error table for each target table

Drop restart log table

These actions can be executed by a BTEQ SQL script that is run as part of a Multiload script or run by an external job scheduling system.

The following is an example of a SQL script that will allow this example to be rerun.

?

code

1

2

3

4

5

6

7

8

9

10

11

12

.logon test/testuser,test

database test;

release mload tran;

drop table uv_tran;

drop table et_tran;

drop table wt_tran;

drop table log_tran;

.logoff

.quit

This script can be executed as part of a Multiload script that executes this script with BTEQ and then executes the Multiload job as illustrated in the following Multiload script.

?

code

1

2

3

4

.system 'bteq < c:\DataIntegration\scripts\tranreset.sql';

.run file c:\DataIntegration\scripts\tranload.ml;

.logoff;

This script can always be run when the intent is to run the entire Multi-load job.

In the next article in this series we will look at the Tpump Utility

In the first article in this series on Teradata Error Handling, we introduced the larger architecture and load utility environment. in this article the focus will be on the Fastload Utility and some of the techniques applicable to handling errors, exceptions and failures that can occur within a High Availability system.

Fastload

The Teradata Fastload utility loads data into empty tables. It has the ability to be restarted after a database reset or after a client process or platform failure. A Fastload job step should provide the capability to either restart at the point of failure or to completely rerun the step.

Restarts

Fastload will automatically restart a job after a database failure after the cause of the failure has been fixed and the database has reset. The database configuration must be exactly the same as when the Fastload job was initiated and the Fastload error tables must not have been de-leted. When these conditions are not true the Fastload job must be re-executed from the beginning.

A checkpoint interval can be established as part of the BEGIN LOADING command. This allows a restart to continue reading an input file at the point after the last checkpoint taken before a failure or inter-ruption occurs. The checkpoint interval is set as number of records read for each checkpoint.

When a database reset occurs the Fastload utility will continue to try to re-establish a database connection. When a connection is established the utility will continue the interrupted job automatically. If a check-point has been set it will continue at the point in the input file where the last checkpoint was taken. Specifying a checkpoint interval will de-crease the time needed for a restart but there are tradeoffs. Using checkpoints can increase the execution time of a Fastload job. The following factors should be considered when specifying a checkpoint in-terval:

Each checkpoint temporarily halts the multiple session data transfer feature of Teradata Fastload, thereby decreasing the speed of the Teradata Fastload job.

For each checkpoint, Teradata Fastload waits for each session to complete sending its current request, which interrupts the data transfer operation.

The record size and the size of the Teradata Database influence how often you should specify checkpoints. On a smaller Teradata Database, specify checkpoints:

Every 50,000 records if each record is more then 4 KB

Every 100,000 records if each record is less than 4 KB

On a larger Teradata Database, specify higher (less frequent) checkpoint values.

The procedure used and the Teradata Fastload response to restarting a paused Teradata Fastload job depends on the phase that the Teradata Fastload job was in when it was paused.

A Fastload job that was paused during loading can be can be restarted, either from the beginning or from the most recent checkpoint if the BEGIN LOADING command specified the checkpoint option. A job that was paused during end loading phase will restart from wherever it was interrupted. This is because the Teradata Database uses internal checkpointing during this phase.

A Fastload job that terminates because of a Fastload script error may be restarted after the error is corrected. Commands that have exe-cuted successfully must not be changed in the script. This will cause the restart to fail. If commands that have successfully executed have to be changed the entire Fastload job must be re-executed.

Restart After Database Reset

The following is an example of an automatic Fastload restart after a database reset has been initiated to fix a database error condition.

The Fastload script for this example follows:

?

code

1

2

3

4

5

6

7

8

9

10

11

12

LOGON test/testuser,test;

SET RECORD FORMATTED;

DEFINE FILE=C:\DataIntegration\Data\tran1.dat;

SHOW;

DROP TABLE TEST.TRAN_ERR;

DROP TABLE TEST.TRAN;

DROP TABLE TEST.TRANFLERR1;

DROP TABLE TEST.TRANFLERR2;

CREATE SET TABLE TEST.TRAN ,NO FALLBACK ,

NO BEFORE JOURNAL,

NO AFTER JOURNAL,

CHECKSUM = DEFAULT

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

(

STORE INTEGER,

TRAN_NO CHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC NOT NULL,

TRAN_DATE DATE FORMAT 'MM/DD/YYYY',

TRAN_TIME TIME(6),

TRAN_TYPE CHAR(1) CHARACTER SET LATIN NOT CASESPECIFIC,

TRAN_TERMINAL CHAR(3) CHARACTER SET LATIN NOT CASESPECIFIC,

CLERK VARCHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC,

CUST_NO CHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC,

NAME VARCHAR(30) CHARACTER SET LATIN NOT CASESPECIFIC,

TOTAL_SALE DECIMAL(10,2),

SALES_TAX DECIMAL(10,2),

DISCOUNT DECIMAL(10,2),

DISC_PCT DECIMAL(5,3),

DISC_TAKEN_FLAG CHAR(1) CHARACTER SET LATIN NOT CASESPECIFIC,

CASH_TENDERED DECIMAL(10,2),

CHANGE_GIVEN DECIMAL(10,2),

CHARGE_TENDERED DECIMAL(10,2),

CHECK_NUMBER VARCHAR(20) CHARACTER SET LATIN NOT CASESPECIFIC,

CHECK_TENDERED DECIMAL(10,2),

BANKCARD_TENDERED DECIMAL(10,2),

PURCHASE_ORDER_NUM VARCHAR(20) CHARACTER SET LATIN NOT CASESPECIFIC)

UNIQUE PRIMARY INDEX ( TRAN_NO );

CREATE ERROR TABLE TEST.TRAN_ERR FOR TEST.TRAN;

38

39

40

41

42

43

BEGIN LOADING TEST.TRAN

ERRORFILES TEST.TRANFLERR1, TEST.TRANFLERR2

CHECKPOINT 50000

INDICATORS;

INSERT TEST.TRAN.*;

END LOADING;

LOGOFF;

This script drops the target table and the error tables if they exist. It creates the target table and executes the load for the target table. A checkpoint will be taken every 50000 input records.

The following is the output from a Fastload job that was restarted automatically after a database reset occurred during the execution of the job.

?

code

1

2

3

4

5

6

7

8

9

10

11

12

13

===================================================================

= =

= FASTLOAD UTILITY VERSION 12.00.00.004 =

= PLATFORM WIN32 =

= =

===================================================================

===================================================================

= =

= Copyright 1984-2007, Teradata Corporation. =

= ALL RIGHTS RESERVED. =

= =

===================================================================

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

**** 16:14:34 Processing starting at: Sun Aug 09 16:14:34 2009

===================================================================

= =


= =

===================================================================

0001 LOGON test/testuser,

**** 16:14:37 Teradata Database Release: 12.00.01.24

**** 16:14:37 Teradata Database Version: 12.00.01.24

**** 16:14:37 Current CLI or RDBMS allows maximum row size: 64K

**** 16:14:37 Character set for this job: ASCII

**** 16:14:38 Number of FastLoad sessions connected = 2

**** 16:14:38 FDL4808 LOGON successful

0002 SET RECORD FORMATTED;

**** 16:14:38 Now set to read 'FORMATTED' records

**** 16:14:38 Command completed successfully

0003 DEFINE FILE=C:\DataIntegration\Data\tran1.dat;

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

**** 16:14:38 FDL4803 DEFINE statement processed

0004 SHOW;

FILE = C:\DataIntegration\Data\tran1.dat

TOTAL RECORD LENGTH = 0

0005 DROP TABLE TEST.TRAN_ERR;


0006 DROP TABLE TEST.TRAN;


0007 DROP TABLE TEST.TRANFLERR1;

**** 16:14:38 RDBMS error 3807: Object 'TEST.TRANFLERR1' does not exist.

0008 DROP TABLE TEST.TRANFLERR2;

**** 16:14:38 RDBMS error 3807: Object 'TEST.TRANFLERR2' does not exist.

0009 CREATE SET TABLE TEST.TRAN ,NO FALLBACK ,

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

NO BEFORE JOURNAL,

NO AFTER JOURNAL,

CHECKSUM = DEFAULT

(

STORE INTEGER,

TRAN_NO CHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC NOT NULL,

TRAN_DATE DATE FORMAT 'MM/DD/YYYY',

TRAN_TIME TIME(6),

TRAN_TYPE CHAR(1) CHARACTER SET LATIN NOT CASESPECIFIC,

TRAN_TERMINAL CHAR(3) CHARACTER SET LATIN NOT CASESPECIFIC,

CLERK VARCHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC,

CUST_NO CHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC,

NAME VARCHAR(30) CHARACTER SET LATIN NOT CASESPECIFIC,

TOTAL_SALE DECIMAL(10,2),

SALES_TAX DECIMAL(10,2),

DISCOUNT DECIMAL(10,2),

DISC_PCT DECIMAL(5,3),

DISC_TAKEN_FLAG CHAR(1) CHARACTER SET LATIN NOT CASESPECIFIC,

CASH_TENDERED DECIMAL(10,2),

CHANGE_GIVEN DECIMAL(10,2),

CHARGE_TENDERED DECIMAL(10,2),

CHECK_NUMBER VARCHAR(20) CHARACTER SET LATIN NOT CASESPECIFIC,

CHECK_TENDERED DECIMAL(10,2),

BANKCARD_TENDERED DECIMAL(10,2),

PURCHASE_ORDER_NUM VARCHAR(20) CHARACTER SET LATIN NOT

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

CASESPECIFI

UNIQUE PRIMARY INDEX ( TRAN_NO );


0010 CREATE ERROR TABLE TEST.TRAN_ERR FOR TEST.TRAN;


0011 BEGIN LOADING TEST.TRAN


CHECKPOINT 50000

INDICATORS;

**** 16:14:38 Indicator mode is set to ON

**** 16:14:38 Number of AMPs available: 2

**** 16:14:38 BEGIN LOADING COMPLETE

===================================================================

= =

= Insert Phase =

= =

===================================================================

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

0012 INSERT TEST.TRAN.*;

**** 16:14:39 Number of recs/msg: 316

**** 16:14:39 Starting to send to RDBMS with record 1

**** 16:14:39 Sending row 50000

**** 16:14:40 Sending row 100000

**** 16:14:41 Sending row 150000

**** 16:14:42 Sending row 200000

**** 16:14:42 Sending row 250000

**** 16:15:08 RDBMS error 2825: NO RECORD OF THE LAST REQUEST WAS FOUND

AFTER DBC RESTART

**** 16:15:08 FastLoad will attempt to retry this request 5 time(s)







**** 16:15:10 The last checkpoint was taken at row: 250000

**** 16:15:10 FastLoad will now restart at row: 250001

**** 16:15:11 Sending row 300000

**** 16:15:12 Sending row 350000

**** 16:15:13 Sending row 400000

**** 16:15:14 Sending row 450000

**** 16:15:15 Sending row 500000

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

**** 16:15:16 Sending row 550000

**** 16:15:17 Sending row 600000

**** 16:15:18 Sending row 649472

**** 16:15:18 Finished sending rows to the RDBMS

===================================================================

= =

= End Loading Phase =

= =

===================================================================

0013 END LOADING;

**** 16:15:42 END LOADING COMPLETE

Total Records Read = 649472

Total Error Table 1 = 0 ---- Table has been dropped


Total Inserts Applied = 649472

Total Duplicate Rows = 0

Start: Sun Aug 09 16:15:19 2009

End : Sun Aug 09 16:15:42 2009

164

165

166

167

168

169

170

171

172

173

174

175

176

0014 LOGOFF;

===================================================================

= =


= =

===================================================================

**** 16:15:43 Logging off all sessions

**** 16:15:44 Total processor time used = '2.64063 Seconds'

. Start : Sun Aug 09 16:14:34 2009

. End : Sun Aug 09 16:15:44 2009


**** 16:15:44 FDL4818 FastLoad Terminated

Restart after client failure

A Fastload job can be restarted after a client process failure. In this case the error tables and target tables must be in the state they were in when the failure occurred, therefore, a restart script must not in-clude the actions to drop error tables, drop target table and create target table. The following is an example of the restart script.

?

code

1

2

3

4

5

LOGON test/testuser,test;

SET RECORD FORMATTED;

DEFINE FILE=C:\DataIntegration\Data\tran1.dat;

SHOW;

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

BEGIN LOADING TEST.TRAN


CHECKPOINT 50000

INDICATORS;

INSERT TEST.TRAN.*;

END LOADING;

LOGOFF;

In this example a client failure occurs while a Fastload job is execut-ing.

===================================================================

= =

= Insert Phase =

= =

===================================================================



**** 16:33:43 Starting to send to RDBMS with record 1

**** 16:33:44 Sending row 50000

**** 16:33:45 Sending row 100000

**** 16:33:46 Sending row 150000

**** 16:33:47 Sending row 200000

**** 16:33:47 Sending row 250000

**** 16:33:48 Sending row 300000

31

32

**** 16:33:49 Sending row 350000

**** 16:33:50 Sending row 400000

When the client failure occurs the Teradata database will log off the Fastload sessions, however, on a network attached client a gateway session will exist until a timeout has expired and the gateway session is cancelled. The default timeout period is 20 minutes. During this time the Fastload job can not be restarted and will return the following error.

?

code

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

C:\DataIntegration\Scripts>fastload <tranrestart.fl ===================================================================

= =


= PLATFORM WIN32 =

= =

===================================================================

===================================================================

= =



= =

===================================================================


===================================================================

= =


= =

===================================================================

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43













0004 SHOW;





CHECKPOINT 50000

INDICATORS;


**** 16:34:29 RDBMS error 2738: TRAN already has Fastload running

===================================================================

= =


= =

44

45

46

47

48

49

50

===================================================================



. Start : Sun Aug 09 16:34:26 2009

. End : Sun Aug 09 16:34:29 2009


**** 16:34:29 FDL4818 Fastload Terminated

The Fastload job can be restarted before the Gateway session timeout if the Gateway sessions for the failed Fastload job are killed. This can be done using the gtwglobal utility in the following way.

?

code

1

2

3

4

5

6

7

8

9

10

11

12

13

C:\Documents and Settings\john>gtwglobal

_______

| | |

| ___ __ ____ | ____ __|__ ____

| / |/ \ ____| ____| ____| | ____|

| --- | / | / | / | | / |

| \___ | \____| \____| \____| |__ \____|

Enter gateway command or enter h for Help:

select host 1

Host 1 has been selected.

14

15

16

17

18

Enter gateway command or enter h for Help:

1>kill user testuser

User TESTUSER has 4 sessions killed

1059 1060 1061 1062

Once the Fastload sessions have been killed the Fastload job can be restarted. The following is an example of the output from the restarted Fastload job after a client failure.

?

code

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

C:\DataIntegration\Scripts>fastload <tranrestart.fl

===================================================================

= =


= PLATFORM WIN32 =

= =

===================================================================

===================================================================

= =



= =

===================================================================


17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

===================================================================

= =


= =

===================================================================













42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

0004 SHOW;





CHECKPOINT 50000

INDICATORS;


**** 16:53:57 FastLoad is restarting

**** 16:53:57 Number of AMPs available: 2

**** 16:53:57 BEGIN LOADING COMPLETE

===================================================================

= =

= Insert Phase =

= =

===================================================================



67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

**** 16:53:57 The last checkpoint was taken at row: 400000

**** 16:53:57 FastLoad will now restart at row: 400001

**** 16:53:58 Sending row 450000

**** 16:53:59 Sending row 500000

**** 16:54:00 Sending row 550000

**** 16:54:01 Sending row 600000

**** 16:54:01 Sending row 649472

**** 16:54:01 Finished sending rows to the RDBMS

===================================================================

= =

= End Loading Phase =

= =

===================================================================

0007 END LOADING;

**** 16:54:21 END LOADING COMPLETE

Total Records Read = 649472



Total Inserts Applied = 649472

Total Duplicate Rows = 0

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

Start: Sun Aug 09 16:54:02 2009

End : Sun Aug 09 16:54:21 2009

0008 LOGOFF;

===================================================================

= =


= =

===================================================================



. Start : Sun Aug 09 16:53:54 2009

. End : Sun Aug 09 16:54:22 2009


**** 16:54:22 FDL4818 FastLoad Terminated

Rerun

In some cases a Fastload job may need to be re-executed when the error tables or target tables are not available or the database configuration has changed. In the previous example the original Fastload job can be rerun because it drops the error and target tables and recreates the target table before loading the data.

In the next article in this series we will look at the Multiload Utility.

A high

Documents

Transcript of A high