1 Job sequences

Basically a job sequence is used to specify a sequence of Data Stage server and parallel jobs to be executed, and the respected actions to take depending on results

Example:

The sample job sequence shows a sequence that will run the job try1. If try1 runs successfully, the Success trigger (try1ok) causes the try2 job to run. If any of the jobs fail, Failure trigger (try1failed, try2failed) of both these jobs

causes the alarm job to run and initiate the update_job_abort job to run.

The main entities involved are o Job activity stage

o Routine stage

o Exec command

o Email Notification

o Wait for file activity

o Run activity on exception

Note: Email Notification, Wait for file activity, Run activity on exception are not covered.

To add an activity to your job sequence, drag the corresponding icon from the tool palette and drop it on the Diagram window.

1.1 Job Activity Stage

1.1.1 Purpose:

This stage specifies a Data stage job. This stage allows one to specify which job needs to be added to the present sequence to be executed in a defined order.

1.1.2 Creation:

There are two ways of doing it

Just drag and drop the required job from the repository window. Drag and drop the icon from the palette and later choose which job it needs to

be referring to.

1.1.3 Description:

There are 3 main pages in the Job Activity properties page.

o Generalo Job

o Triggers

General page: It allows one to specify the general information about the Job.

Job Page:

Job Name Dropdown: Allows the user to specify the name of the job the activity is

to run. User can select a job by browsing the Repository. If user has added the activity by dragging a job from the Repository window, the Job name will already be filled in.

Execution action: Allows you to specify what the activity will do with the job. Choose one of the following from the drop-down list:

1. Run (the default): Makes your specified job to run2. Reset if required then: Resets your job and then runs.(A job

should be either in reset mode or compiled mode to run)3. Run Validate only: Wont run the job it just validates the job.4. Reset only: It resets the job for future use

Triggers page: This page allows user to deal with the various links connected to the job activity and determines the flow of the job execution.

Name: links currently attached to the job activity stage

Expression Type: it can be on of the following

o Unconditional: It means that in the destination activity is fired once the Source

activity has completed regardless of whether other triggers are fired or not.

o Conditional:A conditional trigger fires the target activity if the source activity

fulfills the specified condition. The condition is defined by an expression, and can be one of the following types:

OK: Fires the destination Activity only if the source activity is Successful.

Failed: Fires the destination Activity only if the source activity is job failed.

Warning: Fires the destination Activity only if the source activity is job produced warnings.

Custom: Allows the user to define a custom expression. When the expression returns true the destination activity is fired.

UserStatus: Allows the user to define a status message to write to the log.

o Otherwise:It is a default trigger which will fire the destination activity when all

the conditional triggers have not fired.

1.1.4 Special Instructions:

It is beneficial using reset if required option (under Execution action) in scheduling of jobs during debugging of jobs since it saves time and keeps the logic running.

It is always beneficial in to give the description in the “general” page of the activity.

1.2 Routine stage

1.2.1 Purpose:

This Stage specifies a routine to be executed. This can be any routine in the DataStage Repository (but not transforms).

1.2.2 Creation:

There are two ways of doing it

Just drag and drop the required job from the repository window. Drag and drop the icon from the palette and later choose which routine it needs

to be referring to.

1.2.3 Description:

There are 3 main pages in the Routine Activity properties page.

o Generalo Routineso TriggersGeneral Page: This page provides the general information about the routine.

Routine page:

Routine name: Allows the user to select the required from the drop down if he had selected the routine stage from the palate. If he had dragged and dropped the routine from the repository then this field will be already filled with the corresponding routine name.

Arguments: This Page allows user to provide values for any arguments that the routine requires. The grid displays all the arguments expected by the routine. You can:

• Type in an expression giving the value for the argument in the Value Expression column. Literal values must be enclosed in inverted commas

• Select an argument and click Insert Parameter Value to use another parameter or argument in the sequence to provide the value. A dialog box appears displaying a tree of all the available parameters and arguments occurring in the sequence before the current activity. Choose the required parameter or argument and click OK. You can use this feature to determine control flow through the sequence.

• Click Clear to clear the value expression from the selected parameter.• Click Clear All to clear the expression values from all parameters

Trigger page:

Name: It will contain the names of the links attached to the Routine stage.

Expression type: Expression Type: it can be on of the following

o Unconditional: It means that in the destination activity is fired once the Source

activity has completed regardless of whether other triggers are fired or not.

o Conditional:A conditional trigger fires the target activity if the source activity

fulfills the specified condition. The condition is defined by an expression, and can be one of the following types:

OK: Fires the destination Activity only if the source activity is Successful.

Failed: Fires the destination Activity only if the source activity is job failed.

Return Value: Fires the destination Activity only if the Routine activity produced Return value equal to specified value.

Custom: Allows the user to define a custom expression. When the expression returns true the destination activity is fired.

o Otherwise:

It is a default trigger which will fire the destination activity when all the conditional triggers have not fired

1.2.4 Example:

In this example there is a jobActivity (try1) and 3 Routine activities

(reportError_andgenerate_errorno, log_and_abort and log_and_restore).When try1 fails reportError_andgenerate_errorno is executed. It reports error by generating an error report

in "error.txt”. The conditional trigger “Return Value” is used to fire the Routine Activity log_and_abort or log_and_restore i.e. the desired routine triggered according to the return value of the Routine reportError_andgenerate_errorno. More correctly when the return value equates to 2 log_and_abort is triggered and when it is 1 log_and_restore is triggered.

Trigger page of Routine Activity reportError_andgenerate_errorno:

Routine page of Routine Activity log_and_restore:

1.2.5 Special Instructions:

The insert parameter option is a useful and versatile option. It is beneficial to use this option instead of hard coding it with values.

It is always beneficial in to give the description in the “general” page of the activity.

1.3 Exec command

1.3.1 Purpose:

This stage is used to make use of existing operating system commands in the execution of our jobs. This stage provides the user with a handle to the system commands to be executed as per requirement.

1.3.2 Creation:

Drag and drop the Exec Command icon from the palette and later choose which command it needs to be referring to.

1.3.3 Description:

There are 3 main pages in the Exec Command properties page.

o Generalo Exec Commando Triggers

General Page:

This page provides the general information about the Stage.

Exec Command page:

This page gives the user the facility to specify the operating system command that he wants to execute and the required parameters, the command expects.

Triggers page:

Name: It will contain the names of the links attached to the Routine stage.

Expression type: Expression Type: it can be on of the following

o Unconditional: It means that in the destination activity is fired once the Source

activity has completed regardless of whether other triggers are fired or not.

o Conditional:A conditional trigger fires the target activity if the source activity

fulfills the specified condition. The condition is defined by an expression, and can be one of the following types:

OK: Fires the destination Activity only if the source activity is Successful.

Failed: Fires the destination Activity only if the source activity is job failed.

Return Value: Fires the destination Activity only if the System command returned value equal to specified value.

Custom: Allows the user to define a custom expression. When the expression returns true the destination activity is fired.

o Otherwise:

It is a default trigger which will fire the destination activity when all the conditional triggers have not fired

1.3.4 Example:

The sample job sequence shows a sequence that will run the job try1. If try1 runs successfully, the Success trigger (try1ok) causes the try2 job to run. If any of the jobs fail, Failure trigger (try1failed, try2failed) of both these jobs causes the alarm job to run and initiate the “update_job_abort” job to run. “update_job_abort” invokes a defined system command which updates the file “all_job_abort.txt” and on failure fires the email notification activity

The exec Command page of update_job_abort:

Here we are making use of a script with absolute path “/applsds/gtkadm/gcdbgproj/kamal/scripts/update.sh”. This script takes the file name for further processing and to append the error.

1.3.5 Special Instructions:

When ever we use system commands like scripts make sure that the user has permission on the folder in which the script resides especially if we are creating any files for storage.