Flat File Validation
-
Upload
prathyush-kanchireddy -
Category
Documents
-
view
218 -
download
0
Transcript of Flat File Validation
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 1/12
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 2/12
Table of Contents
1. Assumption2. Summary3. Objective
4. Source definition5. 3 Target definition5.1 Target T_FILE_PROCESS_STATUS5.2 Target EMPLOYEE6. 4 Mappings.6.1 Mapping to validate the source file: m_ValidateSourceFile6.1.1 Transformations used6.2 Mapping to load source file: m_LoadFile6.2.1 Transformations used7. 5 Sessions8. 6 Workflow
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 3/12
Assumptions:
• Assume that the flat file is having one header (stats with H) and one trailor (Starts with T).
• Flat file should exist in the local directory.
• File size should not be zero (if it zero we have option to check file size at informatica
level).
• File is having some records more than two (Including header and trailor)
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 4/12
Flat file validation process
Summary:
The intention of this case study is to give an idea about validation of a flat file source usinginformation in file trailor. File will contain total record count in trailor and same will be verified withactual record count received in file before loading to target. If file is found to be valid, loading willbe done else a record will be inserted in status table indicating that file to be processes is invalid.
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 5/12
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 6/12
3. Target Definition
3.1 Target T_FILE_PROCESS_STATUSFirst target is relational table (oracle) T_FILE_PROCESS_STATUS and has below structure.
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 7/12
This table will be used to load file information when the file is not valid. In case of a valid filenothing will be loaded in this table. Example data in this table for invalid & valid file will be asshown below.
Process_Date Header_SeqNo Trailor_count File_Record_count Is_File_Valid
01/12/2010 1500 12 14 N
02/12/2010 1500 14 14 Y
3.2 Target EMPLOYEESecond target is EMPLOYEE table with below structure.
4. Mappings
4.1 Mapping to validate the source file: m_ValidateSourceFile
Below shown is a prototype of the mapping to check if total record count of file matches with
record count specified in trailor. If there is a mismatch then a status record will be inserted inT_FILE_PROCESS_STATUS. For a valid record count in trailor, no record will be inserted intarget table.
4.1.1 Transformations used
a) exp_Header_Trailor:
This expression is used after Source Qualifier to identify file header and file trailor. In thisexpression total number of records in file is also calculated.
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 8/12
In expression define following variable ports and output ports.
Eg;
Input column; empno
Please see the below expression used in different Variable ports:
Port Name Expression
v_HeaderSeqNo IIF (SUBSTR (Empno, 1, 1) ='H', SUBSTR (Empno, 2),
NULL)
v_TrailorRecCnt IIF SUBSTR (Empno, 1, 1) ='T', SUBSTR (Empno, 2),NULL)
v_FileRecCnt v_FileRecCnt + 1
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 9/12
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 10/12
Output port o_IsFileValid uses below expression to check if trailor record count matches with filerecord count. If there is a mismatch then o_IsFileValid flag is set to ‘N’.
Port Name Expression
O_isFile_Valid IIF (o_FileTrailor != o_FileRecCount,'N')
d) FIL_INVALID_FILE
This filter will pass the row only when o_IsValidFile flag is ‘N’ i.e. trailor record count does notmatch with file record count.Filter condition is o_IsFileValid = 'N'.
4.2 Mapping to load source file: m_LoadFile
This is simple source to target mapping with a filter in place to skip file header and trailor.
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 11/12
4.2.1 Transformations used
FIL_HEADER_TRAILOR
Filter is used to skip file header and trailor. Only data records will be passed and loaded to targetT_EMPLOYEE.
Filter condition used is SUBSTR (Empno, 1, 1)! = ‘H’ AND SUBSTR (Empno, 1, 1) != 'T'
5. Sessions
Session tasks will be created corresponding to each mapping. Session s_ValidateSourceFile isfor mapping m_ValidateSourceFile.
Session s_LoadFile is for mapping m_LoadFile. As date in source file is in DD-MON-YY format,DateTime Format String is changed from default to DD-MON-YY. All other settings are default.
7/22/2019 Flat File Validation
http://slidepdf.com/reader/full/flat-file-validation 12/12
6. Workflow (wf_LoadFile):
Simple workflow as shown below is created using a start task and two session tasks.
Condition to execute session wf_LoadFile is defined as below expression.
IIF ($s_ValidateSourceFile.TgtSuccessRows = 0,
$s_ValidateSourceFile.Status = succeeded,$s_ValidateSourceFile.Status = failed)
With this condition is place, session s_LoadFile will execute only when no row is being inserted totarget by session s_ValidateSourceFile. As per the mapping logic, m_ValidateSourceFile willinsert the row in target only when file is failed for validation. So no row inserted in target meansfile is valid and should be loaded to EMPLOYEE table using m_LoadFile mapping.Using sample flat file to execute this workflow will insert no row in T_FILE_PROCESS_STATUS.12 rows will be inserted to EMPLOYEE table.