Etl with talend (data integeration)
description
Transcript of Etl with talend (data integeration)
![Page 1: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/1.jpg)
ETLwith talend
POOJA B. MISHRA
![Page 2: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/2.jpg)
What is ETL?Extract is the process of reading data from a database
Transform is the process of converting the extracted data from its previous form into the form it needs to be in so that it can be placed into another database. Transformation occurs by using rules or lookup tables or by combining the data with other data
Load is the process of writing the data into the target database
![Page 3: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/3.jpg)
Process flow
![Page 4: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/4.jpg)
Terms closely related and managed by ETL processes data migrationdata management data cleansingdata synchronization data consolidation.
.
![Page 5: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/5.jpg)
Different ETL tools
•Oracle ETL•Ab Initio•Pentaho Data Integration -Kettle Project (open source ETL)•SAS ETL studio•Cognos Decisionstream•Business Objects Data Integrator (BODI)•Microsoft SQL Server Integration Services (SSIS) •Informatica PowerCenter•Talend
![Page 6: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/6.jpg)
Prerequisites Talend Open Studio for Data Integration
◦ http://www.talend.com/download VirtualBox
◦ https://www.virtualbox.org/wiki/Downloads Hortonworks Sandbox VM
◦ http://hortonworks.com/products/hortonworks-sandbox/#install
![Page 7: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/7.jpg)
How to set up – Step 1
![Page 8: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/8.jpg)
Step 2
![Page 9: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/9.jpg)
Step 3
![Page 10: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/10.jpg)
Step 4
![Page 11: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/11.jpg)
Step 5
![Page 12: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/12.jpg)
Talend InterfaceWorkspace
Repository tree
Component configuration
Palette
WorkspaceRepository tree
Palette
Repository tree
Workspace
Palette
Component configuration
![Page 13: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/13.jpg)
What are supported data input formats?•SQL
•MySQL
•PostgreSQL
•Sybase
•Teradata
•MSSQL
•Netezza
•Greenplum
•Access
•DB2
•Hive
![Page 14: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/14.jpg)
What kinds of datasets can be loaded?
Talend Studio offers nearly comprehensive connectivity to:
Packaged applications (ERP, CRM, etc.), databases, mainframes, files, Web Services, and so on to address the growing disparity of sources.
Data warehouses, data marts, OLAP applications - for analysis, reporting, dashboarding, scorecarding, and so on.
Built-in advanced components for ETL, including string manipulations, Slowly Changing Dimensions, automatic lookup handling, bulk loads support, and so on.
![Page 15: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/15.jpg)
What are data export formats?
![Page 16: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/16.jpg)
Talend - ELT DB Components – Step 1
![Page 17: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/17.jpg)
Step 2
![Page 18: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/18.jpg)
Step 3
![Page 19: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/19.jpg)
Step 4
![Page 20: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/20.jpg)
Step 5
![Page 21: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/21.jpg)
Step 6
![Page 22: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/22.jpg)
Challenges involvedData volumes are growing exponentiallyData velocity is moving fasterAs information systems grow in complexity, the disparity of sources is growing as wellAll these target structures have different data transformation requirements and different tolerances in terms of latencyTransformations involved in ETL processes can be highly complex
![Page 23: Etl with talend (data integeration)](https://reader038.fdocuments.us/reader038/viewer/2022102613/546b2d64af7959651f8b56a6/html5/thumbnails/23.jpg)
Thank You!!