expressor customer webinar with American Tower
-
Upload
guest2295a71 -
Category
Technology
-
view
258 -
download
3
Transcript of expressor customer webinar with American Tower
![Page 1: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/1.jpg)
Lessons learned from American Tower Building a flexible and affordable enterprise data
warehouse with the expressor semantic data integration system
![Page 2: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/2.jpg)
2
Who is ATC?
• Cellular and broadcast tower ownership and operation– Over 30,000 towers
• Leader in tower industry worldwide– Operations in US, Mexico, Brazil, India
• $1.6B revenue (worldwide), 1200 employees (US)
![Page 3: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/3.jpg)
3
Business Challenges
• International operations– 4 markets at present, plus future expansions
• Business model– Real estate
• Outdated reporting environment– Single purpose data marts
– Lengthy, redundant data extracts
– No clear definition of contents
– Poor reporting
![Page 4: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/4.jpg)
4
Enterprise Data Warehouse Program
• Started October 2008
• Improve user experience for reporting– Integrated data
– Improved reporting tool
– Faster data refreshes
• Three-part solution– Data Warehouse: Kimball methodology, SQL Server 2005
– BI reporting tool: Cognos 8
– Governance process: Business-led management of DW
![Page 5: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/5.jpg)
5
Selecting the right tools
• Reporting tool (Cognos 8)– Lengthy vendor selection process, close business involvement
• Database for DW (SQL Server 2005)– Technical evaluation (data volumes and future capacity reqs)
– Experience of existing personnel
• ETL tool (initially SSIS)– Experience of existing personnel
– Budget (used existing SQL Server licenses)
– Technical evaluation (could live with shortcomings)
![Page 6: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/6.jpg)
6
ETL Structure - High Level
• Three-step process
• Separate jobs for each process
Extract Transform Load
Raw data pulls Prepare data Load Facts & Dimensions (SCD)
Source System EDW
![Page 7: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/7.jpg)
7
ETL Structure - Detail
• Three-layer process for each step
Extract
Scheduler
Metadata Wrapper
ETL Execution
Control Flow, timing
Data quality, logging, etc.
Actual data transfer / processing
![Page 8: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/8.jpg)
8
SSIS Issues
• Functionality– Bulk updates
– Awkward scripting (two languages, not well integrated)
• Performance– Oracle extracts not performing optimally
![Page 9: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/9.jpg)
9
Opportunity for expressor
• Became aware of expressor mid-2009
• Proof of concept to establish benefits– Performance: 8-24x faster for Oracle extracts (1-4
channels)
– Scripting: expressor datascript very powerful
– Functionality: bulk updates, general capabilities
• Acquisition made much easier by low cost of adoption
![Page 10: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/10.jpg)
10
ETL Structure - expressor
• Three-step process
Extract Transform Load
Raw data pulls Prepare data Load Facts & Dimensions (SCD)
Improved performanceSemantic rationalization
ScriptingETL functionality
Bulk updates
expressor benefits
![Page 11: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/11.jpg)
11
ETL Structure
• Three-layer process for each step
Extract
Scheduler
Metadata Wrapper
ETL Execution
Control Flow, timing
Data quality, logging, etc.
Actual data transfer / processing
Change execution method via metadata
Replace with expressor
![Page 12: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/12.jpg)
12
expressor Downstream Benefits
• Semantic dictionary– Clarify confusing business terms
• Multiple formats for Tower Number
• Differing business terms for same concept (milestone / date)
– Direct input from BAs into data modeling / ETL
• Growth potential– Add channels as needed
• Development / Maintenance– Simpler ETL
– Fewer stored procedures or “inventive” solutions
![Page 13: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/13.jpg)
13
expressor Challenges
• expressor datascript is different from MS / .Net world– Very powerful scripting language
• Single library– All semantic terms share a namespace
• Process / Flow control
• Involving BAs in semantic rationalization– Requires process change outside development team
![Page 14: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/14.jpg)
14
Lessons Learned
• SSIS is “free”, but you get what you pay for– Functionality limitations; we didn’t know what we were missing
– Performance
• Transition requires effort– Small learning curve for expressor datascript
• Semantic Rationalization process impacts– Work with all affected groups
• ETL Architecture preparation pays off– Plan for scalable hardware and flexible software
![Page 15: expressor customer webinar with American Tower](https://reader030.fdocuments.us/reader030/viewer/2022032611/55d1149ebb61eb80708b47e9/html5/thumbnails/15.jpg)
15