Data Warehouse Testing: The QuerySurge Data Wizards
-
Upload
rtts -
Category
Technology
-
view
2.256 -
download
5
description
Transcript of Data Warehouse Testing: The QuerySurge Data Wizards
![Page 1: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/1.jpg)
built by
Bill HaydukCEO/President
RTTS
Harry Potter and Data Warehouse TestingIntroducing the QuerySurge Data Wizards
Martin Henry-LesterProduct Specialist
RTTS
Laura PoggiMarketing Manager
RTTS
Webinar
![Page 2: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/2.jpg)
About FACTSFounded: 1996
Primary Focus:
consulting services, software
Locations: New York, Atlanta, Philly, Phoenix
Geographic region:North America
Customer profile:Fortune 500, > 450 clients
Software:QuerySurge
The Software Quality Experts
RTTS is the leading provider of software quality for critical business systems
![Page 3: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/3.jpg)
built by
The average organization loses $8.2 million annually through poor Data Quality.
- Gartner
46% of companies cite Data Quality as a barrier for adopting Business Intelligence products.
- InformationWeek
Data Quality Best Practices boost revenue by 66%. - Research firm Sirius Decisions
![Page 4: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/4.jpg)
4
About QuerySurge
built by
![Page 5: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/5.jpg)
QuerySurge is the
premier test tool built
to automate Data Warehouse testing
and the ETL Testing Process
What is QuerySurge?
built by
![Page 6: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/6.jpg)
What does
QuerySurge ™do?
built by
QuerySurge finds bad data
• Most firms test < 10% of their data
• Your CxOs rely on BI apps to make strategic decisions
• BI apps sit on top of DWHs that have at best, untested data & at worst, bad data
• QuerySurge tests up to 100% of your data quickly & finds bad data
![Page 7: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/7.jpg)
QuerySurge™ Architecture
built by
Que
rySu
rge
Arch
itect
ure
Target
Sources
![Page 8: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/8.jpg)
8
Architecture-Under-Test
Sources
SQL Q
uery
Da
ta se
t
SQL Query
Data set
Data Set #1
Data Set #2
© 2011 Real-Time Technology Solutions, Inc.
QuerySurge Architecture
User(s)
Web interface
CompareData Sets
QuerySurge™: How it Works
built by
![Page 9: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/9.jpg)
QuerySurge Roles & Uses
Testers - functional testing - regression testing
ETL Developers - unit testing
Data Analysts- review, analyze data - verify mappings and
failures.
Operations teams - monitoring
built by
![Page 10: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/10.jpg)
built by
Design Library Create Query Pairs (source & target SQLs)
10
QuerySurge™ Modules
Scheduling Build groups of Query Pairs Schedule Test Runs
![Page 11: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/11.jpg)
Deep-Dive Reporting Examine and automatically
email test results
Run Dashboard View real-time execution Analyze real-time results
QuerySurge™ Modules
built by
![Page 12: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/12.jpg)
built by
QuerySurge™ Gallery: QueryPair Creation
Source Query
Target Query
![Page 13: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/13.jpg)
built by
QuerySurge™ Gallery: Run Dashboard
QueryPair details
Runtime activity
Result sets
![Page 14: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/14.jpg)
built by
QuerySurge™ Gallery: Summary Report
Duration= 34 sec
720,000 data sets
14,000 data failures
![Page 15: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/15.jpg)
built by
QuerySurge™ Gallery: Detail Report
Individual QueryPair passes
Individual QueryPair fails
![Page 16: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/16.jpg)
Business Problem
built by
In a survey of potential customers we asked:
What are your top
DWH testing challenges?
![Page 17: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/17.jpg)
They need to automate the testing effort
Their Top 6 Responses
They need to speed up the testing cycle
built by
They need to be able to schedule test runs
They need to be able to test across different platforms
They need to share test results with their group
They need to verify more data than they currently do (typically <10%)
![Page 18: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/18.jpg)
automates the testing effort the kickoff, the tests, the comparison, emailing the results
speeds up testing up to 1,000 times faster than manual testing
schedules test runs run now, every Tuesday at 11pm or right after ETL process
tests across different platformsany JDBC-compliant database, DWH, D-Mart, flat file, XML
views & shares results through automated emailing of reports
the QuerySurge solution…
verifies more data verifies upwards of 100% of all data
built by
![Page 19: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/19.jpg)
QuerySurge Value-Add
QuerySurge provides value by either:
in testing data coverage from < 10% to upwards of 100%
in testing time by as much as 1,000 x
combination of in test coverage while in testing time
19built by
![Page 20: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/20.jpg)
Return on Investment (ROI)
redeployment of head count by 75% because of an increase in coverage
a savings over manual testing (minus queries, manual compares, other)
an increase in better data due to shorter / more thorough testing cycle, saving $millions by preventing key decisions made on bad data.
20built by
![Page 21: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/21.jpg)
FYI - Training Courses
built by
CoursesFundamentals of QuerySurge 1 dayIntroduction to SQL for QuerySurge 1 day Advanced SQL Techniques for QuerySurge 1 dayData Warehouse Fundamentals 1 day
Delivery Optionsin the Cloud in our Classroom private on-site training
new
![Page 22: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/22.jpg)
built by
![Page 23: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/23.jpg)
QuerySurge™ new release!!!
What do and
QuerySurge™ have in common?
23
They both use to get the job done!!!
Introducing the
built by
![Page 24: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/24.jpg)
24
Fast and Easy. No programming needed.
built by
• Perfect for data warehouse testing, data migrations & upgrades
• Great for verifying that all rows have come over without issue
• Superb at verifying table-to-table transfers
• Ability to graphically create SQL stubs for advanced scripting
![Page 25: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/25.jpg)
Uses:
Verify that the amount of rows match the amount that were expected
built by
Roles:
ETL Developer, Operations, Testers
SQL Skill set / Value added:
beginner: no coding needed
experienced user: saves time
![Page 26: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/26.jpg)
Uses: - Verify data warehouse loads
when no transformation- Verifies data migrations,
upgrades
built by
Roles: ETL Developers, Testers
SQL Skill set / Value added: beginner: no coding neededexperienced user: saves
time
![Page 27: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/27.jpg)
Uses: - Utilize either wizard to create stub
statements for writing SQL query pairs
built by
SQL
SQL
SQL
SQLSQL
SQL
SQL SQL
SQL
SQL
Roles: ETL Developers, Advanced Testers
SQL Skill set / Value added: For experienced users: - No need to look up schema(s) or
table names- Saves tons of time
![Page 28: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/28.jpg)
Visually select source and target database (or DWH)
Wizards’ ease-of-use Select Row Count Wizard or Table Compare Wizard
Select schema(s) from source and targetSelect matching tables in source and target
QuerySurge automatically creates test suite
Now you are ready to run!
![Page 29: Data Warehouse Testing: The QuerySurge Data Wizards](https://reader036.fdocuments.us/reader036/viewer/2022062418/556132ffd8b42a92358b4714/html5/thumbnails/29.jpg)
Ensuring Data Warehouse Quality
QuerySurgebuilt by
Demo