Lynn Winterboer : Test automation

© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com

Test Automation: Agile Enablement for Data Warehousing and Business Intelligence TeamsPresented to Mile High Agile (MHA) 2016April 4, 2016Denver, Colorado

Lynn Winterboer, Agile Analytics Educator and Coach, @AgileLynnBrad Ewald, DW/BI Solution Architect, @JBEwald


Agenda

• Why is test automation important for agile data teams?

• Why aren’t all data teams automating their tests today?

• What is the path to test automation?

• What does simple DW test automation look like?


Why is test automation important to agile data teams?

© 2016 by Winterboer Agile Analytics www.WinterboerAgileAnalytics.com© Ken Collier, 2016, All Rights Reserved

Agile Demands Something Different

Agile calls for small increments of “potentially shippable code”.

That means QA is essential on a regular, frequent basis.

How can we be sure that something we do

here

Doesn’t break something we did back

here

Test Suite Test Suite Test

Suite Test Suite

Test Suite

Test Suite Test

Suite

Manual testing quickly becomes infeasible.The test suite grows larger each iteration.


Testing is Central to Agile

Agile BI development is driven by tests

Acceptance criteria are the definition of “done”

Passing tests are the measure of “done”

Regression tests are the measure of “still done”


Agile Teams Understand…

… that test automation is a key technical enabler

to “being agile”


Why aren’t all data teams automating their tests

today?


DW/BI Test Automation ChallengesData industry does not focus on testing as much as other disciplines:

Education

Team Members

Skills & Discipline


DW/BI Test Automation ChallengesExisting tools are not easily adapted to the data world:

Development Languages

Web, Mobile and UI Focus


DW/BI Test Automation ChallengesGood test data sets are hard to come by:

Large volumes (on small environments)

Shared test environments

Data sensitivity

Small sets need extra planning and design


What is the path to test automation?


Agile Testing Perspectives

High QualityDW/BI System

3) Tools &Automation

1) Attitudes &Culture

2) Skills &Practices

Testing is a Team SportLearning CultureBuild Quality In

Testing SkillsTest Design, Organization & MaintenanceRepeatable TestsFeedback Loops

Test EnvironmentBuild AutomationTest Data SetsTest Automation Tool


Test Automation Pyramid

AppTests

Acceptance Tests

Unit & Component Tests

Manual Tests

Lisa Crispin & Janet Gregory, Agile Testing: A Practical Guide for Testers and Agile Teams

Push as many tests as possible to this layer

Write these in business domain language

Brittle tests that are affected by UI changes

Subjective user feedback is essential


Good Places to Start Unit tests on new development

Regression tests you wish you had, before building something new

Deployment tests to ensure each build migrated correctly

Painful test setup or test execution prone to error


What does simple data warehousing test automation

look like?


Basic ApproachComponent under test

SourceSchema

TargetSchema

ActualResults

TestData

Test Runner

leave no trace

leave no trace

Setup Execute Test Report Clean Up

Static Test Data

ExpectedResults

Test Queries

load test data

compare results

executetests

based on specific input data


Demonstration


Testing FrameworkFunctionality and Components:

Example data set SQL Server and related model

Form for creating and describing tests Microsoft Access form

Test execution SQL Server stored procedures (individual tests) Power Shell scripts (batch execution)

BI Reports and Dashboards for monitoring MS Power BI


What Constitutes a Test?

Actual Result(or Dataset)

Expected Result(or Dataset)

Always a Query SQL Query or Static Value


Test Scenarios

1. Record Comparison

2. Sales Revenue Subject Area Test

3. Operational Data Quality Test


Testing Demo


Session Intent

1) Why: Test Automation is key to DW/BI agile success

2) How: Automate a solid testing practice; don’t automate chaos

3) Show: DW/BI test automation is not that hard


DW/BI Test Automation ToolsTool Description

DBFithttp://dbfit.github.io/dbfit/

Open source database testing tool

iCEDQhttp://icedq.com

A unified platform for ETL testing and data migration testing

Analytix Data Serviceshttp://analytixds.com

Enterprise data mapping and ETL code automation, including new automatic generation of reusable test cases associated with Source to Target Mappings

WhereScapehttps://www.wherescape.com

Data warehouse automation software, including a test harness framework with user-configurable profiling methods.

Tricentis Toscahttp://www.tricentis.com/tricentis-tosca-testsuite/

DW-friendly test automation tool

http://dbfit.github.io/dbfit/

http://icedq.com/

http://analytixds.com/

https://www.wherescape.com/

http://www.tricentis.com/tricentis-tosca-testsuite/

http://www.tricentis.com/tricentis-tosca-testsuite/


DW/BI Test Automation ReferencesReference DescriptionAgile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousingby Dr. Ken Collier

Chapter 7: Test-Driven Data Warehouse Development

Agile Data Warehousing for the Enterprise: A Guide for Solution Architects and Project LeadersBy Ralph Hughes

Part V: Agile EDW Quality Management Planning

Automated Data Warehouse Testing: Beginner’s Step-by-Step Guideby G. Suden

Java-based test automation for simple data warehousing tests

Agile Testing: A Practical Guide for Testers and Agile Teamsby Lisa Crispin and Janet Gregory

Great introduction to agile testing for any type of development

More Agile Testing: Learning Journeys for the Whole Teamby Lisa Crispin and Janet Gregory

Includes several chapters on test automation as well as one dedicated to agile testing for DW/BI


Thank You!

Lynn WinterboerAgile Analytics Educator &

Coachwww.WinterboerAgileAnalytics.comlynn@WinterboerAgileAnalytics.com

@agilelynn

Brad EwaldDW/BI Solution Architect

Children’s Hospital Colorado

[email protected]@jbewald

http://www.lynnwinterboer.com/

http://www.lynnwinterboer.com/

mailto:[email protected]



Lynn Winterboer : Test automation

Software

Transcript of Lynn Winterboer : Test automation