Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP...
Transcript of Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP...
![Page 1: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/1.jpg)
Georg Thurner
© 2019 by .
Automated Continuous Testing
Mike Schlabinger, VP Global Sales Enablement
![Page 2: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/2.jpg)
Low
Risk
High
Risk
Medium
Risk80%20%
Business RiskCoverage
TestCases
The time needed fortesting is infinitely larger
than the time available.
Critical Limit
Risk-Based Testing
Have the right test cases.Do the right things.
![Page 3: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/3.jpg)
80%Manual Testing
20%Automated Testing
Present
The reliance onmanual testing is the
top technical challengein app development
World Quality Report, 2015/16
Test automationrequires developers
UI LoadAPI
![Page 4: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/4.jpg)
+85%Automated Testing
Future
Exploratory Tests
Automated UI Tests
API Tests
Orchestrated Service Virtualization
Load Tests
80%Manual Testing
20%Automated Testing
Present
![Page 5: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/5.jpg)
OpsOperations
DevDevelopment
Model-BasedTest Automation
APITesting
BI &Big Data
ServiceVirtualization
ExploratoryTesting
Risk-BasedTesting
TestManagement
ManualTesting
Analytics
ContinuousIntegration
Test DataManagement
ImpactAnalysis
LoadTesting
![Page 6: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/6.jpg)
6
3 nightmares of test automation
2 Test Data Management… needs to be stateful, fully integrated
1 Maintenance trap … you need to provide MBTA
… for all interfaces, E2E systems & APIs
Test System Provisioning … Service Virtualization3… especially for complex enterprise landscapes
6
![Page 7: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/7.jpg)
Automation Approaches
Record & ReplayLinear Framework
SophisticatedFramework
Model-BasedAutomation
1st Generation 2nd Generation 3rd Generation
Script-Based
10%Automation
30%Automation
90%Automation
Tosca Recorder
![Page 8: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/8.jpg)
Logical Name
Enter Product Data
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Enter Insurant Data
Verify Quote Details
Next
Start Date
Insurance Sum [$]
Payment Option
Create Vehicle Insurance
Price List
Insurance Sum
Gross Premium [$]
Value
Audi
200
5
Petrol
35.000
10.000
{CLICK}
Private
{Year}
B-CD 123
{CLICK}
01/03/2016
7Mio
Yearly
1.535,22
Action
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Verify
Select
Select
Business Information
Model-BasedBrowser
Do you see the business information?
Script-BasedBrowser
CodeSynchronization Behavior
CodeRecovery Handling
CodeData Driven Framework
CodeKeyword Driven Framework
1
2
3
4
Assert.AreEqual("1.535,22",
driver.FindElement(By.XPath("//table[@id='PriceList']/t
body/tr[10]/td[2]/b"))
.GetAttribute("value"));
![Page 9: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/9.jpg)
System
SUT
Model Test Cases
Technical Information Business Information
Extract System Model
Create Test Cases
Feed Data
Automate System
The Big Picture
![Page 10: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/10.jpg)
Technical Information
Next >>
Enter Vehicle Data
Make Audi
Engine Performance [kW] 200
Number of Seats 5
Fuel Petrol
Year of Construction 2016
List Price [$] 35.000
License Plate Number B-CD 123
Annual Mileage [mi] 10.000
Usage Private
sampleapp.tricentis.com
Technical Name Value Range
-
-
1;2;3;4;5;6;7;8;9
Petrol;Diesel;Gas;Other
-
-
{CLICK};{RIGHTCLICK}
Private;Commercial
2016;2015;2014;2013
-
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Insurant Data
Quote Details
Product Data
Next {CLICK}
Start Date 01/03/2016;01/04/2016
Insurance Sum [$] 3Mio;7Mio;10Mio;15Mio
Payment Option Yearly;Quarterly;Monthly
Scan Application….
![Page 11: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/11.jpg)
Logical Name
Enter Insurant Data
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance
Value
Audi
200
5
Petrol
35.000
10.000
{CLICK}
Private
{Year}
B-CD 123
Action
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Enter Product Data
Next
Start Date
Insurance Sum [$]
Payment Option
{CLICK}
01/03/2016
7Mio
Yearly
Input
Input
Input
Input
Verify Quote Details
Price List
Insurance Sum
Gross Premium [$] 1.535,22 Verify
Select
Select
Business Information
Technical Name Value Range
Technical Information
-
-
1;2;3;4;5;6;7;8;9
Petrol;Diesel;Gas;Other
-
-
{CLICK};{RIGHTCLICK}
Private;Commercial
2015;2014;2013;2012
-
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Insurant Data
Quote Details
Product Data
Next {CLICK}
Start Date 01/03/2016;01/04/2016
Insurance Sum [$] 3Mio;7Mio;10Mio;15Mio
Payment Option Yearly;Quarterly;Monthly
It’s like playing Lego.
![Page 12: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/12.jpg)
Next >>
Enter Vehicle Data
Make Audi
Engine Performance [kW] 200
Number of Seats 5
Fuel Petrol
Year of Construction 2016
List Price [$] 35.000
License Plate Number B-CD 123
Annual Mileage [mi] 10.000
Usage Private
sampleapp.tricentis.com
Logical Name
Enter Product Data
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Enter Insurant Data
Verify Quote Details
Next
Start Date
Insurance Sum [$]
Payment Option
Create Vehicle Insurance
Price List
Insurance Sum
Gross Premium [$]
Value
Audi
200
Petrol
35.000
10.000
{CLICK}
Private
{Year}
B-CD 123
{CLICK}
01/03/2016
7Mio
Yearly
1.535,22
Action
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Verify
Select
Select
Business Information
5
![Page 13: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/13.jpg)
Tricentis Tosca allows for easy tests across any technology!
Ad
ap
ter
Ad
ap
ter
Ad
ap
ter
Ad
ap
ter
> 30 technology specific adapters
Model Based Test Automation
Customer-specific
extensions
Technologies
…
Mo
bile
Java
Wo
rd, X
L, …
Win
32
, C+
+, V
B
De
lph
i
32
70
, Em
ula
tors
Da
tab
ase
s
RE
ST
JSO
N
SA
P
xBro
wse
r
We
bS
erv
ice
s, X
ML
.NE
T, W
PF
Gu
pta
Po
we
rBu
ilde
r
PD
F
File
co
nte
nt
![Page 14: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/14.jpg)
100%
0%
SprintStart
SprintEnd
API
UI
Start testing early.Shift left test automation.
API Testing
Degree ofCompletion
Time
Efficiency
Gain
+4xCreation
+6xMaintenance
+20xExecution Future test automation must focus on API testing.
![Page 15: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/15.jpg)
Next >>
Enter Vehicle Data
Make Audi
Engine Performance [kW] 200
Number of Seats 5
Fuel Petrol
Year of Construction 2016
List Price [$] 35.000
License Plate Number B-CD 123
Annual Mileage [mi] 10.000
Usage Private
sampleapp.tricentis.com
...
<section class="step">
<div class="field">
<label class="main">Make</label>
<select id="make" name="Make">
<option value="Audi">Audi</option>
<option value="BMW">BMW</option>
<option value="Ford">Ford</option>
...
</select>
<span class="error"></span>
</div>
<div class="field">
<label class="main">Engine Performance[kW]</label>
<input id="performance" name="[kW]" type="text">
<span class="error"></span>
</div>
...
<div class="field">
<label class="main">Number of Seats</label>
<select id="nofseats" name="Number of Seats">
<option value="1">1</option>
<option value="2">2</option>
...
</select>
<span class="error"></span>
</div>
...
</section>
...
sampleapp.tricentis.com
Non Visual InterfaceHTML
![Page 16: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/16.jpg)
Next >>
Enter Vehicle Data
Make Audi
Engine Performance [kW] 200
Number of Seats 5
Fuel Petrol
Year of Construction 2016
List Price [$] 35.000
License Plate Number B-CD 123
Annual Mileage [mi] 10.000
Usage Private
sampleapp.tricentis.com
<Envelope xmlns = 'http://…'>
<Body>
<CalculatePrice xmlns = 'http://…'>
<Request>
<VehicleData xmlns = 'http://…'>
<Make>Audi</Make>
<Performance>200</Performance>
<Year>2016</Year>
<Seats>5</Seats>
<Fuel>Petrol</Fuel>
<ListPrice>35000</ListPrice>
<PlateNumber>B-CD 123</PlateNumber>
<Usage>Private</Usage>
<AnnualMileage>10000</AnnualMileage>
</VehicleData>
<InsurantData xmlns = 'http://…'>
</InsurantData>
<ProductData xmlns = 'http://…'>
<StartDate>01/03/2016</StartDate>
<InsuranceSum>7000000</InsuranceSum>
<PaymentOption>Yearly</PaymentOption>
</ProductData>
</Request>
</CalculatePrice>
</Body>
</Envelope>
sampleapp.tricentis.com
Non Visual InterfaceWebservice
![Page 17: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/17.jpg)
<Envelope xmlns = 'http://…'>
<Body>
<CalculatePrice xmlns = 'http://…'>
<Request>
<VehicleData xmlns = 'http://…'>
<Make>Audi</Make>
<Performance>200</Performance>
<Year>2016</Year>
<Seats>5</Seats>
<Fuel>Petrol</Fuel>
<ListPrice>35000</ListPrice>
<PlateNumber>B-CD 123</PlateNumber>
<Usage>Private</Usage>
<AnnualMileage>10000</AnnualMileage>
</VehicleData>
<InsurantData xmlns = 'http://…'>
</InsurantData>
<ProductData xmlns = 'http://…'>
<StartDate>01/03/2016</StartDate>
<InsuranceSum>7000000</InsuranceSum>
<PaymentOption>Yearly</PaymentOption>
</ProductData>
</Request>
</CalculatePrice>
</Body>
</Envelope>
sampleapp.tricentis.com
Non Visual InterfaceWebservice
Audi
200
5
Petrol
35.000
10.000
Private
{Year}
B-CD 123
01/03/2016
7Mio
Yearly
1.535,22
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Verify
Select
Select
Business Information
API Test CaseWebservice
Enter Product Data
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Usage
Year of Construction
License Plate Number
Enter Insurant Data
Verify Quote Details
Start Date
Insurance Sum [$]
Payment Option
Price List
Insurance Sum
Gross Premium [$]
Webservice
Request
Webservice
Response
![Page 18: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/18.jpg)
Logical Name
Enter Product Data
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Enter Insurant Data
Verify Quote Details
Next
Start Date
Insurance Sum [$]
Payment Option
Create Vehicle Insurance
Price List
Insurance Sum
Gross Premium [$]
Value
Audi
200
5
Petrol
35.000
10.000
{CLICK}
Private
{Year}
B-CD 123
{CLICK}
01/03/2016
7Mio
Yearly
1.535,22
Action
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Verify
Select
Select
Business Information
UI Test CaseBrowser
Audi
200
5
Petrol
35.000
10.000
Private
{Year}
B-CD 123
01/03/2016
7Mio
Yearly
1.535,22
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Verify
Select
Select
Business Information
API Test CaseWebservice
Enter Product Data
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Usage
Year of Construction
License Plate Number
Enter Insurant Data
Verify Quote Details
Start Date
Insurance Sum [$]
Payment Option
Price List
Insurance Sum
Gross Premium [$]
![Page 19: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/19.jpg)
Logical Name
Enter Product Data
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Enter Insurant Data
Verify Quote Details
Next
Start Date
Insurance Sum [$]
Payment Option
Create Vehicle Insurance
Price List
Insurance Sum
Gross Premium [$]
Value
Audi
200
5
Petrol
35.000
10.000
{CLICK}
Private
{Year}
B-CD 123
{CLICK}
01/03/2016
7Mio
Yearly
1.535,22
Action
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Verify
Select
Select
Business Information
UI Test CaseBrowser
Audi
200
5
Petrol
35.000
10.000
Private
{Year}
B-CD 123
01/03/2016
7Mio
Yearly
1.535,22
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Verify
Select
Select
Business Information
API Test CaseWebservice
Enter Product Data
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Usage
Year of Construction
License Plate Number
Enter Insurant Data
Verify Quote Details
Start Date
Insurance Sum [$]
Payment Option
Price List
Insurance Sum
Gross Premium [$]
Manual Testers become Automation Specialists
![Page 20: Automated Continuous Testing - KMS Solutions · Automated Continuous Testing Mike Schlabinger, VP Global Sales Enablement. Low Risk High Risk Medium Risk 20 80 % Business Risk Coverage](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecee63b9648e02c7b7f9a7c/html5/thumbnails/20.jpg)
QuestionsBecause answers exist only to questions...
?