Automated Performance Testing for Desktop Applications by Ciprian Balea
-
Upload
3pillar-global -
Category
Business
-
view
816 -
download
0
Transcript of Automated Performance Testing for Desktop Applications by Ciprian Balea
Automated Performance Testing for Desktop Applications by Ciprian Balea
2
THE SCARY QUESTION OF APPLICATION DEVELOPMENT
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
How does it perform in real conditions?Never as first expected!
3
BENCHMARKING• What impact does the app have on the system running it? (is the
system still reliable)• How does the app perform when running on the host system?
(does it keep all functionality)
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Ideally,• consuming as little resources as possible without loosing
functionality• the app should not break /slow the system down
4
HOW TO PREDICT THE FUTURE?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Run it multiple times and average out.
A single run doesn’t accurately predict real life usage.
5
HOW TO GET A CREDIBLE RESULT?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Avoid inconsistencies induced by the outside world => no Internet connection, own
webserver & assets
6
WHAT’S OUR APPLICATION ABOUT?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
A Custom Web Browser
7
HARDWARE SETUP
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
8
BIG PROBLEM #1: HOW TO KNOW HOW MUCH TIME THE PAGE TOOK TO LOAD
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Based on the HTML content, loading time cannot be computed (element loading order is not consistent across
multiple runs).
Small solution:
9
BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Small solution (a):
10
BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Small solution (b):
11
BIG PROBLEM #3: HOW TO MANIPULATE THE APPLICATION’S GUI
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Small solution:
12
WHAT DATA WE COLLECT?• Test 1
• Webpage load time• CPU usage• Memory usage• Disk I/O
• Test 2• Timing a “complex” operation
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
13
TEST 1: READING SYSTEM
STATISTICS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
14
HOW IS THE DATA COLLECTED?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Not at the start of the game
Not at the end of the game
Throughout the entire
game
15
HOW IS THE DATA COLLECTED?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
16
RUNNING THE TEST• One test sequence:
• clear cache• open the browser• start reading system data• load webpage• close the browser• save result in an Excel spreadsheet
• Full test:• Restore the “clean” snapshot of the test
environment• Automatically runs the test sequence for
1000 times on the native browser• Automatically runs the test sequence for
1000 times on the custom browser
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
17
RUNNING THE TEST
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
18
TEST 2: TIMING A
“COMPLEX” OPERATION
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
19
“COMPLEX” OPERATION
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Open the browser
Load a webpage
Bookmark page
Go to homepage
Open bookmark
Wait for page to load
20
RESULTS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
21
RESULTS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
22
CONCLUSIONSPerformance testing is very important as it can reveal performance flaws.
It is very time and effort consuming to do it manually.
It can be automated with some initial effort, but the returned value is huge.
Offers very conclusive results on what needs to be improved in the application to boost performance.
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
23
Q&A
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
THANKS!Now let’s get to
work!
24© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential