With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache...
Transcript of With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache...
![Page 1: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/1.jpg)
Performance testing
With Apache JMeter? Frank Pientka
![Page 2: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/2.jpg)
© Materna GmbH 2016 www.materna.com
Where and who is Materna GmbH?
2© Materna GmbH 2016
Founded: 1980Employees: 1,700Turnover 2015: € 210 million
ITK-Full-service provider in the premium segment
Family-owned company in the ITC sector
Target groups: IT organisations and user departments in both private and public sectors
Dortmund
Düsseldorf
Bad Vilbel
Stuttgart
Erlangen
München
Dresden
Bremen Hamburg
BerlinWolfsburg
Lüneburg
Köln
Nürnberg
Heidelberg
FI
SE
DK
PL
SK
UKNL
DECZ
NOR
Orlando, Florida, USA
![Page 3: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/3.jpg)
© Materna GmbH 2016 www.materna.com
Who is Frank Pientka?
Dipl.-Informatiker (TH Karlsruhe)
Principal Software Architect at Dortmund
iSAQB founding member
heise.de/developer/Federlesen columne
over 20 years IT experiencelots of articles and talks
![Page 4: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/4.jpg)
© Materna GmbH 2016 www.materna.com
Agenda
Why performance testing? Testing right! What is Apache JMeter? What’s new in Jmeter 3.0? JMeter main elements? Have a (test-)plan! Distributed performance testing Further steps, extensions, challenges Resume
4
![Page 5: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/5.jpg)
© Materna GmbH 2016 www.materna.com
Why performance tests?
5
![Page 6: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/6.jpg)
© Materna GmbH 2016 www.materna.com
What went wrong while testing?
How secure&safe?
The Heisenbug!
Volkswagen St. (VW) share
6
![Page 7: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/7.jpg)
© Materna GmbH 2016 www.materna.com
Java performance testing tools (selection)
JUnitPerf ab Gattling
7
![Page 8: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/8.jpg)
© Materna GmbH 2016 www.materna.com
May I introduce Apache JMeter? 18 years of experience
Since 1998 for Apache Tomcat loadtest (Stefano Mazzocchi) Recent version 3.0 (2016-05-17) Supported protocols
Web - HTTP, HTTPS, FTP SOAP/XML-RPC Database via JDBC, MongoDB, HADOOP LDAP JMS Mail - SMTP(S), POP3(S) and IMAP(S) Native commands or shell scripts
+ Highly extensible and configurable, tutorial, example templates, demos- Limited reporting capacities, Short outdated documentation, i18n
8
![Page 9: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/9.jpg)
© Materna GmbH 2016 www.materna.com
https://jmeter-plugins.org/stats/
9
![Page 10: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/10.jpg)
© Materna GmbH 2016 www.materna.com 10
Apache JMeter 3.0 Changes http://jmeter.apache.org/changes.html
Test plan creation and debugging improvements Reporting improvements (dynamic HTML-Dashboard) Protocols and Load Testing improvements UX Improvements Core improvements Components updates Incompatible changes, deprecations
![Page 11: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/11.jpg)
© Materna GmbH 2016 www.materna.com
A JMeter test plan example
11
![Page 12: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/12.jpg)
© Materna GmbH 2016 www.materna.com
How does Apache JMeter simulate a single user?
http://jmeter-expert.blogspot.de
12
![Page 13: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/13.jpg)
© Materna GmbH 2016 www.materna.com 13
Typical steps
Create Testplan Test Testplan Run Testplan and evaluate results Document Testplan and results
Extras/schematic.cmd Testplan.jmx Testplan.html
TestPlanCheck --jmx <filename> [ --stats --tree-dump ]
jmeter -n -t <test JMX file> -l <test log file> -e -o <Path to output folder>
![Page 14: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/14.jpg)
© Materna GmbH 2016 www.materna.com
How does Apache JMeter simulate multiple users?
http://jmeter-expert.blogspot.de
14
![Page 15: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/15.jpg)
© Materna GmbH 2016 www.materna.com
The main elements of a JMeter testplan
http://jmeter-expert.blogspot.de
15
![Page 16: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/16.jpg)
© Materna GmbH 2016 www.materna.com16
JMeter testplan elements
Workbench• Test Recorder
ConfigTestElement• Request Defaults• Autorisation Manager• Cookie Manager
Sampler• HTTP Request
Timer
Assertion
Listener• SummaryReport• Results Tree• Results Table
Start
![Page 17: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/17.jpg)
© Materna GmbH 2016 www.materna.com
Login testing scenario
17
Thread GroupThread: 1000Ramp-up: 500
ConstantTimer300ms
HTTP sampler GETHostname/urlStart
End
AssertiomCheck combination
ist OK
RegExHTTP Status code
HTTP sampler POSTHostname/url
J_username=$userJ_password=$pwd
ConstantTimer300ms
HTTP sampler GETlogout
AssertiomCheck login
string
![Page 18: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/18.jpg)
© Materna GmbH 2016 www.materna.com
Example: Tomcat admin uses manager app
18
![Page 19: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/19.jpg)
© Materna GmbH 2016 www.materna.com
Example: Tomcat manager app JMeter testplan (skeleton & meat)
19
![Page 20: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/20.jpg)
© Materna GmbH 2016 www.materna.com
JMeter test plan recorder (HTTP Proxy Server) elements
21
Alternative: BlazeMeter's Google Chrome extension, .har file to a JMeter test plan (.jmx file) converter
![Page 21: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/21.jpg)
© Materna GmbH 2016 www.materna.com 22
Tips&Tricks
Use user defined variables with same name as in request Place Login CSV Data Set Config element on login page request Use Counter element and reference {counter} variable in request i.e. {user} Use non-GUI mode: jmeter -n -t test.jmx -l test.jtl Use as few Listeners as possible e.g. disable the “View Result Tree” Use CSV output rather than XML. Only save the data that you need. Use as few Assertions as possible. Disable all JMeter graphs as they consume a lot of memory Check, document testplanTestPlanCheck': --jmx <filename> [ --stats --tree-dump ]
![Page 22: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/22.jpg)
© Materna GmbH 2016 www.materna.com 25
Four Ways To Launch a JMeter Test without JMeter GUI
1. Command line: jmeter -n -t your_script.jmx2. JMeter Ant Task ant extras/build.xml3. JMeter Maven Plugin. pom.xml mvn verify jmeter:jmeter4. from Java code
![Page 23: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/23.jpg)
© Materna GmbH 2016 www.materna.com
JMeter distributed testing with Master-Slave
SystemunderTest
Slave1
Slave1
Slave3
Master-GUIjmeter.propertiesremote_hosts=
Slave1,Slave2,Slave3
Remote start Slave1
Remote start Slave3
Remote start All
In bin/jmeter.properties "remote_hosts=localhost:1099,localhost:2010" add IP addresses of Slave-Clients, server.rmi.port=1099, server.rmi.localport=1099, client.rmi.localport=2010 On Master: jmeter -n -t script.jmx -R slave1,slave2,slave3.„Remote Start All" distribute and execute testplan and receive results, SET SERVER_PORT=1099, set REMOTE_HOSTS=master:1099"Remote Stop“, control jmeter-server.log, and have check and assert elements, to handle errors
Remote stop
26
![Page 24: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/24.jpg)
© Materna GmbH 2016 www.materna.com
Distributed testing
27
EC2, VPNSSH-Tunneling
Running Apache JMeter on Amazon EC2ec2 Script
![Page 25: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/25.jpg)
© Materna GmbH 2016 www.materna.com
How to document test plans in HTML?
Bildschirm als Bild speicherncd apache-jmeter-3.0\extrasschematic.cmd test-plan.jmxant -Dtest=test-plan run reportTestPlanCheck.bat --jmx testplan.jmx --stats --tree-dump
Cold/warm start!Think Time!
28
![Page 26: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/26.jpg)
© Materna GmbH 2016 www.materna.com
Apache JMeter Dashboard 3.0
29
http://jmeter.apache.org/usermanual/generating-dashboard.html
Save JPG.image
![Page 27: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/27.jpg)
© Materna GmbH 2016 www.materna.com
Report Dashboard with APDEX (Application Performance Index) table
30
https://www.blazemeter.com/blog/5-key-things-you-need-know-about-jmeter-30jmeter.bat -n -t WorkBench_JMETER.jmx -l WorkBench_JMETER.log -e -o reporthttp://www.apdex.org
![Page 28: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/28.jpg)
© Materna GmbH 2016 www.materna.com 31
https://sense.blazemeter.com
![Page 29: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/29.jpg)
© Materna GmbH 2016 www.materna.com
Realtime results: Async BackendListener with Graphite/Grafana
32
![Page 30: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/30.jpg)
© Materna GmbH 2016 www.materna.com
https://flood.io/har2jmx
33
![Page 31: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/31.jpg)
© Materna GmbH 2016 www.materna.com
Continuous integration
34https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin
![Page 32: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/32.jpg)
© Materna GmbH 2016 www.materna.com
Useful extensions http://jmeter-plugins.org
35
![Page 33: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/33.jpg)
© Materna GmbH 2016 www.materna.com
Further challenges with JMeter
How to integrate JMeter in the CI-Build? How to document test plans and results? What further reporting tools are needed? Extending the Web Test Plan (ForEach Controller with User Defined
Variables (UDV), While Controller with StringFromFile Function, RegEx) How to use JMeter with Selenium/JS/mobile Apps/Cloud?
36
![Page 34: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/34.jpg)
© Materna GmbH 2016 www.materna.com
Further info
Gatling http://gatling.io Grinder3: http://grinder.sourceforge.net JMeter Info's: http://jmeter.apache.org, http://sourceforge.net/projects/jmeterforwindows JMeter extensions: http://jmeter-plugins.org, http://www.atlantbh.com/jmeter-components JMeter Tips: http://blazemeter.com/apache-jmeter, http://jmeter-expert.blogspot.de JMeter Master-Slave-Test: http://jmeter.apache.org/usermanual/remote-test.html JMeter MAVEN Plugin: https://github.com/jmeter-maven-plugin, Online-Reports: https://sense.blazemeter.com/ https://blazemeter.com/blog/open-source-load-testing-tools-which-one-should-you-use JMeter-ec2 script: https://github.com/oliverlloyd/jmeter-ec2
37
![Page 35: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/35.jpg)
© Materna GmbH 2016 www.materna.com
Eights Anniversary of the Twitter Fail Whale
38
![Page 36: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/36.jpg)
© Materna GmbH 2016 www.materna.com 39
Fragen?
![Page 37: With Apache JMeter? Frank Pientka - Home: DOAG e.V. · PDF filePerformance testing With Apache JMeter? Frank Pientka ... Why performance testing? Testing right! What is Apache JMeter?](https://reader030.fdocuments.us/reader030/viewer/2022021423/5aa642767f8b9afa758e438d/html5/thumbnails/37.jpg)
© Materna GmbH 2016 www.materna.com
Vernetzt. Kontakt.
Materna GmbHFrank PientkaVoßkuhle 3744141 Dortmund+49 231 [email protected]