Manage Pipeline of Performance Testwith Jenkins

Vol.01 Mar/07/2017Takeshi TakizawaEcosystem Service Department, Rakuten, Inc.

Self-introduction

2

DEMO

3

Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)

Test Client

Test Client

Test Client

Performance Test Pipeline

4

There are some issues😖😖😖

5

Performance Test Pipeline

6

• It’s difficult to estimate concrete requests 📈• Need to clarify that requests come permanently

or instantly 📊

Performance Test Pipeline

7

• Need programming knowledge 📚

Need programming knowledge

• It’s not 10 QPS 🙅

8

• Expected: Send requests in parallel

Need programming knowledge

9

Test Client APP

Need programming knowledge

• It’s not 10 QPS 🙅

10

• Actual: Send requests sequentially

Need programming knowledge

11

Test Client APP

Performance Test Pipeline

12

• Need infrastructure knowledge 📚• How to install packages• How to configure number of file descriptor

• Sometimes Test ENG doesn’t have enough permission to build server so need to ask INFRA ENG 🐢

Performance Test Pipeline

13

• Managing distributed test clients is complicated 👯

When I want to send heavy requests

• 1 Test Client doesn’t make enough load

14

Client APP

I can send 1000 QPSI can respond 2000 QPS

When I want to send heavy requests

• Scale Up Spec of Client Server?

15

Client APP

I can send 2500 QPS I can respond 2000 QPS

When I want to send heavy requests

• Scale Out is better because more flexible

16

Client

APPClient

Client

We can send 3000 QPS

I can respond 2000 QPS

But

17

Managing distributed test clients is complicated

• Concurrency 👯– Kick test clients at the same time– Only one Client is running on one Server

18

Performance Test Pipeline

19

• It’s boring to generate Test Report from distributed logs manually 🙏

Performance Test Pipeline

20

• Sometimes Test ENG doesn’t have enough permission to delete server so need to ask INFRA ENG 🐢

• People make mistakes 😲

Automate / Improve Test Pipeline

21

Not Coveredby this slide

Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)HashiCorp (https://www.hashicorp.com/)Chef Software, Inc. (https://www.chef.io/)

Need programming knowledge

• Gatling code is writable & readable 🙆

22

Test ENG doesn’t have enough permission

23

• Provide Jenkins jobs to create/delete new Test Client Servers (Jenkins Slaves)

Manage Test Clients with Jenkins Pipeline

24

Pipeline Code Sample

25

26

27

28

29

30

31

Collect Test Logs

• Use stash / unstash DSL

32

Generate Report from Test Logs

• gatling command w/ --reports-only option

33

Generate Report from Test Logs

34

Publish Test Report

• Use other Jenkins plugins through pipeline– HTML Publisher Plugin– Compress Artifacts Plugin

35

36

Automate / Improve Test Pipeline

37

Not Coveredby this slide

Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)HashiCorp (https://www.hashicorp.com/)Chef Software, Inc. (https://www.chef.io/)

Manage Pipeline of Performance Test with Jenkins

38

Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)

We’re Hiring Senior QA Engineer!!

Looking for a talented and experienced Senior QA engineer

with over 3 years of experience in software QA who can take on challenging tasks in running

full-scale testing on company’s worldwide services

https://jobs.rakuten.careers/careersection/rakuten_ext_cs/jobdetail.ftl?job=00004405

39