How to Make Test AutomationFor Cloud-Based System

by Nick Babich

Telephony Automation

TeamAlexey Georgy Alexandra

Nikita Alexey Igor2

Contents

1.RingCentral product and system specifics.

2.Automation: What-When-Where-How and Why

3.Questions and Answers3

4

Classic PBX System

5

Cloud Telephony System

RingCentral: Cloud Business Communications

6

RingCentral ServiceTelephony is a cornerstone service for the RingCentral. It

consist of many different features and each new release brings more to production.

7

8

Quality

Cost

Delivery

AutomationWhat?When?Where?How?Why?

9

What?Type of Automation 10

11

Testing Types

● Functional Tests● Acceptance

Tests

● A/B Tests

● Unit Tests ● Stress Tests● Security

Tests

Business Facing

Technology Facing

Supp

ort

Prog

ram

min

g

Criti

que

Pr

oduc

t

12

13

Most Common Use-Case

How to Define Scope for Automation● Automate the most critical cases (P0 cases,

especially one which are hard-to-execute manually).

● Automate the most frequently used scope (included in regular regression cycles).

● Automation for the most automatable scope (easy to automate and move to the execution).

Make a distinction between “ephemeral” and “long-standing” cases

14

Test Automation Criteria

15

When?General Automation

Flow16

Product Release Cycle

17

Agile Development

18

Testing Process

Create new test scope Define a scope for automation & automate it

Execute automated scope

Who: Scrum team QA engineers.

Task: Create test design for new features.

Deliverable: Prioritized test scope, proposal for automation

Who: Automation QA engineers.

Task: Automate test cases

Deliverable: Test scripts ready for execution

Who: Scrum team QA or DEV.

Task: Execute automated test cases.

Deliverable: Execution results. Understanding product quality.

19

Test Automation Plan

Development Team

Step 1: Starting development new telephony feature.

Development Team

Step 2: Prioritized test design for new functionality

Development Team

Step 3: Automation for the most critical test cases

Telephony Automatio

n Team

Step 4: Continue automation for 2-nd and 3-rd category

Telephony Automatio

n Team

Step 5: Delivering automated cases to the customers

20

Where?Test Environment21

22

Test Environment and Production

Test Environment Update

ADB

Codebase change or

Manual request

Start Automatic

Deployment System

SVNNew Build

23

Automatic Deployment System

3

2

1

Deploy → Verify (Post Deployment Verification) → Execute Regular Scope

24

25

Distributed Test Execution Scheme

How?Test Execution

Scheme26

Tools• Framework and automated tests cases (scripts).• Test Environment• Dashboard for test execution results and reports.• Bug tracking system

27

Technologies We Use

28

How It Works

29

Execution Scheme

Test Environment Deployment

Test Scope Execution

Send Execution Results to

Dashboar

d

New Build Available

30

Framework and Automated Test Cases

Business Layer

Test Scripts Layer

Application Part (exe)Command line application with a set of parameters (What we execute and Where)Set of modules required for test scripts (WebDriver, Rest API, Oracle ORA) Automated test cases (test scripts)

Powered by 31

Test Case Overview

32

Test design and Test script

33

Test script report

Powered by 34

• Start time (timestamp)• Group (e.g. feature)• Execution result (Passed / Failed / Skipped)• Screenshots and attachments (if test failed during execution ).

Execution Results for the Scope● Archive for all test reports

● Tested build (Branch and Build number)● Backend log data and system traces

Powered by 35

Execution Report in Dashboard• Configuration properties• Backend build (from execution results)• Execution Statistics (passed / failed / blocked)

36Powered by

Powered by

Dashboard Sample

37

Why?38

39

Metaphor Behind Test Automation

40

Cost: Manual vs. Automated Testing

41

Benefits of Automated Testing

Global Continuous Integration Team

6 Development Team

Regression Team

Total number of test cases: 5500Total number of Automated: 2100

On a daily basis

On a weekly basis

On a daily basis

Automated

Test Automation Disadvantages

42

1. Test scripts base requires regular support and actualization.

2. Troubleshooting for environment deployment and execution results analysis.

3. It works only on long-term basis.

Important Moments●Feedback from

stakeholders●Task Prioritization●Popularization of

automation 43

44

Contact Information

nick@babich.bizwww.babich.bi

z

www.linkedin.com/in/nbabich 45

twitter.com/101babich