NG40 Test Case Execution
21
NG40 Test Case Execution, Verification and Modification Virtual Machines 1
-
Upload
sunny-sapru -
Category
Documents
-
view
41 -
download
2
description
ng4t
Transcript of NG40 Test Case Execution
NG40
Test Case Execution,
Verification and Modification
Virtual Machines 1
Execute a Test Case
Company & Products Page 2
Test case execution from CLI:
Test case name
NG40 Shell command
Test Case Output
Company & Products Page 3
End of test case execution Prompt
Test verdict: None (NONE, PASS, FAIL)
Verify
Company & Products Page 4
Command “verify” returns statistic
All those parameters can be used for test verification (test result)
Verify
Company & Products Page 5
- Statistic are global values – information of a complete test campaign
-> Statistic for a single test case: Reset before test case execution
1) Reset manually from CLI: -> resetstat 2) Reset automatically from test scenario config file -> callscenario.conf
Test Case Structure
Company & Products Page 6
Call Model
Test Case
Test Verdict (optional)
Verdict = PASS
Company & Products Page 7
More Complex Test Verdicts
Company & Products Page 8
- bigger, smaller
- sets - direct comparison of messages
- absolute numbers
- equations
If a test case fails…
Company & Products Page 9
- Leave NG40 Shell -> shutdown, abort - ls -lsrt log - List of logfiles as depicted below
- vi log/ <logfile name> - shift-g -> Root Cause
Call Model
Company & Products Page 10
A Call Model consists of (phases):
- Begin Scenario - Loop Scenario - End Scenario - Recovery Scenario (optional)
Hint: Duration of the Loop Scenario is defined in the Test Case as following: period[x].duration = <time in seconds> period[x].duration = 0 infinite
time
traffic
loop
end begin
Call Model - Phases
Company & Products Page 11
1. Begin Scenario: - Executed once at the beginning - Intention is to change the state of the emulated subscriber - From “idle mode” to “test mode”
2. Loop Scenario: - Executed several times as defined in the Test Case
- period[x].duration = 60 [in seconds] - period[x].duration = 0 [infinite ]
3. End Scenario: - Executed once at the end - Intention is to change the state of the emulated subscriber - From “test mode” to “idle mode”
Call Model - Phases
Company & Products Page 12
4. Recovery Scenario (optional): - Will be executed in case of irrecoverable errors - Intention is to bring the subscriber back into a defined state - Recovery Scenario will be triggered after:
- Timeouts, like no Tracking Area Update Accept after Tracking Area Update Request
- Reject Messages, like Security Mode Reject - Abnormal Cause values, like CS domain temporary not available - Predefined limit of message repetitions
Combination of Call Models
Company & Products Page 13
- Every Call Model has a unique name
- Several Call Models may coexist in one Test Case
- Typical example for concurrent call models: - group[m] = called party - group[n] = calling party
- Starting time and duration of the periods are defined in the test case
Concurrent In sequence
period[a].group[x] period[b].group[y]
period[c].group[z]
period[x].group[m] period[x].group[n]
period[x].group[o]
Combination of Call Models
Company & Products Page 14
Learnings:
- Groups will be executed concurrently
- Periods will be executed sequentially
Test Structure - Example
Company & Products Page 15
Call Models
Groups
Name Group index Period index
Important Design Principles
Company & Products Page 16
1. All groups (group[m], group[n], group[o]) are independent from each other
2. A emulated Mobile Subscriber can only belong to one group • Subscriber with IMSI = 262 01 9876543210 belongs to group[m] • Same subscriber must not belong to any other group
3. A certain group can be used only once at a time
4. Call models are subscriber-agnostic
Why? Answer: A mobile subscriber cannot assume several roles, like Called Party and Calling Party at the same time.
Test Flow – Generic Overview
Company & Products Page 17
period[0] BEGIN_SCENARIO LOOP_SCENARIO END_SCENARIO
period[1] BEGIN_SCENARIO LOOP_SCENARIO END_SCENARIO
period[2] BEGIN_SCENARIO LOOP_SCENARIO END_SCENARIO
… period[m] BEGIN_SCENARIO
LOOP_SCENARIO END_SCENARIO
time
Scenarioswitchmode
Company & Products Page 18
Two ways how subscribers move through SCENARIOS and PERIODS: 1. concurrent or direct: - All subscribers pass scenarios independently
from the remaining subscribers - E.g. some subscribers still in END_SCENARIO, of in period[0], while others already in period[1] and period[2]
period[0] BEGIN_SCENARIO LOOP_SCENARIO END_SCENARIO period[1] BEGIN_SCENARIO LOOP_SCENARIO END_SCENARIO period[2] BEGIN_SCENARIO LOOP_SCENARIO END_SCENARIO
Scenarioswitchmode
Company & Products Page 19
Two ways how subscribers move through SCENARIOS and PERIODS: 2. up/down: - All subscribers move “together” to the next scenario
- Next scenario after last subscriber passed previous scenario - e.g. all subscribers in LOOP_SCENARIO of period[1]
period[0] BEGIN_SCENARIO LOOP_SCENARIO END_SCENARIO
period[1] BEGIN_SCENARIO LOOP_SCENARIO END_SCENARIO period[2] BEGIN_SCENARIO LOOP_SCENARIO END_SCENARIO
Test Case Structure – Summary
Company & Products Page 20
1. A Test Case is a series of Periods
2. Every Period is a combination of Subscriber Groups - Subscriber Groups belonging to the same Period run concurrently - Therefore the requirement that a certain subscriber must belong to exactly one group 3. To every Group assigned is exactly one Call Model
This structure is depicted at the next slide.
Test Case Structure – Summary
Company & Products Page 21
Test Case = rt_ps_call
Mobile Subscriber
group[0]
Mobile Subscriber
group[x]
Mobile Subscriber
group[y]
Standard Subscriber
group[x]
Call Model
loop
end begin
Call Model
loop
end begin
Call Model
loop
end begin Call Model
Test Case
Group
period[0] period[1] period[2] … Period
sequentially
International Roamer
group[y]
Fraudulent Subscriber
group[z]
concurrently ps_call
