1

IVT Sandbox Guide

2

Contents

Overview .............................................................................................................................................. 3

Pre-requisites ........................................................................................................................................ 3

Step 1: Reserve the IVT Sandbox: .......................................................................................................... 3

Step 2: Attend Reservation and Review Workflow Commands ............................................................... 9

Step 3: Open Test Plan ........................................................................................................................ 11

Step 4: Execute Test Plan..................................................................................................................... 11

Step 5: Submit Results ......................................................................................................................... 11

Endpoint IVT Test Section Summary & Steps ........................................................................................ 12

Section 1: DUT (Device-under-Test) Integration ................................................................................... 12

Section 2: Manual Features and Services Tests .................................................................................... 12

Section 3: Load Test ............................................................................................................................ 13

Section 4: Negative Tests .................................................................................................................... 13

Section 5: Preliminary IVT Test Report Generation ............................................................................... 14

Sandbox Instructions and Commands .................................................................................................. 15

Access Sandbox Servers via SSH or RDP ............................................................................................... 18

Invite Users to an Active or Scheduled Reservation .............................................................................. 19

Enable/Disable Texting Feature .......................................................................................................... 20

Sandbox Idle Timeout ......................................................................................................................... 21

Instructions to retrieve CDR from CUCM .............................................................................................. 22

Remote Phone Control Instructions ..................................................................................................... 25

Remote Phone Control: 2-Way Audio Path Simulation ......................................................................... 27

Command Details and Input ................................................................................................................ 29

Useful Links ........................................................................................................................................ 30

3

Overview Welcome to the Cisco DevNet Sandbox IVT Program. This document will guide you through the execution of an Interoperability Verification Test (IVT) in the DevNet Sandbox.

Pre-requisites If you are using this sandbox to complete an IVT in order to get a Cisco Compatibility Logo, you need to make sure you have completed the following:

Registered/Approved Solution Program Partner Member (SPP) Approved SB IVT Request for product to be tested

Step 1: Reserve the IVT Sandbox:

Go to https://developer.cisco.com/sandbox

Click here to Login

4

Login to DevNet Portal using CCO ID

Create a new account if you do not have a CCO ID Go to Sandbox Click either one to go to Sandbox main page

5

Click here to view all Sandboxes & Reservations

Click here to view all Sandboxes Click here to view reservations

6

Reserve an Endpoint IVT Sandbox

7

Access information and credentials will be sent to user’s email when sandbox is ready

8

Reservation Schedule Conflict

System will list available timeslots if your reservation schedule conflicts with an existing reservation

9

Step 2: Attend Reservation and Review Workflow Commands Click here and select “Reservations” to view all your active, scheduled or ended reservation

Clicking on the active or pending reservation link will take you to the respective IVT Sandbox

10

Active EP IVT Sandbox Reservation

11

Step 3: Open Test Plan Open the test plan located under the “Endpoints” tab in the “Instructions” pane of the Sandbox Portal. This document will be used in conjunction with the Test Plan to instruct you on executing this IVT. Refer to Section Endpoint IVT Test Section Summary & Steps to guide you through this IVT.

Step 4: Execute Test Plan Begin test execution and record test result for each test case. This completed document is required for completion of the IVT and will be reviewed by Cisco and stored with your IVT record.

Step 5: Submit Results After you have completed all tests defined in the test plan and all workflow steps – go to the Wrap_Up command Category to generate a preliminary IVT Test Report. This step collects test logs & performance data during the IVT session. You will need to submit the completed IVT test plan & test report to Cisco. .

12

Endpoint IVT Test Section Summary & Steps

Section 1: DUT (Device-under-Test) Integration

Test Case: EP-1

Workflow Steps Description

1

EP Sandbox “Commands” pane, open Part1: Setup Run Cmd: Start_Traffic (Duration: 8 Days)

Starts basic call traffic (AB) to provide background traffic during IVT

2

Pause 10 mins after starting traffic in step 2

Ensure traffic is fully started before any further testing

3 EP Sandbox “Commands” pane, open Part1: Setup Run Cmd: My_Endpoint_Registration

Provision 3 DUT(s) in CUCM using Sandbox command

4 Refer to EP-1 test case in Test Plan to update the 3 DUT(s) device attributes accordingly. Verify device network settings and dial tone

Update device settings to validate DUT integration

5 EP Sandbox “Commands” pane, open Part1: Setup Run Cmd: Record_Connectivity_Validation

Record test step with a “Pass” when DUT is successfully integrated with CUCM

Section 2: Manual Features and Services Tests

Test Cases: EP-2 to EP-42

Workflow Steps Description

1 Launch Remote Phone Control via RDP Remote Phone Control User Instructions

Access Credentials

2 Execute test cases: EP-2 to EP-42 RPC Tool: Phone screen updates are delayed for Cisco IP Phone models - 89XX & 9951 due to firmware limitation. Known issue. Workaround: Use 79XX, 88XX and 9971 models

Make manual calls using Remote Phone Control Tool (RPC)

3 Execute and validate each test case by following the test steps for each test case in the Test Plan

Execute all the manual test cases

4 EP Sandbox “Commands” pane, open Part2: Functional Testing Run Cmd: Record_Manual_Call_Test_Execution

Record test step when all the manual test cases are executed.

13

Section 3: Load Test

Test Case: EP-43

Workflow Steps Description

1 Execute the Load Test preparation test steps in EP-43 Enable auto-answer for the 3 DUT(s) listed in test case EP-43

2 EP Sandbox “Commands” pane, open Part3: Load Testing Run Cmd: Load_Test (Duration: 8hrs)

Starts basic call traffic (AB) for 8 hrs

3 Check Output window for traffic statistics when traffic run ends

Email notification upon completion Text notification - optional

Section 4: Negative Tests

Test Cases: EP-44 to EP-47

Workflow Steps Description

1 EP Sandbox Commands pane, open Part4: Negative Testing Run Cmd: Start_Traffic_and_Restart_PUB (Duration: 30 mins) Note: Refer to test case EP-44 for detail test steps

Starts basic call traffic (AB) CUCM-PUB is restarted automatically after 5 mins of background traffic

3 EP Sandbox Commands pane, open Part4: Negative Testing Run Cmd: Start_Traffic_and_Restart_SUB (Duration: 30 mins) Note: Refer to test case EP-45 for detail test steps

Starts basic call traffic (AB) CUCM-SUB is restarted automatically after 5 mins of background traffic

4 Execute test case EP-45 & EP-46 Note: Refer to test case EP-46 & EP-47 for detail test steps

DUT power failure and network failure tests executed manually

5 EP Sandbox Commands pane, open Part4: Negative Testing Run Cmd: Record_Negative _Test_Execution

Record test step after executing all the negative tests

14

Section 5: Preliminary IVT Test Report Generation

Test Result Submission for Review

Workflow Steps Description

1 Test Wrap-up and Submission Complete Test Report and submit for review.

2 EP Sandbox Commands pane, open Part5: Wrap Up Run Command: Generate_Report

Generates Preliminary IVT Test Report upon IVT completion with the following options: Preview - Email a preview of IVT Test Report to User Submit - Email IVT Test Report to User and Cisco Clear_All - Clears all logs and counters

15

Sandbox Instructions and Commands 1) From Sandbox portal, click on “Instructions” for sandbox description, access methods etc. 2) Click on Commands for sandbox specific workflow commands

16

17

18

Access Sandbox Servers via SSH or RDP To access any servers in the sandbox topology, hover mouse on server in the virtual sandbox topology, then select “Attributes”. You can use this method for getting credentials for any server that is enabled for access in this sandbox

19

Invite Users to an Active or Scheduled Reservation

Shared user will be notified via an email with sandbox access details

20

Enable/Disable Texting Feature

Email notifications sent to users for the following events: 1. Sandbox Reservation started 2. Sandbox is Ready 3. Any commands with “Start_Traffic” is completed 4. Load_Test Completion 5. Generate_Report 6. Sandbox reservation ends 7. Any errors encountered in any command steps

Text Notifications sent to users for the following events: 1. Sandbox is Ready 2. Any commands with “Start_Traffic” is completed 3. Load Test Completion

Note: Text message for “Lab is Ready” will only be active for future reservations

21

Sandbox Idle Timeout When Sandbox is sitting idle for a certain period, you will see a screenshot similar to the one below. To get back into the SB Portal, do the following: Click on “Go to Login” Select “Go to Sandbox” if you are still logged into the DevNet Portal

22

Instructions to retrieve CDR from CUCM

1. Login to CUCM Serviceability Page: https://10.10.20.1/ccmservice/ (Administrator/ciscopsdt)

2. Navigate to ToolsCDR Analysis and Reporting

3. Opens a CDR Analysis and Reporting window. Close the warning window.

23

4. Navigate to CDRSearchBy User/Phone Number/SIP URL

5. Enter the Calling or Called DN and click “Add Phone Number”

24

6. CUCM uses GMT timezone for CDR(s). Convert the Call Start Time to GMT 7. Enter the start & end date and time accordingly and click “OK” 8. Another window opens with the CDR summary record. Click on “View” to open the detailed CDR

25

Remote Phone Control Instructions This guide provides instructions on how to use the Remote Phone Control Cisco IP Telephony utility application. PhoneView provides a real-time remote view of all your IP phones and allows you to group, filter and control the states of those phones from the RPC console. It simplifies management of Cisco IP Phones. In particular it provides an intuitive graphical display of multiple IP phones' screens and allows you to manage them either individually or in sensible groups.

NOTE Phone screenshot updates are delayed for models 89XX & 9951. This is a limitation on the phone firmware. 79XX, 88XX & 9971 models perform much more efficiently.

Remote Phone Control User Instructions

Steps Comments

1 RDP to LabServices Server: 10.10.20.100 Username: ABC\Administrator Password: ciscopsdt

2 Double click Phoneview shortcut Launch RPC Console

3 RPC Console: FileOpenMavericks.pvd Load CUCM Phone database

4 RPC Console: GroupUpdateAll Sync up with CUCM Database

5 From Data Explorer, select the following options: Device Poollocal_mav_pool Device Poolremote_mav_pool StatusRegistered

Devices filtered by device pool & status populates the screen

6 RPC Console: Phone Screenshot Automatic Update

Check “Automatic Screenshot”

Enable all automatic updates

7 Select a phone by clicking on the phone screenshot (phones with background image)

Note: Unregistered phones has no DN

Selected phone is highlighted.

8 Right click on phones of interest and select “Automatic Screenshot”

Automatic phone screen updates enabled for selected phones

9 Use phone keypad or command line to dial a number. Command Line Syntax: Dial: XXXX <CR> where XXXXDN Note: CLI dialing is faster than keypad dialing

Make a basic call to DN:XXXX Use appropriate keypad based on phone models

10 Select terminating phone, click “Answer” softkey on phone keypad to answer call

Call in “Connected” state

11 Select phone, click “End Call” softkey on phone keypad to terminate call

Call terminates normally

26

Remote Phone Control Console

Phoneview Tips 1. Phone screen size can be increased or decreased by slider bar at the bottom 2. Data ExplorerModels (lists all phone models supported by tool) 3. For optimum performance, turn on “Automatic Updates” for phones of interest only versus all phones 4. Supports most call features 5. Select 79XX keypad for 79XX Series and 89XX/99XX keypad for 89XX/99XX Series phones 6. Remote phone control supported on CUCM multi-cluster topology 7. All Registered IP Phones has a background image and a DN assigned to it 8. Command Line dialing is faster than keypad dialing (Syntax :- Dial:XXXX <CR> where XXXX is the DN) 9. Phone screenshot updates are delayed for models 89XX & 9951. This is a limitation on the phone

firmware. 79XX, 88XX & 9971 models perform much more efficiently. 10. If RPC Tool is unstable, exit out of application and re-launch a new instance

27

Remote Phone Control: 2-Way Audio Path Simulation Two-Way Audio Path Simulation for calls originated & terminated to RPC Tool: 1. Enable Audio and Video Playback on a local PC accessing RPC Tool Server via RDP 2. Launch RDPOptionsLocal Resources SettingsPlay on this computer

3. Establish a stable call between Phone A and Phone B using the RPC Tool 4. From RPC, select Phone A by clicking on it’s phone display – it will be highlighted

5. Click on the “headphone” icon to start a monitoring session for that phone

6. Green speaker icon with monitoring notice displayed on the phone 7. Repeat steps 4-5 for Phone B 8. On the CLI, enter “Play:AreYouThere.raw” (male voice) and hit “Send” on Phone B 9. Caller at Phone A should hear a male audio 10. Select Phone A, on the CLI: “Play:AreYouThereF.raw” (female voice) and hit “Send” 11. Caller at Phone B should hear a female audio Important Note: Phoneview PC Volume On IP Phone Volume On No Headsets connected to IP Phones controlled by Phoneview

28

29

Command Details and Input Commands are used to drive the IVT workflow for the required user interaction and steps to be followed in the sequence presented. Command output can be reviewed in the Command Output Window at the bottom of the screen or in the Activity Log. Command actions are recorded as part of the test report data. Workflow Commands These logging commands are used only once and in sequence. Every step generates log during an IVT session. EP IVT Commands Description

Start_Traffic Starts basic call traffic to simulate background traffic during IVT Testing

My_Endpoint_Registration Run command to provision & register any CUCM supported endpoint(s)

Record_Connectivity_Validation Run command after verifying product integration and validation with traffic

Record_Manual_Call_Test_Execution Run command to record completion of manual test case execution as defined in the manual functional tests section of the Test Plan.

Load_Test Run command to start basic call traffic run (Default Duration: 8 hours)

Start_Traffic_and_Restart_PUB Run command to start traffic and after 5 mins of traffic, CUCM-PUB is restarted automatically

Start_Traffic_and_Restart_SUB Run command to start traffic and after 5 mins of traffic, CUCM-SUB is restarted automatically

Record_Negative_Test_Execution Run command to record completion of negative test case execution as defined in the negative tests section of the Test Plan.

Generate_Report Run command to preview and submit a preliminary ivt test report or clear test report

Preview: Collect logs and sends an email of the test report to sandbox user

Submit: Collect logs and sends an email of the test report to sandbox user and Cisco IVT team

Clear_All: Clears all logs and counters for a fresh start of IVT

30

Utility Commands These user commands are generic and used interchangeably with the logging commands. Commands Description

Device_Status Run command to list the device status for a maximum range of 1000 DN(s)

RepairEnvironment Run command to reset the environment for a fresh start. Requires access code from IVT Sandbox Support Team. The historical logs and performance data is retained.

Note: To erase old reporting data, select “Wrap_Up” category from SB Command pane: Run “Generate_Report” with “Clear_All” option

Stop_Traffic Run command to stop basic call traffic

Command Queue Support The table lists the commands that supports command queue. EP IVT Commands Support Queue (Y/N)

Start_Traffic N

Record_Connectivity_Validation N

Record_Manual_Call_Test_Execution Y

Load_Test Y

Start_Traffic_and_Restart_PUB N

Start_Traffic_and_Restart_SUB N

Record_Negative_Test_Execution N

Useful Links Sandbox Community Support: https://communities.cisco.com/community/developer/sandbox DevNet Sandbox: https://developer.cisco.com/site/devnet/sandbox/ DevNet: https://developer.cisco.com/