Useful Tools for Testing Aled Smith. This presentation will be looking at two useful tools for...
-
Upload
arleen-pope -
Category
Documents
-
view
217 -
download
0
Transcript of Useful Tools for Testing Aled Smith. This presentation will be looking at two useful tools for...
• This presentation will be looking at two useful tools for performing tests:1. Virtual Machine Creator
• https://svn.cern.ch/reps/en-ice-svn/trunk/utilities/SIC/virtual-environment-setup/vm_creator/
2. Unit-Test-Harness• https://svn.cern.ch/reps/en-ice-svn/trunk/utilities/SIC/virtual-environment-setup/Unit_Test_Harness
• A use case for each of these tools will also be demonstrated.
Useful Tools for Testing
VM Creator: OverviewWhat is it?• The Virtual Machine Creator is a Python based script that fully automates the process of creating and
configuring OpenStack virtual machines.
How does it work?• The VM creator uses three main components:
• Fabric - A Python based SSH wrapper.• JenkinsAPI - A Python based API for Jenkins.• Nova - A Python based API to OpenStack.
What can it do?• Fabric provides the Virtual Machine Creator with a Python based SSH interface to a virtual machine so that
any SSH tasks like installing and configuring software can be executed from a Python script. It also provides built-in functions to perform tasks like sudo commands and file transfers.
• JenkinsAPI allows the Virtual Machine Creator to interact with Jenkins and performs tasks like registering new slave nodes and executing Jenkins jobs.
• Nova allows the Virtual Machine Creator to interact with OpenStack and perform tasks like creating, deleting and modifying virtual machines.
Jenkins OpenStack
JenkinsAPI Nova
SSH
Fabric
VM Creator
• This use case will create a CERN Scientific Linux 6 Jenkins slave virtual machine running with WinCC-OA installed for the purposes of running test projects using Jenkins.
VM Creator: Use Case
Generate new key-pair
Create virtual machine
Expand main partition
Add ‘jenkinst’ service account
InstallWinCC-OA
InstallSubversion
Install PVSSBootstrapper
Install Java
Connect VM to Jenkins
Install X-Window
• This basic flow chart shows the main tasks that the Virtual Machine Creator must complete for this particular use case.
• However, the Virtual Machine Creator is not restricted to any specific use case and the user can pick and choose the functions they’d like to use and also add their own.
• This use case will create a CERN Scientific Linux 6 Jenkins slave virtual machine running with WinCC-OA installed for the purposes of running test projects using Jenkins.
VM Creator: Use Case
Generate new key-pair
Create virtual machine
Expand main partition
Add ‘pvss’ local account
InstallWinCC-OA
InstallSubversion
Install PVSSBootstrapper
Install X-Window
• This basic flow chart shows the main tasks that the Virtual Machine Creator must complete for this particular use case.
• However, the Virtual Machine Creator is not restricted to any specific use case and the user can pick and choose the functions they’d like to use and also add their own.
• For example, in this use case, a CERN Scientific Linux 6 virtual machine is created with WinCC-OA installed that is not a Jenkins slave, has a local account called ‘pvss’ and has Kerberos setup so that DFS can be mounted and accessed.
Create virtual machine
Setup Kerberos
MountDFS
VM Creator: Use Case• The WinCC-OA Jenkins slave creator use case described previously has been implemented and incorporated
into a Jenkins job.
Jenkins Job: https://icejenkins.cern.ch/job/CREATE_JENKINS_SLAVE/
What is it?• The Unit-Test-Harness is a tool that automates the process of unit testing framework components within a
WinCC-OA project.
What can it do?• It can be invoked from command line using Python or from a GUI panel within the project.• It can automatically create a new WinCC-OA project and install the required components• It can automatically run the unit tests and publish the results in Junit XML format.
How does it work?• The user creates a ctl script containing the unit tests.• If executed from the GUI panel, the Unit-Test-Harness finds any unit tests within the project and presents
them to the user. The user can then select the tests that they want executing.• If executed from the command line, the user needs to describe the components that need to be
installed/tested, their sources and the location of the unit test ctl script in an XML file.• The Unit-Test-Harness can then parse the XML and either create a new test project or use an existing one.• It then performs the desired units tests and publishes the results in Junit XML format.• Finally, the Unit-Test-Harness can also delete the project and clean-up any downloaded files.
Unit-Test-Harness: Overview
Start via GUI panel.
Find available unit tests within project.
Run the unit tests.
Publish the results of the unit tests.
End via GUI panel.
User selects unit tests.
CTRL Python
Delete project
Start via command line.
Use existing project?
Check project is registered and start it.
Create project, install desired components
and start project.
Start Unit-Test-Harness within project.
Copy results from project and stop project
Clean-up project?
End via command line.
Yes
No
Yes
No
Unit-Test-Harness: Overview
• This use case is used to test whether an RDA3 client correctly receives different formats of data while the RDA3 server continuously iterates the data it publishes.
• It checks that the client is receiving the data correctly by using a listener that checks each iteration of data that is received and records any missed or incorrect iterations.
• This use case requires some logic to be run outside of the Unit-Test-Harness such as starting the RDA3 Server.
• The Unit-Test-Harness can therefore be run separately in two ways:1. With runTestsImediatley=False, the Unit-Test-Harness will setup the project
and download and install the desired components but not run the unit tests.2. With useExistingProject = True, the Unit-Test-Harness executes the units
tests and retrieves the results on the already created project.• This use case was also incorporated into a Jenkins job and is shown in the
following flow chart:
Unit-Test-Harness: Use Case
Unit-Test-Harness: Use CaseStart via GUI panel.
Find available unit tests within project.
Run the unit tests.
Publish the results of the unit tests.
End via GUI panel.
User selects unit tests.
CTRL Python
Start Jenkins job
Start RDA3 client
Start RDA3 server
Start listener and leave running for fixed time
End of Jenkins job
Publish test results
Jenkins
Initialise listener values
Stop listener
Delete project
Start via command line.
Use existing project?
Check project is registered and start it.
Create project, install desired components
and start project.
Start Unit-Test-Harness within project.
Copy results from project and stop project
Clean-up project?
End via command line.
Yes
No
Yes
No
runUnitTests.py
runUnitTests.py
runTestsImmediately = False
useExistingProject = True
• In this screenshot, you can see the XML code within the XML configuration file that is passed to the Unit-Test-Harness through the command line.
• The XML file describes the components that need to be installed, their sources and also defines the location of the test suite file.
• Specifying just the test suite tells the Unit-Test-Harness to execute all of the unit test cases but these can also be specified individually using the <testCase> tag.
Unit-Test-Harness: Use Case
• In this screenshot, you can see a snapshot of the code within the test suite file.
• In this ctl script, the user defines the unit test cases and their logic.
• Each unit test should have an assert routine.
• The Unit-Test-Harness supports the following assert routines:
• assertError()• assertEmpty()• assertEqual()• assert()• assertTrue()• assertFalse()
Unit-Test-Harness: Use Case
Jenkins Job: https://icejenkins.cern.ch/job/RDA3_CLIENT_MONITOR/
Any questions?
Useful Links:Virtual Machine Creator: • https://svn.cern.ch/reps/en-ice-svn/trunk/utilities/SIC/virtual-environment-setup/vm_creator/
Unit-Test-Harness: • https://svn.cern.ch/reps/en-ice-svn/trunk/utilities/SIC/virtual-environment-setup/Unit_Test_Harness
Fabric:• http://docs.fabfile.org/en/1.10/
JenkinsAPI:• http://jenkinsapi.readthedocs.org/en/latest/
Nova:• http://docs.openstack.org/developer/python-novaclient/