Post on 08-Jan-2018
description
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
ARC tools for revision and nightly functional tests
Jozef Cernak, Marek Kocan, Eva Cernakova
(P. J. Safarik University in Kosice, Kosice, Slovak Republic)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Testing strategy in ARC• ARC-EMI testing tools
– Build testing• Nightly (every night)• Revision (after each commit of developers)
– Automatic functional testing of nightly builds• Future plans
– Increasing number of automatic functional tests– Automatic performance testing
• Conclusion
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 2
Outline
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Several roles in project: developers, testers and ARC release manager • Adopted tests:
– Unit tests(developers)– Build tests (testers)
• Revision (Kosice) • Nightly (Copenhagen)• EMI-RC (ETICS)
– Functional tests (testers, developers)• Manual • Automatic (Kosice)
– Regression testing (testers, developers)– Performance testing (testers)
• Manual• Automatic in development phase
– Large scale testing (in preparation phase)Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 3
Testing practice in ARC
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611 ARC-EMI testing tools
Relation between ARC and EMI testing tools
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 4
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Block diagram of automatic revision and Functional toolsSimple database server with PHP and Python scripts
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 5
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Infrastructure (local cluster)– 2 SUN 2200M2 servers (2x4GB RAM, 250GB
disk, AMD 64 bits)– 2 desktops (2x 1GB RAM, 500GB disk 32 bits)– 1 MAC mini (2GB RAM, 160GB, 32 bits)
• Software– MySQL, PHP, Python– Code is free
• Distributed data processing – SVN
Technical details
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 6
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Features– Supported are several platforms of OS:
• CentOS (32,64), • Debian(32,64), • and MAX OS (64)
– Detailed identification of errors during building process• Autogen• Configuration• make• make dist• Build
Revision automatic testing
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 7
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
– User friendly interface to define search criteria• Daily• Weakly• Monthly • Custom
– Revision number, etc (see next examples)
• Direct connection with SVN• Warning messages for developers if code is broken
– CCCC metric – Quick mechanisms to identify breaking event
…continued.
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 8
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Example of www interface for automatic testing toolsHome page: http://arc-emi.grid.upjs.sk/ (Revision database is slow)
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 9
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Several criteria to search results of revision testing
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 10
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611 Example of the results of daily revisions
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 11
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Features– Automatic download code from SVN based on several criteria:
• trunk, • revision code,• nightly builds
– Automatic build– Automatic deployment– User friendly interface for:
• submitting proposals of test cases• to search the results of functional tests
– Tests are grouped into two main groups:• functional tests of server,• functional tests of client.
Automatic functional tests
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 12
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611 Interface to enter search criteria
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 13
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611 The results of automatic functional tests
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 14
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• client_arcsub_to_arex_gridmap_simplejob_jsdl– Tries to submit a simple JSDL job to a grid-map secured A-REX service.
• client_arcsub_to_grid-manager_gridmap_simplejob_jsdl– Tries to submit a simple JSDL job to a grid-map secured ARC0 CE.
• client_arcsub_to_GIIS_simplejob_jsdl– Submission of a simple JSDL job to a GIIS service.
• client_arcsub_to_ISIS_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud.
• client_arcsub_random_broker_isis_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud using random broker.
• client_arcsub_random_broker_giis_simplejob_jsdl– Submission of a simple JSDL job to an GIIS cloud using random broker.
• client_arcsub_fastestqueue_broker_isis_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud using fastest queue broker.
• client_arcsub_fastestqueue_broker_giis_simplejob_jsdl– Submission of a simple JSDL job to a GIIS service using fastest queue broker.
• Failed– There was some error before we managed to start testing. The error could have been in
code retrieval, installation or somewhere else.
Functional test scenaria
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 15
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Requirements:– several servers and clients– Monitoring of several distributed processes:
• resources: – Memory– CPU– Network usage
• Performance– Scalability– Number of concurrent requests
• Reliability of services:– Ratio between successful tasks and total tasks.
Examples of tasks: job submission, file transfer and etc.
Automatic performance tests
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 16
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Proposed infrastructure:– Grid servers and clients, we plan to use
existing infrastructure– Communication layer • XML-RPC client-server architecture
– Central database server– Common Python library on each server and
client (for example start stop service)– Central control program
… continued.
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 17
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Automatic testing covers:– Build process (ARC, ETICS),– Functional,– Performance tests (in progress).
• Utilization of automatic test tools increases probability to find weak part of the software. On the other hand, manual tests are important.
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 18
Conclusions
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Testing in ARC– http://wiki.nordugrid.org/index.php/Testing
• Revision tests– http://download.nordugrid.org/builds/– http://arc-emi.grid.upjs.sk/revisionTests.php
• Functional tests– http://arc-emi.grid.upjs.sk/functionalTests.php
• Code:– http://svn.nordugrid.org/trac/workarea/browser/
ARCTestScriptshttp://svn.nordugrid.org/trac/workarea/browser/ARCTestScripts
References
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 19
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Thank you
Use Insert Header & Footer to set this field
20Use Insert Header & Footer to set this field
EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611