Multi-platform Automated Software Building and Packaging
description
Transcript of Multi-platform Automated Software Building and Packaging
Multi-platform Automated Software
Building and PackagingAndres ABAD RODRIGUEZ (IT-GT / CERN)
V.E. GOMES GOUVEIA (IT-GT/CERN), D. MENESES (IT-GT/CERN), F. CAPANNINI (INFN), A. AIMAR (IT-GT/CERN), A. DI MEGLIO (IT-DI/CERN)
v
Products & Languages
BDII
AMGA
Logging and Bookkeeping
APEL
FTS
UNICORE Container
EMIR
MPI
UNICORE security SAGA
VOMS
StoRM
ARC Infosys
gLite Security
CANL
Infosys
Argus
dCache
CREAM CE
LFC
WNoDeS
DPM
ARC Clients
ARC CE
EMI Messaging
WMS
The EMI (European Middleware Initiative) project is the integration of the main European pre-existing middlewares (gLite, UNICORE, ARC and dCache) into a single consistent set of packages with uniform distribution and repositories.
The products that became part of EMI were built using different tools and dedicated services. The software, millions of lines of code, is written in several programming languages and platforms.
Despite this heterogeneous initial situation, a single common solution was needed for building, testing, packaging and QA.
The ETICS service, already used by the gLite components, was selected for the EMI project.
Build/TestArtifacts
ConfigurationWeb Service
Report/MetricsDB
ConfigurationDB
Condor Execution Engine
RepositoryWeb Service
Virtual WorkerNodes
EMI build, test and packaging infrastructure
ETICS allows the specification of the check out, build and test commands, dependencies specification and environment definition of the whole software stack.
The ETICS jobs are run on demand created Virtual Worker Node. The platforms supported in EMI are Scientific Linux 5 and 6 (32 and 64 bits) and Debian 6 (64 bits). This solution reduces the number of physical machines to the minimum (only for the main servers).
Multiple metrics can be collected during the build execution. There are many out of the box metrics plugins for static source code analysis such as Findbugs, pUnit, etc. The system can be easily extended to support other QA tools. The reports and artifacts are stored in the repository and the worker node is reused.
Other interesting features are multi node testing and the repository graphical interface from where the artifacts can be easily accessed and browsed.
EMI repository
RPM packages DEB packages Source andBinary TAR.GZ Reports & metrics
ETICS also builds software using standard tools such as Mock and Pbuilder. The artifacts produced by the system are compliant with the EPEL and Debian guidelines. The compliance level is automatically monitored running the RPMLint and Lintian plugins.
The packages created, reports and metrics generated are stored in the EMI repository. These packages are ready to be included in the Debian, Fedora EPEL and EGI UMD repositories.
For more information: http://cern.ch/emi-tools-chep2012
EPEL Repository EGI UMD Repository Squeeze Repository
YAIM
glExec
LCAS
Scientific Linux