Testing as a Service with HammerCloud
description
Transcript of Testing as a Service with HammerCloud
![Page 1: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/1.jpg)
IT-SDC : Support for Distributed Computing
Testing as a Service with HammerCloud
Ramón Medrano LlamasCERN, IT-SDC
15.10.2013
![Page 2: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/2.jpg)
Testing as a Service with HammerCloud 2IT-SDC
Agenda
Introduction New use cases Architectural changes Tests performed Future work
15.10.2013
![Page 3: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/3.jpg)
Testing as a Service with HammerCloud 3IT-SDC
HammerCloud
The grid testing framework ATLAS, CMS, LHCb 50 M jobs/year 20 machines cluster hosted by CERN
IT voboxes
15.10.2013
![Page 4: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/4.jpg)
Testing as a Service with HammerCloud 4IT-SDC 15.10.2013
![Page 5: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/5.jpg)
Testing as a Service with HammerCloud 5IT-SDC 15.10.2013
![Page 6: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/6.jpg)
Testing as a Service with HammerCloud 6IT-SDC
New use cases
Stress testing of sites Functional testing of sites AFT/PFT testing suite Benchmarking testing NEW! Cloud resource validation NEW! Athena nightly build system NEW! XRootD federation (FAX) NEW! ROOT I/O and WAN tests NEW!
15.10.2013
12,000 test/year
![Page 7: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/7.jpg)
Testing as a Service with HammerCloud 7IT-SDC
Example of FAX system testing
15.10.2013
![Page 8: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/8.jpg)
Testing as a Service with HammerCloud 8IT-SDC 15.10.2013
![Page 9: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/9.jpg)
Testing as a Service with HammerCloud 9IT-SDC 15.10.2013
functional testing(predictablebackground)
stress testing(on demand,
non predictable)
time
jobs
This behavior needs some elasticity
![Page 10: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/10.jpg)
Testing as a Service with HammerCloud 10IT-SDC
Testing as a Service
Need to cope with increasing demand,
Requested by users and tools on demand
Elastic testing infrastructure Testing anything that allows:
submit() status() cancel() 15.10.2013
![Page 11: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/11.jpg)
Testing as a Service with HammerCloud 11IT-SDC
Current infrastructure
20 machines manually managed Capped scalability Long time to provision Difficult lifetime management Security and updates Challenging software deployment
15.10.2013
![Page 12: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/12.jpg)
Testing as a Service with HammerCloud 12IT-SDC
The Agile Infrastructure
Private IaaS cloud OpenStack based Federates Meyrin and Wigner 15,000 hypervisors by 2015 300,000 VMs by 2015 Configuration management tools
Puppet, Foreman
15.10.2013
![Page 13: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/13.jpg)
Testing as a Service with HammerCloud 13IT-SDC
Requirements
1. Elastic infrastructure (OpenStack)2. Cloud orchestrator3. Code sanitation (Gerrit)4. Configuration Management (Puppet)5. Deployment procedures
15.10.2013
![Page 14: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/14.jpg)
Testing as a Service with HammerCloud 14IT-SDC
Dynamic testing scheduling
1. Test is requested (user, API, cron)2. Creation of the VMs on demand
Isolation Reliability Elasticity
3. Configuration and startup of the VM4. …test runs…5. Cleanup and destroy
15.10.2013
![Page 15: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/15.jpg)
Testing as a Service with HammerCloud 15IT-SDC 15.10.2013
![Page 16: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/16.jpg)
Testing as a Service with HammerCloud 16IT-SDC
Dynamic testing scheduling
Scheduling on time is hard IaaS predictable?
Provisioning times Boot times
HC will learn how is performing
Tool to estimate boot performance
15.10.2013
![Page 17: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/17.jpg)
Testing as a Service with HammerCloud 17IT-SDC
Provisioning latency
Big improvement on OpenStack Grizzly
Dynamic adaption of this time at runtime
15.10.2013
![Page 18: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/18.jpg)
Testing as a Service with HammerCloud 18IT-SDC
Shared storage
Needed a way to store all the logs Ideally accessible to others NFS for now,
HDFS, elasticsearch from the Agile Monitoring?
15.10.2013
![Page 19: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/19.jpg)
Testing as a Service with HammerCloud 19IT-SDC
Testing API
Tools will need an API to request tests And get information.
Already done with LHCb RSS Athena Nightlies API ongoing
15.10.2013
![Page 20: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/20.jpg)
Testing as a Service with HammerCloud 20IT-SDC
Current status
Orchestration framework developed Currently deploying on OpenStack
OpenStack is now production ready Thinking about configuration
frameworks
15.10.2013
![Page 21: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/21.jpg)
Testing as a Service with HammerCloud 21IT-SDC
Conclusions
New kind of testing requirements Big infrastructure changes Shift to a new cloudy paradigm
Needs a transition period
15.10.2013
![Page 22: Testing as a Service with HammerCloud](https://reader036.fdocuments.us/reader036/viewer/2022081604/568161b3550346895dd17d04/html5/thumbnails/22.jpg)
Testing as a Service with HammerCloud 22IT-SDC
Questions?
15.10.2013