IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

LHC@home and CernVM – a new approach toporting large-scale applications to BOINC

Daniel Lombrana Gonzalez, Ben Segaland Artem Harutyunyan

Citizen Cyberscience CentreCERN

March 20, 2011

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Outline

1 Introduction

2 Technical challenges

3 Employed solutions

4 CernVM + Co-Pilot

5 Test4theory project

6 Conclusions

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

The beginning

The challenge

Why don’t you run real LHC physics on BOINC?

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

The challenge

Allow “any” PC to run a full LHC physics application.Make those commodity computers look like a “standard”CERN Data Center.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

We did it!

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Outline

1 Introduction

2 Technical challenges

3 Employed solutions

4 CernVM + Co-Pilot

5 Test4theory project

6 Conclusions

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Challenge 1: Porting source code

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Challenge 1: Porting source code

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Challenge 1: Porting source code

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Challenge 2: Job management systems

Jobs must be fed into BOINC PCs, but CERN physicsexperiments have their own and don’t want to use theBOINC distributions system.Volunteer computing resources are not “managed” andthus cannot be “trusted”.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Outline

1 Introduction

2 Technical challenges

3 Employed solutions

4 CernVM + Co-Pilot

5 Test4theory project

6 Conclusions

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

CernVM + Co-Pilot

Using Virtualization and CernVM itis possible to solve the challenges(1) for application deployment onvolunteers’ machines.With CernVM’s Co-Pilot it is possibleto connect existing Gridinfrastructures of LHC experimentswith the BOINC volunteer resources,solving challenges (2).

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

CernVM solution

CernVM isa baseline Virtual Software Appliance for the participants ofCERN LHC experiments.

The goal isto remove a need for the installation of the experiment softwareand to minimize the number of platforms.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

CernVM solution

CernVM isa baseline Virtual Software Appliance for the participants ofCERN LHC experiments.

The goal isto remove a need for the installation of the experiment softwareand to minimize the number of platforms.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

CernVM Co-Pilot solution

A framework for the delivery and execution of the workloadon the remote virtual machines.Consists of components developed to ease the integrationof cloud resources into existing Grid infrastructures.Components communicate using Jabber/XMPP instantmessaging protocol.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Outline

1 Introduction

2 Technical challenges

3 Employed solutions

4 CernVM + Co-Pilot

5 Test4theory project

6 Conclusions

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

CernVM

Using virtualization and CernVM solve the problem ofporting the source code and deploying over PCs.CernVM Co-Pilot can connect existing Grid infrastructureof LHC experiments with BOINC resources.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

BOINC + CernVM infrastructure

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Co-Pilot Adapters

Each LHC experiment that wants to connect to a set ofCernVM machines via Co-Pilot, needs an adapter.Each adapter fetches jobs from the experiment’s preferredjob scheduler, submits them to the CernVM machines, andreturns the results to the scheduler.Co-Pilot has a built-in security barrier, preventing untrustedCernVM machines to access Grid resources.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Outline

1 Introduction

2 Technical challenges

3 Employed solutions

4 CernVM + Co-Pilot

5 Test4theory project

6 Conclusions

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Monte Carlo events generation

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Alpha testing

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Active forums

More than 500 posts in theforums so far.Very active volunteers,reporting all the problems theyfind.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

High schools

Three young students were invited to test the project.One of them became very helpful in the first stages of thetesting phase.The obtained feedback allowed us to improve the project,knowing that young students could collaborate withcomplex projects.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Numbers

Since the beginning of the project 25000 jobs have beensuccessfully executed.80% of the jobs were set to generate 300000 events (therest were smaller ones that generated 50000 events).Totaling 6250 millions of events simulated since December2010.There are on average 20 on-line users at any given point intime (peaks up to 30 users).We are running 1200-1300 jobs daily.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Outline

1 Introduction

2 Technical challenges

3 Employed solutions

4 CernVM + Co-Pilot

5 Test4theory project

6 Conclusions

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Conclusions

A custom virtualized execution environment has beenadded to BOINC.Very complex and unmodified applications can be runwithin the system.Adapting Grid services to Co-Pilot is now understood.

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Conclusions

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

IntroductionTechnical challenges

Employed solutionsCernVM + Co-PilotTest4theory project

Conclusions

Questions

teleyinex@gmail.comIcons from Tango and Gnome Desktop projects (Creative Commons & GPL License)

D. Lombrana, B. Segal & A. Harutyunyan LHC@Home