Seminario di inizio terzo anno -...
Transcript of Seminario di inizio terzo anno -...
![Page 1: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/1.jpg)
Seminario di inizio terzo annoDicembre 7, 2016
Jacopo Pellegrino
![Page 2: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/2.jpg)
Cloud
● What “cloud” is● My work on cloud
computing
ABM & MAS
● What “ABM & MAS” are● The idea CIs as agents
ABCM
● Model design● Tools for implementation:
○ GAMA○ JADE○ GitLab○ PostgreSQL
Summary
● Current status● Plans for the last year
2
![Page 3: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/3.jpg)
Cloud Computing
● A brief introduction
● Cloud for HEP
● Cloud Toy
● Cloud Bursting
3
![Page 4: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/4.jpg)
Cloud ComputingThe cloud computing a widely adopted paradigm in the context of HEP.
CIs provide access to a shared pool of computational, network and storage resources.
Resources are easily available to users.
Setting up and maintaining a cloud infrastructure may not be trivial by the providers’ point of view.
4
![Page 5: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/5.jpg)
Cloud Computing
The main idea behind this work is to address two common issues of cloud infrastructures:
● Usability: simplifying the setup and installation process.
● Efficiency: making the usage of resources more dynamic, flexible and efficient.
The aim is to improve existing cloud infrastructures and give sites with limited manpower/knowledge easier access to cloud technologies.
5
![Page 6: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/6.jpg)
Cloud Computing
Automatization of Installation process:
● Creation of kickstart file
● Preparation of customized ISO
● Preparation of bootable usb drive
● Installation on server
● Test of the installation
6
![Page 7: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/7.jpg)
Cloud Computing
● Creation of kickstart file: configuration parameters, software packages, disks partitioning, and the like...
● Preparation of customized ISO: start from a standard netinstall iso file, make it look for the kickstart at boot time
● Preparation of bootable usb drive: burn the iso into a usb drive so that it is possible to boot from it
● Installation on server: plug the usb drive and reboot
● Test of the installation: the server is ready in about 1 hour7
![Page 8: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/8.jpg)
Cloud Computing
Efficiency of the infrastructure:
A CI is used by different stakeholders.
The aim is to dynamically change the quota of resources.
A “private” CI is introduced.
T1
T2
T4
T3
T2 has 100 coresT4 has 200 coresTogether they have 300
The aim is to manage these 300 cores dynamically for T2 and T4
T3
T2
T4
T2
T4
T1
8
![Page 9: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/9.jpg)
Cloud Computing
The “public” OpenNebula only knows that T2 and T4 have 300 cores (user-level control).
The “private” OpenNebula can dynamically change quotas between T2 and T4 according to actual needs (full sys-man control).
Variations of quotas can be performed via oneadmin user.
9
![Page 10: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/10.jpg)
Cloud Computing
The ONE-cloudbursting-driver is currently under test.
It enables OpenNebula-based cloud to "burst" Virtual Machines (VM) to external OpenNebula clouds using built-in OpenNebula XML-RPC and OCCI interfaces.
Aim: start VMs on Public OpenNebula from the Private OpenNebula.
10
![Page 11: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/11.jpg)
PRIVATE ON PUBLIC ON
ONEcloudburstingdriver
remoteVMDIRAC
rOCCI server
OCCI client
rOCCI server
Instantiate new VM
Instantiate new VM
changequotas
Cloud Computing
11
![Page 12: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/12.jpg)
Cloud Computing
This work has been presented at:
● SIF2016: New approaches for an (almost) automatic set-up of elastic cloud infrastructures; Pellegrino J., Amoroso A., Astorino F., Bagnasco S., Bianchi F., Destefanis M., Maggiora M., Yan L.; SIF2016, 26-30/09/2016; Padova, Italia
● CHEP2016: A modular (almost) automatic set-up for elastic multi-tenants cloud (micro)infrastructures; F. Astorino, A. Amoroso, S. Bagnasco, N. A. Balashov, F. Bianchi, M. Destefanis, M. Maggiora, J. Pellegrino, L. Yan, T. Yan, X. Zhang, X. Zhao; CHEP2016, 10-14/10/2016; San Francisco, USA
12
![Page 13: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/13.jpg)
ABM & MAS● A brief introduction
● An Agent Based Cloud Model
13
![Page 14: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/14.jpg)
● Intelligent Agent: a computer systems capable of interaction with the environment and autonomous actions to reach design goals.
● Agent-Based Models and Multi-Agent Systems:
○ systems made up of agents that interact with each other and the environment they live within to reach personal or global goals
○ study the evolution of distributed and dynamic complex systems in many fields
○ approach taken from nature (hive, ants...)
ABM & MAS
14
![Page 15: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/15.jpg)
ABM & MAS
In ABM the focus is on the internal architecture of the agent.
In MAS the focus in on the interaction between simple agents.
The main idea of these approaches is that agents have simple rules and limited knowledge but a global intelligence emerges from the interaction.
example: segregation model15
![Page 16: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/16.jpg)
ABM & MAS
16
![Page 17: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/17.jpg)
ABM & MAS
17
![Page 18: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/18.jpg)
ABM & MAS
Cloud computing is widely adopted in the context of HEP.
● many sites (globally distributed) have a C.I.● sites host several experiments● groups from different sites work on the same experiment● there may be federations of C.I.
Such a scenario can be viewed as a dynamic and distributed complex system.
18
![Page 19: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/19.jpg)
ABM & MAS
● Design and implement a MAS to model a community of cloud infrastructures devoted to HEP computing.
● Agents are related by means of a trust matrix which is updated after every interaction.
● The trust between agents regulates:
○ the propagation of information among the network
○ the possibility to share resources
19
![Page 20: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/20.jpg)
Agent-Based Cloud Model
● Model Design
● Tools adopted
○ GAMA○ JADE○ GitLab○ PostgreSQL
20
![Page 21: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/21.jpg)
ABCM
In the current design of the model agents have “scheduled actions” to perform (sending a job request).
These actions trigger the interaction with other agents (creating a job, sending a cfp, ...).
The outcome of these actions causes variations of the trust matrix.
The interaction triggered by a job request is summarized by the following diagram
21
![Page 22: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/22.jpg)
ABCM
22
![Page 23: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/23.jpg)
ABCM
23part 1 part 2
![Page 24: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/24.jpg)
ABCM
24part 3 part 4
![Page 25: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/25.jpg)
ABCM
GAMA 1.6 has been the first modeling tool adopted.
“GAMA is a modeling and simulation development environment for building spatially explicit agent-based simulations.” *
GAML language very close to JAVA with some limits.
The platform presented bugs during the simulations.
These bugs may have been fixed in version 1.7.
25*http://gama-platform.org
![Page 26: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/26.jpg)
ABCMJADE 4.4 has been adopted is place of GAMA.
“...is a software Framework fully implemented in the Java language. It simplifies the implementation of multi-agent systems through a
middle-ware that complies with the FIPA specifications...” *
● PROs: ○ Java language, much more freedom○ Industrial Tool○ Agents in different machines
● CONs:○ Lack of GUI
26*http://jade.tilab.com
![Page 27: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/27.jpg)
ABCMDue to the complexity of the model, a versioning framework was needed to keep track of changes.
For this task GitLab has been adopted.
27
![Page 28: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/28.jpg)
In order to solve the problem of lacking GUI, the data produced by agents have been collected by a global agent and written into a PostgreSQL database.
Initially an HTML5 GUI has been made which read from the database by means of a jsp page.
ABCM
global
jsphtml5
javascript bootsrap
.json
28
![Page 29: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/29.jpg)
ABCM
The HTML5 GUI required many steps to run and was not easy to set up.
GAMA can interact with databases and provided a very easy-to-use GUI.
From these considerations the idea to take advantage of the built-in GUI of GAMA 1.7:
global
JADE GAMA
globaldisplay
29
![Page 30: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/30.jpg)
global
JADE
GAMA
globaldisplayABCM
30
![Page 31: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/31.jpg)
Summary● Current status
● Plans for the last year
● Talks and publications
31
![Page 32: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/32.jpg)
Current status
● Cloud○ the tool for automatic installation is ready○ the cloud bursting tool is currently under test
● ABCM○ the model is partially completed○ documentation is up to date
● Thesis○ first chapters written
32
![Page 33: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/33.jpg)
Plans for the last year
33
● Cloud○ set up the cloud bursting tool○ test the infrastructure
● ABCM○ proceed with the model implementation○ simulate complex distributed computing real cases (HEP
experiments, e.g. BESIII and BELLEII) with nation-wide stakeholders (e.g. INFN)
○ perform analysis of the results● Thesis
○ add and complete the remaining parts
![Page 34: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/34.jpg)
Thank you for the attention
34
![Page 35: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/35.jpg)
Talks and publications
35
● ICME2015: A Multi Agent Approach for Autonomous Digital Preservation; Pellegrino J., Allasia W., Maggiora M.; ICME2015 conference proceedings, workshop HMMP, 03/07/2015; Torino, Italia
Link: http://ieeexplore.ieee.org/document/7169866/authors
● SIF2016: New approaches for an (almost) automatic set-up of elastic cloud infrastructures; Pellegrino J., Amoroso A., Astorino F., Bagnasco S., Bianchi F., Destefanis M., Maggiora M., Yan L.; SIF2016, 26-30/09/2016; Padova, Italia
![Page 36: Seminario di inizio terzo anno - unito.itdottorato.ph.unito.it/Studenti/Pretesi/XXX/pellegrino.pdf · The HTML5 GUI required many steps to run and was not easy to set up. GAMA can](https://reader034.fdocuments.us/reader034/viewer/2022050220/5f6603053d98690dc2609133/html5/thumbnails/36.jpg)
Question Time
36