Grid and Cloud Infrastructures for Agriculture Antun Balaz Institute of Physics Belgrade, Serbia
description
Transcript of Grid and Cloud Infrastructures for Agriculture Antun Balaz Institute of Physics Belgrade, Serbia
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Grid and Cloud Infrastructures for Agriculture
Antun BalazInstitute of Physics Belgrade, Serbia
Joint agINFRA and SCI-BUS workshop &agINFRA Open Stakeholder Day
30 May 2013, SZTAKI, Budapest, Hungary
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Overview
• agINFRA infrastructure• Hardware resources• Grid resources• Scientific Gateways• agINFRA RESTful gateway• Key results
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
agINFRA Infrastructure
• Technically divided into three main layers:– Hardware resources (Grid and Cloud)– Scientific Gateways (Catania SG, WS-PGRADE SG, RESTful gateway)– Integrates services (Organic.Edunet, AGRIS, CIARD RING, AgLR-TF)
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Hardware resources• Resource centers
– PARADOX cluster – 88 x (2 x 4-core Xeon [email protected] GHz) = 704 CPU (1 GB of RAM / core)50 TB of disk space, Gigabit Ethernet
– Roma Tre cluster – 350 CPUs; 100TB– Catania cluster – 800 CPUs; 700 TB – SZTAKI cluster – 8 CPUs– PARADOX ugrade – 106 x (2 x 8-core Xeon [email protected] GHz)
= 1696 CPU (2 GB of RAM / core)100 TB of disk space, Infiniband
• Virtualized resources– SZTAKI Desktop Grid
90 000 volunteer resources; 3 Tflops.– EDGeS@home - 15 000 CPUs
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Grid resources [1/2]
• agINFRA Virtual Organization: vo.aginfra.euhttps://voms.ipb.ac.rs:8443/voms/vo.aginfra.eu https://voms.ct.infn.it:8443/voms/vo.aginfra.eu
• MyProxy service (PX) – myproxy.ipb.ac.rs
• Computing Elements (CREAM)– cream.ipb.ac.rs:8443/cream-pbs-aginfra – 704 CPUs– ce-01.roma3.infn.it:8443/cream-pbs-grid – 328 CPUs– ce-02.roma3.infn.it:8443/cream-pbs-grid – 208 CPUs– grid012.ct.infn.it:8443/cream-lsf-aginfra – 20 CPUs
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Grid resources [2/2]• Logical File Catalogue (LFC)
– lfc.ipb.ac.rs• Storage Elements (SE) – 850 TB
– dpm.ipb.ac.rs– prod-se-02.ct.infn.it– storm-01.roma3.infn.it
• Metadata Catalogue (AMGA)– amga.ct.infn.it
• Workload Management System (WMS)– wms.ipb.ac.rs– wms-demo.ipb.ac.rs– wms-aegis.ipb.ac.rs
• Logging and Bookkeeping (LB)– wms.ipb.ac.rs– wms-demo.ipb.ac.rs– wms-aegis.ipb.ac.rs
• Information System (BDII)– bdii.ipb.ac.rs
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Scientific Gateways
• Catania Scientific Gatewayhttps://aginfra-sg.ct.infn.it/
• gUSE/WS-PGRADE Portal (workflows)http://aginfra-portal.lpds.sztaki.hu/liferay-portal-6.0.5/
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
agINFRA RESTful gateway• Cataloging
– User’s configurations of Grid-ported applications– Datasets’ metadata information and locations– Arbitrary user-defined metadata information– Querying of these information
• Off-line processing– Collect new datasets – Transform existing datasets– Produce new datasets’ metadata information
• Data management– Automatic replication of datasets– Ensures existence of the same dataset on different storage systems– Exposure through the HTTP protocol
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Key results• Design, specification and deployment of the agINFRA infrastructure• Establishment and operation of agINFRA Grid infrastructure (VO:
vo.aginfra.eu, VOMS, BDII, LFC, WMS/LB, PX, AMGA, DPMs, CREAMs)• Establishment and deployment of Cloud infrastructure (CLEVER and
Okeanos)• Deployment of agINFRA Scientific Gateways (Catania and gUSE/WS-
PGRADE SGs)• Development of agINFRA RESTful gateway• Development and deployment of data infrastructure
– dataset exposure layer (RESTful gateway)– data management layer (Grid RESTful API, HTTP exposure)– data transformation layer (agDCtoLOM, agLOMtoAK, agLOMtoRDF)– data processing layer (agDataHarvest, agMetaExtract,
agRecommender)