8 th Lecture COP 4991 Component-Based Software Development Instructor: Masoud Sadjadi...

8th Lecture

COP 4991 COP 4991 Component-Based Software DevelopmentComponent-Based Software Development

Instructor: Masoud Sadjadihttp://www.cs.fiu.edu/~sadjadi/Teaching/

Grid Computing and LA GridGrid Computing and LA Grid

8th Lecture 2COP-4991: Component-Based Software Development

AcknowledgementAcknowledgement

Andrew Tanenbaum Laukik Chitnis Sanjay Ranka Onyeka Ezenwoye Jorge Rodriguez


AgendaAgenda

Motivation

Grid Computing Overview

Grid Middleware

LA Grid


Speed up Using Parallel Speed up Using Parallel ProcessingProcessing

(a) A program has a sequential part and a parallelizable part.

(b) Effect of running part of the program in parallel.


Speed up Using Parallel Speed up Using Parallel ProcessingProcessing

Real programs achieve less than the perfect speedup indicated by the dotted line.


Parallel Computer Parallel Computer ArchitecturesArchitectures

(a) On-chip parallelism. (b) A coprocessor. (c) A multiprocessor.

(d) A multicomputer. (e) A grid.


Instruction-Level ParallelismInstruction-Level Parallelism

(a) A CPU pipeline. (b) A sequence of VLIW instructions. (c) An instruction stream with bundles marked.


MultiprocessorsMultiprocessors

(a) A multiprocessor with 16 CPUs sharing a common memory.

(b) An image partitioned into 16 sections, each being analyzed by a different CPU.


MulticomputersMulticomputers

(a) A multicomputer with 16 CPUs, each with its own private memory.

(b) The bit-map image of Fig. 8-17 split up among the 16 memories.


Taxonomy of Parallel Taxonomy of Parallel Computers (1)Computers (1)

Flynn’s taxonomy of parallel computers.


Taxonomy of Parallel Taxonomy of Parallel Computers (2)Computers (2)

A taxonomy of parallel computers.


UMA Multiprocessors Using Crossbar UMA Multiprocessors Using Crossbar SwitchesSwitches

(a) An 8 × 8 crossbar switch. (b) An open crosspoint. (c) A closed crosspoint.


NUMA MultiprocessorsNUMA Multiprocessors

A NUMA machine based on two levels of buses. The Cm* was

the first multiprocessor to use this design.


BlueGene (1)BlueGene (1)

The BlueGene/L custom processor chip.


BlueGene (2)BlueGene (2)

The BlueGene/L. (a) Chip. (b) Card. (c) Board. (d) Cabinet. (e) System.


Google (1)Google (1)

Processing of a Google query.


Google (2)Google (2)

A typical Google

cluster.


AgendaAgenda

Motivation


Grid Middleware

LA Grid


Grid ComputingGrid Computing

Grid computing is an emerging computing model that – tries to solve large-scale computation problems – by taking advantage of many networked computers – to model a virtual computer architecture – that is able to distribute process execution – across a parallel infrastructure.

Source: www.wikipedia.org


Ian Foster and Carl KesselmanIan Foster and Carl Kesselman

“A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities.” 1998

“The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources, as is required by a range of collaborative problem solving and resource-brokering strategies emerging in industry, science, and engineering. This sharing is, necessarily, highly controlled, with resource providers and consumers defining clearly and carefully just what is shared, who is allowed to share, and the conditions under which sharing occurs. A set of individuals and/or institutions defined by such sharing rules form what we call a virtual organization.” 2000


A Grid Checklist

coordinates resources that are not subject to centralized control …

… using standard, open, general-purpose protocols and interfaces …

… to deliver nontrivial qualities of service.

Virtual Organizations– Group of individuals or institutions defined by sharing rules to share

the resources of “Grid” for a common goal.– Example: Application service providers, storage service providers,

databases, crisis management team, consultants.


The Grid isn’t a new conceptThe Grid isn’t a new concept

Using multiple distributed resources to cooperatively work on a single application has been around for decades– Networked OS (70s)– Distributed OS (80s and 90s)– Heterogeneous Computing http://www.acis.ufl.edu/hcw2006/– Parallel Distributed Computing– Metacomputing http://www.cnds.jhu.edu/research/metacomputing/


How is a grid different?How is a grid different?

Grids focus on site autonomy

Grids involve heterogeneity

Grids involve more resources than just computers and networks

Grids focus on the user


Grid ComputingGrid Computing

A dynamic multi-institutional network of computers that come together to share resources for the purpose of coordinated problem solving.

resource

application

institutional boundary

Achieved through:

1. Open general-purpose protocols

2. Standard interfaces


So, what is a Grid again?!So, what is a Grid again?!

Simply put, a Grid is a congregation of different sites collaborating to increase productivity

For example, Grid3

A total of 35 sites

Over 3500 CPUsA total of 35 sites

Over 3500 CPUs


Broad Division of GridBroad Division of Grid

Data Grids– Managing and

manipulating large amounts of data.

– Main objective is to share large amounts of data that is otherwise impossible with out the grid

Compute Grids – For compute-intensive

tasks. – Emphasis is on the

federation of CPU cycles and distribution of compute intensive tasks

There is no consensus on these categorization and it only aids in understanding the requirements


Examples of Distributed Apps.Examples of Distributed Apps.

High Energy Physics applications – Monte Carlo simulations– CMS experiment

Finding interesting astronomical patterns– Sloan Digital Sky Survey

Coastal ocean monitoring and predicting– SURA Coastal Ocean Observing and Prediction (SCOOP)

Prime number generator– Cracking DES

Cannot be done on a single machine You want to divide the application and run it on a

distributed and decentralized environment


Why do you want a grid?Why do you want a grid?

Different perspectives– User: I want to run my scientific application on the grid so that I can

get results in 10 hours instead of 10 days

– Organization: Our next big experiment will generate tera-bytes of data and we want to distribute, share and analyze the data

– Organization: We want to tap into the existing grids and share resources


A User PerspectiveA User Perspective

I need– More CPU cycles– More disk space– More bandwidth– Better software tools– All of the above

Alternatives to grid– Simple CPU cycle stealer– Simple SRM (Storage Resource

Manager)

Run my app in 10 hrs that usually takes 10 days on my pentium


Sys admin perspectiveSys admin perspective

How do I distribute the load on the machines? How do I reduce the overhead on the central server How do I manage local and remote users? What should be the policies?

I got root !


Organizational PerspectiveOrganizational Perspective

Federation of scientists – distributing, sharing and analyzing data

Tapping into existing grids Cost-effective: A grid can be built from commodity software

and hardware without spending millions on the next super duper computer.

Reliability: If a site fails, we can simply move our jobs to another site (this can be seen as a user perspective as well)

Where do you want to run your job today?


Distributed App. Distributed App. RequirementsRequirements

Requires– A lot of resources– Reservation of resources at a particular time– Monitoring of status of the submitted jobs to multiple sites– Storage that is not easily available at a single place


Grid Building BlocksGrid Building Blocks

Computational Clusters Storage Devices Networks Grid Resources and Layout:

– User Interfaces– Computing Elements– Storage Elements– Monitoring Infrastructure…

Some slides from Jorge Rodriguez’s presentation on “Building, Monitoring and Maintaining a Grid”


Dell Cluster at UFlorida’s High Performance Center

Computer ClustersComputer Clusters

Cluster Management“frontend”

Tape Backup robots

I/O Servers typically RAID fileserver

Disk Arrays

The bulk are Worker Nodes

A few Headnodes, gatekeepers and

other service nodes


A Typical Cluster InstallationA Typical Cluster Installation

Network Switch

Pentium III

Pentium III

Pentium III

Head Node/Frontend Server

Pentium III

Worker Nodes

WANWAN

Cluster Management• OS Deployment• Configuration• Many options

ROCKS (kickstart)OSCAR (sys imager)Sysconfig

• •

Computing Cycles Data Storage Connectivity

I/O Node + Storage


Layout of Typical Grid SiteLayout of Typical Grid Site

Computing Fabric

Grid MiddlewareGrid Level Services

++

=>

A Grid Site

=>VDTVDT

OSGOSG

ComputeElement

StorageElement

User Interface

Authz server

Monitoring Element

Monitoring Clients Services

Data Management

Services

Grid Operations

The Gr id


~1200 node workflow, 7 levelsMosaic of M42 created onthe Teragrid using Pegasus

A Small Montage WorkflowA Small Montage Workflow

A Typical Grid Application A Typical Grid Application Workflow Workflow


Simple Expectations from the Simple Expectations from the GridGrid

Simply put, we need the grid to do the following operations for each transformation:– Find the input datasets– Apply the transformations (process the input)– Store the output datasets

and publish its “presence” so that collaborating scientists can find it


Qualities expected from the Qualities expected from the gridgrid

And of course, we would like these operations to be performed:– Efficiently

As quickly as possible

– Seamlessly for easy collaboration

– Fairly fair to all collaborators

– Securely security against loss (fault tolerance), unauthorized access


AgendaAgenda

Motivation


Grid Middleware

LA Grid


Grid MiddlewareGrid Middleware

In an effort to view the ‘Grid as a Workstation’, a set of grid software and services act as middleware between the user and the grid of machines.

These services can be roughly categorized as follows:– Security services– Information Services– Data Management– Job Management– Virtual Data System


GLOBUS TOOLKIT 4 – GT4GLOBUS TOOLKIT 4 – GT4

Open source toolkit developed by The Globus Alliance that allows us to build Grid applications.

Organized as a collection of loosely coupled components. Consists of services, programming libraries, and

development tools.

High-level services – Resource Monitoring and Discovery Service– Job Submission Infrastructure– Security Infrastructure– Data Management Services


GT4 ComponentsGT4 Components


Services offered in a GridServices offered in a Grid

Resource Managemen

t Services

Data Management

Services

Information Services

Security Services


Security ServicesSecurity Services

Forms the underlying communication medium for all the services

Secure Authentication and Authorization Single Sign-on

– User need not explicitly authenticate himself every time a service is requested

Uniform Credentials Ex: GSI (Globus Security Infrastructure)


User Proxy

Creates a proxy for single-sign on

A

Plain unix authentication

GSI enabled GRAM

B

Kereberos authentication

GSI enabled GRAM


GSI certificateGSI certificate

A GSI certificate includes four primary pieces of information:– A subject name, which identifies the person or object that the

certificate represents. – The public key belonging to the subject. – The identity of a Certificate Authority (CA) that has signed the

certificate to certify that the public key and the identity both belong to the subject.

– The digital signature of the named CA.

A third party (a CA) is used to certify the link between the public key and the subject in the certificate. In order to trust the certificate and its contents, the CA's certificate must be trusted. The link between the CA and its certificate must be established via some non-cryptographic means, or else the system is not trustworthy.


Grid ProxyGrid Proxy

Once your user certificate is in place, you need to create a grid proxy which is used for accessing the Grid

In Globus, you can do this using – grid-proxy-init

A proxy is like a temporary ticket to use the Grid, default in the above case being 12 hours.

Once this is done, you should be able to run “grid jobs”– globus-job-run site-name /bin/hostname


Gridmap fileGridmap file

A gridmap file at each site maps the grid id of a user to a local id– The grid id of the user is his/her subject in the grid user certificate– The local id is site-specific; – multiple grid ids can be mapped to a single local id

Usually a local id exists for each VO participating in that grid effort

– The local ids are then used to implement site specific policies Priorities etc.


GridmapGridmap file entry file entry The gridmap-file is maintained by the site administrator Each entry maps a Grid DN (distinguished name of the user; subject name) to

local user names

##Distinguished Name Local username#

“/DC=org/DC=doegrids/OU=People/CN=Laukik Chitnis 712960” ivdgl“/DC=org/DC=doegrids/OU=People/CN=Richard Cavanaugh 710220” grid3“/DC=org/DC=doegrids/OU=People/CN=JangUk In 712961” ivdgl“/DC=org/DC=doegrids/OU=People/CN=Jorge Rodriguez 690211” osg


Grid MonitoringGrid Monitoring

A grid is complex and quite big– To efficiently use it, monitoring of the resources is needed

Why?– To check the availability of different grid sites– Discovery of different grid services– To check the status of “jobs”– Maintain information on “health” of the different sites for informed

decision making scheduling


Type of InformationType of Information

The information can be broadly categorized as– Static, and– Dynamic

Static information about a site includes– Its resource tallies

Number of worker nodes, processors Storage capacities Architecture and Operating systems

– Path of default staging area for grid users $app, $data


Dynamic InformationDynamic Information

The dynamic content includes– The number of jobs running on each site– The CPU utilization of different worker nodes– Overall site “availability”

Such time-varying information is more critical for efficient scheduling of grid jobs

Monitoring such information presents the question of trade-off between accuracy (pertinence) and intrusion by the monitoring activities in the processor time share.


Data ManagementData Management

Grids are usually used for analyzing and manipulating large amounts of data

Data management services provide a flexible mechanism to move and share data

Ex: GridFTP, RLS (Replica Location Service)


Data ManagementData Management

Motivating factors– Large number of data files

Distributed across different sites

– Big files! ~TB, PB??

Data is a first-class citizen in the Grid!


GridFTPGridFTP

Globus-url-copy GridFTP-compliant client from the Globus team Copy files from one URL to another URL

– One URL is usually a gsiftp:// URL– Another URL is usually a file:/ URL– To move a file from remote GridFTP-enabled server to local

machine


globus-url-copyglobus-url-copy

To put file onto server reverse URLsglobus-url-copy file:/home/lchitnis/file1

gsiftp://grinpc03.phys.ufl.edu/data/file1

By default 1 data channel used– monitor performance using –vb flag

$ globus-url-copy -vb gsiftp://grinpc03.phys.ufl.edu/ppadala/grid/file1 file:/tmp/file1 9437184 bytes 658.09 KB/sec avg 512.95

KB/sec inst


Performance tuningPerformance tuning Multiple channels dramatically boosts transfer rate$ globus-url-copy -vb -p 4 gsiftp://ldas-

cit.ligo.caltech.edu:15000/usr1/grid/largefile file:/tmp/largefile

523960320 bytes 5814.25 KB/sec avg 5568.27 KB/sec inst

Still faster by using large TCP windows$ globus-url-copy -vb -p 4 -tcp-bs 1048576 gsiftp://ldas-

cit.ligo.caltech.edu:15000/usr1/grid/largefile file:/tmp/largefile


Still faster by using large memory buffers$ globus-url-copy -vb -p 4 -bs 1048576 -tcp-bs 1048576

gsiftp://ldas-cit.ligo.caltech.edu:15000/usr1/grid/largefile file:/tmp/largefile


Examples in this chapter adapted from David Gehrig’s presentation on “Grid Data Management”


Job Management ServicesJob Management Services

Provide a uniform, standard interface to remote resources including CPU, Storage and Bandwidth

Main component is the remote job manager

Ex: GRAM (Globus Resource Allocation Manager)


Job Management on a GridJob Management on a Grid

User

The Grid

VO

VO

VO

VO

Condor

PBS

LSF

fork

GRAM

Narration: note the different local schedulers

Site A

Site B

Site C

Site D


Commands to run a jobCommands to run a job

globus-job-run– globus-job-run runs in the foreground and defaults to sending

output to your terminal. – In its basic form, it is roughly equivalent to rsh, but has

considerably more functionality for running complex jobs on the Grid.

globus-job-submit– globus-job-submit is for submitting jobs to a remote batch job

scheduler such as LSF or the Portable Batch System (PBS). – With globus-job-submit, you can submit a job, log out, and log back

in later to collect the output. That is, globus-job-submit runs in the background and defaults to sending output to the machine running the command.

– Retrieve output with globus-job-get-output, and then clean up with globus-job-clean.


Commands to run jobs on gridCommands to run jobs on grid

globusrun– The globusrun command runs specifications written in the Globus

Resource Specification Language (RSL).

– globusrun can run jobs either in the foreground or background, and can send output to your terminal or to the machine running the command.

– The trend in Globus software development is toward considering globusrun as middleware, which can be used by application specific shell scripts to manage job submission.

– In fact, globus-job-run and globus-job-submit are simply shell scripts that use globusrun for job submission, but present a simpler interface to users.


AgendaAgenda

Motivation


Grid Middleware

LA Grid


WHAT IS LA GRID?WHAT IS LA GRID?

“First-ever comprehensive computing grid to link faculty, students, and researchers from institutions across the United States, Latin America and Spain to collaborate on complex industry applications for business and societal needs. ”

http://lagrid.fiu.edu/http://lagrid.fiu.edu/


LA-GRID @ CIS.FIULA-GRID @ CIS.FIU

Globus-enabled hosts:– ip-comm.cs.fiu.edu– la-blade-01.cs.fiu.edu– la-blade-02.cs.fiu.edu– la-blade-03.cs.fiu.edu– la-blade-04.cs.fiu.edu– la-blade-05.cs.fiu.edu– la-blade-06.cs.fiu.edu– la-blade-08-ppc.cs.fiu.edu– la-blade-09-ppc.cs.fiu.edu– tsunami-linux.cs.fiu.edu– starscream.cs.fiu.edu


Resources as of July 2006 Resources as of July 2006

Resources as of July 2006


GETTING READY FOR LAGRIDGETTING READY FOR LAGRID

Globus Identity.Get a Globus Identity certificate issued by the Certificate Authority.http://www.cs.fiu.edu/~esj/globus.html

Security Enrollment.Enroll as a Secure Globus User with Eric Johnson.

Setup environment


How to use Globus@SCISHow to use Globus@SCIS The directory /etc/grid-security exists on all SCIS machines

that are grid enabled

The following should be added to your .cshrc file toensure that your environment is correctly set up for Globus.

if ( -d /etc/grid-security ) then setenv GLOBUS_LOCATION /depot/globus-4 setenv LD_LIBRARY_PATH $GLOBUS_LOCATION/lib:$LD_LIBRARY_PATH ins2path $GLOBUS_LOCATION/bin ins2path $GLOBUS_LOCATION/sbin setenv MANPATH $GLOBUS_LOCATION/man:$MANPATH if ( `uname -m` == "ppc64" ) then

setenv JAVA_HOME /opt/ibm/java2-ppc-50 else

setenv JAVA_HOME /depot/J2SE-1.5 endif

endif


Globus Authentication ModelGlobus Authentication Model

Users have a Globus Identity certificate which issigned by the local certificate authority.

Each system participating will have a mappingof globus ID to local user name.

Each system must recognize the certificate authoritythat signed a users certificate.

Here at SCIS we recognize certificate signed by us, and those signed by other LA-GRID sites.


How to create and use an How to create and use an Identity Identity (1)(1)

After setting up your environment as specified above, logout and then log in again and run the below command to generate a personal grid identity certificate.

grid-cert-request

This will create the following files in $HOME/.globus

usercert_request.pem (request to sign certificate)userkey.pem (private key - encrypted)usercert.pem (public key - signed)


How to create and use an How to create and use an Identity Identity (2)(2)

After you have created the request then you need to mail it to the localcertificate authority:

cat $HOME/.globus/usercert_request.pem | mail [email protected]

Then the CA (Mr. Eric Johnson) will mail you back a signed certificate which you will want to put into $HOME/.globus/usercert.pem

(it can take up to a day for the CA to process the request)


Commands to log in / logoutCommands to log in / logout grid-proxy-init

– This "logs you into" the globus system.

grid-proxy-info – Use this to see your status.

grid-proxy-destroy – Use this to log out.

How to test – When logged into a globus enabled hosts and logged into globus system:

globus-url-copy -vb -dbg gsiftp://OTHERGLOBUSHOST.cs.fiu.edu/dev/zero file:///dev/null

– REPLACE OTHERGLOBUSHOST with host name of another globus host. You should see no errors.


How to deploy jobsHow to deploy jobs

Here is a simple test run on the local machine. – -s option says to connect stdout and stderr to this local

process.

globusrun-ws -submit -s -c /bin/hostname

globusrun-ws -submit -S -f test1.rsl globusrun-ws -submit -S -F https://HOSTNAME:8443/wsrf/services/ManagedJobFactoryService -f test1.rsl

globusrun-ws -submit -S -f test2.rsl globusrun-ws -submit -S -F https://HOSTNAME:8443/wsrf/services/ManagedJobFactoryService -f test2.rsl


test1.rsltest1.rsl This copies over and executable (/bin/echo) and runs it with two arguments.

stdout and stderr are stored to files with those names.– This will only run on Intel arch machines.

<job><executable>my_echo</executable><directory>${GLOBUS_USER_HOME}</directory><argument>Hello</argument><argument>World!</argument><stdout>${GLOBUS_USER_HOME}/stdout</stdout><stderr>${GLOBUS_USER_HOME}/stderr</stderr><fileStageIn> <transfer> <sourceUrl>gsiftp://lion.cs.fiu.edu:2811/bin/echo</sourceUrl> <destinationUrl>file:///${GLOBUS_USER_HOME}/my_echo</destinationUrl> </transfer></fileStageIn><fileCleanUp> <deletion> <file>file:///${GLOBUS_USER_HOME}/my_echo</file> </deletion></fileCleanUp>

</job>


test2.rsltest2.rsl This copies over a binary executable and runs it.

– stdout and stderr are stored to files with those names.– This will only run on Intel arch machines.

<job><executable>/bin/hostname</executable><directory>${GLOBUS_USER_HOME}</directory><stdout>${GLOBUS_USER_HOME}/stdout</stdout><stderr>${GLOBUS_USER_HOME}/stderr</stderr><fileStageIn> <transfer> <sourceUrl>gsiftp://la-blade-01.cs.fiu.edu:2811/bin/echo</sourceUrl> <destinationUrl>file:///${GLOBUS_USER_HOME}/my_echo</destinationUrl> </transfer></fileStageIn><fileCleanUp> <deletion> <file>file:///${GLOBUS_USER_HOME}/my_echo</file> </deletion></fileCleanUp>

</job>


ReferencesReferences State and Events for Web Services: A Comparison of Five WS-Resource

Framework and WS-Notification Implementations. M. Humphrey, G. Wasson, K. Jackson, J. Boverhof, M. Rodriguez, Joe Bester, J. Gawor, S. Lang, I. Foster, S. Meder, S. Pickles, and M. McKeown, 4th IEEE International Symposium on High Performance Distributed Computing (HPDC-14), Research Triangle Park, NC, 24-27 July 2005.

C. Lee and D. Talia. "Grid programming models: Current tools, issues, and directions.", In Grid Computing: Making The Global Infrastructure a Reality, F. Berman, A. Hey, and G. Fox, editors, John Wiley & Sons, 2003.

“Enabling Peer-to-Peer Interactions for Scientific Applications on the Grid, ” V. Matossian* and M. Parashar, Proceedings of the 9th International Euro-Par Conference (Euro-Par 2003), Lecture Notes in Computer Science, Editors: H. Kosch, L. Boszormenyi, H. Hellwagner, Springer-Verlag, Klagenfurt, Austria, Vol. 2790.

Grids: Top Ten Questions. J.M. Schopf and B. Nitzberg, Scientific Programming, special issue on Grid Computing, 10(2):103 - 111, August 2002.

Berry, A. Djaoui, A. Grimshaw, B. Horn, F. Maciel, F. Siebenlist, R. Subramaniam, J. Treadwell, J. Von Reich. Informational Document, Global Grid Forum (GGF), January 29, 2005.

8 th Lecture COP 4991 Component-Based Software Development Instructor: Masoud Sadjadi...

Documents

Transcript of 8 th Lecture COP 4991 Component-Based Software Development Instructor: Masoud Sadjadi...