COMP Superscalar: Bringing GRID superscalar and GCM together

of 25 /25
COMP Superscalar: Bringing GRID superscalar and GCM together Enric Tejedor Universitat Politècnica de Catalunya [email protected] V ProActive and GCM User Group, GridCOMP conference October 21, 2008. INRIA, Sophia Antipolis, France Rosa M. Badia Barcelona Supercomputing Center [email protected]

description

COMP Superscalar: Bringing GRID superscalar and GCM together. Enric Tejedor Universitat Politècnica de Catalunya [email protected]. Rosa M. Badia Barcelona Supercomputing Center [email protected]. V ProActive and GCM User Group, GridCOMP conference - PowerPoint PPT Presentation

Transcript of COMP Superscalar: Bringing GRID superscalar and GCM together

Page 1: COMP Superscalar: Bringing GRID superscalar and GCM together

COMP Superscalar:Bringing GRID superscalar

and GCM together

Enric TejedorUniversitat Politècnica de

[email protected]

V ProActive and GCM User Group, GridCOMP conferenceOctober 21, 2008. INRIA, Sophia Antipolis, France

Rosa M. BadiaBarcelona Supercomputing

[email protected]

Page 2: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Outline

• Introduction

• Grid Component Model

• GRID superscalar

• Programming model

• Runtime operation

• Initialisation

• Task processing

• Finalisation and error handling

• Conclusions

Page 3: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

1. Introduction

Page 4: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Basic concepts

•The Grid Component Model (GCM)

• CoreGRID NoE, GridCOMP

• Distributed, based on Fractal, intended for the Grid

•GRID superscalar

• Framework to ease the development of Grid-unaware applications

• Simple programming model: Grid as transparent as possible

• Runtime that optimises the performance of the application (exploiting

possible concurrency)

Page 5: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

A componentised Grid framework

• GRID superscalar is suitable to benefit from the GCM features

• Componentisation process: identify the inner functionalities of the runtime,

and assign each one to a separate component

• Result: COMP Superscalar, whose runtime gains in:

• Reusability

• Flexibility

• Deployability

• Separation of concerns

• Ease of development

Page 6: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Componentised runtime

Page 7: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

2. Programming model

Page 8: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

...for (i=0; i<N; i++){ T1 (data1, data2); T2 (data4, data5); T3 (data2, data5, data6); T4 (data7, data8); T5 (data6, data8, data9);}...

Sequential Application

T10 T20

T30

T40

T50

T11 T21

T31

T41

T51

T12

Resource 1

Resource 2

Resource 3

Resource N

.

.

.

Task graph creation

based on data

precedence

Task selection +

parameters direction

(input, output, inout)

Scheduling,

data transfer,

task execution

Synchronization,

results transfer

Parallel Resources(multicore,SMP, cluster, grid)

Programming model

Page 9: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

initialize(f1);

for (int i = 0; i < 2; i++) {

genRandom(f2);

add(f1, f2);

}

print(f2);

Java application

COMPSs programming model

public interface SumItf {

@ClassName(“example.Sum")

@MethodConstraints(OSType = "Linux")

void genRandom(

@ParamMetadata(type = Type.FILE, direction = Direction.OUT)

String f

);

@ClassName(“example.Sum")

...

}

Task constraints

Parametermetadata

Implementation

Java interface

Page 10: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Custom Java Class Loader

Java app code

COMPSs runtime

Annotated interface

Javassist

insertscalls to

Custom Loader

uses

input

Page 11: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

3. Runtime operation

Page 12: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Implementation details

•Base technologies:

• Java as programming language

• ProActive: implementation of the GCM model, used to build the components

• JavaGAT: used for job submission and file transfer

Page 13: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Initialization

•Multicast invocation

• Forwarded to all subcomponents

TA TS JM

FM

FIP FTM

init()

Page 14: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Initialization

•Multicast invocation

• Reduction of return values

TA TS JM

FM

FIP FTMReduction

Reduction

Page 15: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (1)

•Application submits task

• Task Analyser receives the request

TA TS JM

FM

FIP FTM

executeTask(…)

T1genRd

T3genRd

T2add

T4add

f1 f1

f2

Page 16: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (2)

TA JM

FM

FIP FTM

TS

•File Information Provider is contacted

• File accesses of the task are registered

executeTask(…)

Page 17: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (3)

•Task Analyser discovers dependencies

• Dependency-free tasks are sent for scheduling

TA TS JM

FM

FIP FTM

executeTask(…)

Page 18: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (4)

•Task Scheduler decides where to execute tasks

• Job Manager is informed of this decision

TA TS JM

FM

FIP FTM

executeTask(…) Scheduling

algorithm

Resourcecapabilitie

s

Taskconstraint

s

Page 19: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (5)

• Job Manager checks the necessary file transfers

• File Transfer Manager actually performs them

TA TS JM

FM

FIP FTM

executeTask(…)

GAT

Page 20: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (6)

•FTM informs of the end of transfers for a task

• Task Grid job, submitted for execution

TA TS JM

FM

FIP FTM

executeTask(…)

GAT

Page 21: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Task processing (7)

•Callback for the end of a job is received

• The notification is forwarded to the Task Analyser

TA TS JM

FM

FIP FTM

executeTask(…)

GAT

Page 22: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Safe stopping process for the runtime

•COMPSs subcomponents have data dependencies

•Modification of the Life Cycle Controller

• Components stopped in a safe order: TS, TA, JM, FTM, FIP

• Otherwise, a deadlock could happen

TATA TS JM

FM

FIP FTM

LCC

Page 23: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

4. Conclusions

Page 24: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Conclusions

•COMP Superscalar: componentised version of GRID superscalar

• GCM-based

• Hierarchical runtime: separation of concerns

• Task scheduling, file and resource management

•Straightforward programming model for Grid-unaware Java

applications

•Feedback about GCM-ProActive

• Component distribution

• Performance: no noticeable overhead for a medium number of tasks

• Component composition and dynamic reconfiguration: Grid IDE

Page 25: COMP Superscalar: Bringing GRID superscalar and GCM together

V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)

Thank you!

Questions?