8/9/2019 Alchemi-678-2006

1/40

Alchemi: A .NET-basedEnterprise Grid Framework

Grid Computing and Distributed SystemsG!"#$% &ab.#ept. o' Computer $cience and $o'tware

EngineeringThe (ni)ersity o' *elbourne+ Australia

www.gridbus.org ,www.alchemi.net

8/9/2019 Alchemi-678-2006

2/40

Agenda

"ntroduction to Enterprise grids

Alchemi: )er)iew

/rogramming with Alchemi /er'ormance e)aluation: how does

Alchemi stack up against other similare0orts

/ro1ects based on Alchemi

$ummary

8/9/2019 Alchemi-678-2006

3/40

"ntroduction to Enterprise

grids

8/9/2019 Alchemi-678-2006

4/40

2

Enterprise,#esktop Grids

#esirable 3ualities 4arness the unused computing power o' their

e5isting desktop computers to build )irtual

supercomputers to satis'y growing compute-needs ma5imi6e resource utili6ation%

*inimi6e additional in)estment in h,w and s,win'rastructure

!educe impact on users in daily operations Tightly controlled semi-%centrali6ed

en)ironment

8/9/2019 Alchemi-678-2006

5/40

8/9/2019 Alchemi-678-2006

6/40

Enterprise Grid *iddleware:#esign Considerations

(sability Easy to use installers *inimal con;guration

Connecti)ity

8/9/2019 Alchemi-678-2006

7/40?

Enterprise Grid *iddleware:#esign Considerations contd@

!eliability Threads on disconnected workers should be restarted !e3uire smooth restart 'rom last known state in the

e)ent o' a crash

$calability #atabase 'or state = minimal state in memory *ultithreaded

$ecurity Authentication+ Authori6ation+ Auditing AAA% Code security: !un remote 1obs in a sandbo5 with ;ne-

grained permissions *anageability

G(" 'or monitoring,administration

8/9/2019 Alchemi-678-2006

8/40

Alchemi: )er)iew

8/9/2019 Alchemi-678-2006

9/40

Alchemi "ntroduction

8/9/2019 Alchemi-678-2006

10/40D

8/9/2019 Alchemi-678-2006

11/40DD

Alchemi Architecture

Alchemi Manager

e-$cience Application e-8usiness Application e-EngineeringApplication

8/9/2019 Alchemi-678-2006

12/40D

!ole-based $ecurity

IAuthentication:$imple username ,password

IAuthori6ation:!ole-basedpermissions

IAuditing:All 1obs,threadse5ecuted arerecorded in adatabase and linkedto user account used'or Authentication

8/9/2019 Alchemi-678-2006

13/40DJ

*ulti-&e)el Grid

*anagersconnectedhierarchically

D> E5ecutor D> wner

"ntermediate*anagers are

)iewed as E5ecutors

(

(

8/9/2019 Alchemi-678-2006

14/40

D2

As a Node 'or Global Grids

Cross-/lat'orm*anager =

8/9/2019 Alchemi-678-2006

15/40

D7

E5ecution *odel

#edicated E5ecutor -way communication between E5ecutor

and *anager

oluntary E5ecutor D-way communication between E5ecutor

and manager E5ecutor works 'rom behind;rewalls%

#ual bene;t Fle5ible resource management Fle5ible deployment

8/9/2019 Alchemi-678-2006

16/40

/rogramming applications with

the Alchemi 'ramework

8/9/2019 Alchemi-678-2006

17/40

D?

Components

Grid application Consists o'

independent gridthreads

*anager = central

controller #isco)ery+ scheduling+

dispatching+ monitoring

Cross /lat'orm *anager

8/9/2019 Alchemi-678-2006

18/40

DK

Grid Thread *odel

Grid Application ob1ect% consisting o' acollection o' Grid Thread ob1ects% Transparent e5ecution o' threads

E)ent-dri)en

!eusable drag-and-drop components

Easy to use #e)elopment o' new grid applications

8/9/2019 Alchemi-678-2006

19/40

D

Grid Thread in CL

[Serializable]

public class MultiplierThread : GThread

{

private int _A, _B, _Result;

public int Result

{

et { return _Result; ! !

public MultiplierThread"int a, int b#

{

_A $ a;

_B $ b;

!

public %verride v%id Start"#

{

_Result $ _A & _B;

!

!

8/9/2019 Alchemi-678-2006

20/40

Grid Application

static v%id Main"#

{

'''

GApplicati%n a $ ne( GApplicati%n"

ne( G)%nnecti%n"*l%calh%st*, +, -user*, -pass(%rd*#

#;

a'Mani.est'Add"ne( M%dule/ependenc0"t0pe%."MultiplierThread#'M%dule##;

.%r "int i$; i12; i33#

{

MultiplierThread thread $ ne( MultiplierThread"i, i32#;

a'Threads'Add"thread#;

!

a'Thread4inish 3$ ne( GThread4inish"Thread4inished#;

a'Start"#;

'''

!

static v%id Thread4inished"GThread th#

{

'''

!

8/9/2019 Alchemi-678-2006

21/40

D

Grid Mob *odel

Tasks consisting o' 1obs H*&-based representation o' tasks and1obs

Command line inter'ace $ubmission *onitoring !esult retrie)al

/rogrammatic inter'ace Grid-enabling e5isting legacy applications

8/9/2019 Alchemi-678-2006

22/40

H*&-8ased Task #e;nition

1tas56

17ani.est6

1e7bedded_.ile na7e$*Reverse'e8e*6[base9enc%ded c%ntents]1

8/9/2019 Alchemi-678-2006

23/40

Alchemi per'ormance

e)aluation

/er'ormance E)aluation D:

8/9/2019 Alchemi-678-2006

24/40

2

/er'ormance E)aluation D:$tandalone Node 4igh /recision /i

Calculation%

8/9/2019 Alchemi-678-2006

25/40

7

/er'ormance E)aluation :Alchemi as a Global Grid Node%

User Application

Gridbus Broker

[Linux Server@manjra.cs.mu.oz.au]

IBM eServer @

ANU, an!erra

"Linux #

$lo!us%

IBM eServer @

A&elai&e "Linux

#

$lo!us%

IBM luster @

'(A,

Mel!ourne

"Linux#$lo!us%

A luster No&e

@ US), San

)ie*o

"Linux#$lo!us%

+in&os )es-top

luster @ Mel!ourne

".N/ # Alc0emi

Mana*er%

[ma**ie.cs.mu.oz.au]

)es-top ( "i121 (entium 3 +in&os 4555% o6 $7I)S La! runnin* Alc0emi

Alc0emi xecutors Alc0emi xecutors

luster @ AIS/,

/su-u!a, 8apan

"Linux#$lo!us%

8/9/2019 Alchemi-678-2006

26/40

/er'ormance E)aluation : !esources

esource )ocation Con*guration GridMiddle+are

maggie.cs.mu.o6.au

8/9/2019 Alchemi-678-2006

27/40

?

/er'ormance E)aluation : D 1obscompleted by di0erent nodes%

8/9/2019 Alchemi-678-2006

28/40

K

Related Works and Comparison

8/9/2019 Alchemi-678-2006

29/40

/ro1ects based on Alchemi

8/9/2019 Alchemi-678-2006

30/40

J

/ro1ects (sing Alchemi

C$"! &and P

8/9/2019 Alchemi-678-2006

31/40

JD

/ro1ects (sing Alchemi%

stochasti5 Gmb4+ GermanyO Commercial /roduct: Asynchronous E5cel Tasks using

*anagedH&& and Alchemi .Net Grid Computing 'ramework*anagedH&&+ is in use on se)eral hundred trading desks at the world9s largest banks and

hedge-'unds%

Correlation $ystems &td.+ "sraelO Fo5Fi5+ an Alchemi-based tool 'or estimating the

location o' an 4F radio transmitter using $$& technology

$atyam Computers+ "ndiaO micro-array data processing application 'or early

detection o' breast cancer

8/9/2019 Alchemi-678-2006

32/40

J

/ro1ects (sing Alchemi J%

GridCrypt = Grid course student pro1ect #istributed symmetric key cryptography

E5celGrid = Grid course student pro1ect E5cel 'ront-end to Alchemi P Gridbus 8roker )ia E5cel plug-

in% N*A*"T+ Nitte+ Qarnataka+ "ndia

Fault detection using the n-modular redundancy and )oting /ri)ate Grid "nitiati)e The Netherlands%

D node testbed 'or inno)ati)e applications nline 'or R J months

$ilica $outh A'rica% Financial ser)ices company: #istributed 'und administration

system And many more@

8/9/2019 Alchemi-678-2006

33/40

JJ

Alchemi community

pen-source so'tware

4osted on source'orge.net

Acti)e user , de)eloper community

8/9/2019 Alchemi-678-2006

34/40

$ummary

8/9/2019 Alchemi-678-2006

35/40

J7

$ummary

Alchemi is an easy-to-use Enterprise Grid'ramework

0ers a simple way to get up and running

with a

8/9/2019 Alchemi-678-2006

36/40

J

Current $tatus

Alchemi is currently in ).D..2

).D.. includes an impro)ed e5tensible Consoleinter'ace 'or monitoring and administration

E5ecutor P *anager now run in ser)ice-mode as a

$$TE* user% such that they are always runningwithout the need 'or a user to logon

*anager storage is abstracted out+ so that any !#8*$which has a le#b dri)er can be used as the back-end

storage system. "n-memory storage is also possible. New Ma)a A/" 'or 1a)a clients to submit 1obs to Alchemi

*anager web-ser)ice

"mpro)ed 'ault-tolerance and logging

8/9/2019 Alchemi-678-2006

37/40

J?

Future

8/9/2019 Alchemi-678-2006

38/40

JK

*ore "n'o:

http:,,www.alchemi.net $o'tware download

$ource code G/&%

E5amples #ocumentation

!esources 'or users P de)elopers

News

/apers

Contact: ra1+ kna UVcsse.unimelb.edu.auW

8/9/2019 Alchemi-678-2006

39/40

J

Acknowledgements

Team Qrishna Nadiminti Acti)e de)eloper%+ A. &uther /ro1ect

'ounder,#e)eloper% and !. 8uyya C",*entor% "nter'ace to Gridbus 8roker

$. enugopal

!. !an1an E5ecutor $er)ice

!odrigo Assirati #ias and his team Applications and Tools

GridCrypt = A $etiawan+ #. Adiutama+ M. &iman E5celGrid = Q. Nadiminti

$ponsored 8y (ni)ersity o' *elbourne A!C #isco)ery /ro1ects

The (ser and de)eloper community on $F.net

8/9/2019 Alchemi-678-2006

40/40

Thank you