Grid Computing Framework A Java framework for managed modular distributed parallel computing.
-
Upload
eustace-bryan -
Category
Documents
-
view
222 -
download
3
Transcript of Grid Computing Framework A Java framework for managed modular distributed parallel computing.
Grid Computing FrameworkA Java framework for managed modular distributed parallel computing
Presentation Plan
1. Discussion on distributed computing(Concept, examples, insight)
2. What are we doing (new?)(Our concept, innovation, vision)
3. Who is doing what(Technical details, responsibilities, SE Practices)
4. What remains to be done(Or as they say – ‘future scope’)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 2
Grid Computing Framework
Parallel Processing (problem classes)1. SIMD
(Single Instruction Multiple Data)
Used mostly in DSP i.e.Digital Signal Processing applications
11/11/2010 Abhishek Ganguly | Apoorv Pandey 3
Grid Computing Framework – Discussion on distributed computing
Bass Boost
Left | Center | Right
Parallel Processing (problem classes)1. SIMD
(Single Instruction Multiple Data)
2. MISD(Multiple Instruction Single Data)
Very few actually use or implement this aspect of parallelism
Task replication systems may use this technique for fault tolerance or error masking etc
11/11/2010 Abhishek Ganguly | Apoorv Pandey 4
Grid Computing Framework – Discussion on distributed computing
Parallel Processing (problem classes)1. SIMD
(Single Instruction Multiple Data)
2. MISD(Multiple Instruction Single Data)
3. MIMD(Multiple Instruction Multiple Data)
Most commonly adopted distributed processing strategy
11/11/2010 Abhishek Ganguly | Apoorv Pandey 5
Grid Computing Framework – Discussion on distributed computing
Parallel Processing (Solution Approaches)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 6
Grid Computing Framework – Discussion on distributed computing
Parallel Processing (Solution Approaches)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 7
Grid Computing Framework – Discussion on distributed computing
Centralized parallelism
Common Memory (RAM)
Common Die(Multi Core)
Common Board(Multi Processor)
Parallel Processing (Solution Approaches)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 8
Grid Computing Framework – Discussion on distributed computing
Centralized parallelism
Multi Core
Common Memory (RAM)
Common Die(Multi Core)
Common Board(Multi Processor)
• Common Die or Chip. Everything on a single Chip.• Common System bus.• Common L2 cache
• Individual L1 cache• Individual Register file• Individual ALU
• May or may not share a common Clock
Parallel Processing (Solution Approaches)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 9
Grid Computing Framework – Discussion on distributed computing
Multi Processor
Common Memory (RAM)
Common Die(Multi Core)
Common Board(Multi Processor)
• Common Motherboard• Common OS instance• Common RAM
• Individual Clock• Individual System Bus• Individual RAM access
• Higher Throughput than multi core • Higher power consumption than multi core
11/11/2010 Abhishek Ganguly | Apoorv Pandey 10
Grid Computing Framework – Discussion on distributed computing
Distributed parallelismParallel Processing (Solution Approaches)
Reduced Coupling
11/11/2010 Abhishek Ganguly | Apoorv Pandey 11
Grid Computing Framework – Discussion on distributed computing
Distributed parallelism – SUPER COMPUTERParallel Processing (Solution Approaches)
• Highly coupled systems with specialized hardware built for the purpose• May be composed of hundreds of motherboards, each housing multiple CPUs• RAMs spread all over the housing• Interconnected by special high speed bus (Network)• Each board may have its own copy of the OS.
Requires• Special OS• Special software to run atop it• Special programming methodology
11/11/2010 Abhishek Ganguly | Apoorv Pandey 12
Grid Computing Framework – Discussion on distributed computing
Distributed parallelism – CLUSTERSParallel Processing (Solution Approaches)
•These fall between super computers and grid computers•These are networked computers•Computers are connected through high speed LAN•Each computer has its own instance of OS running for it
These are the highest deployed distributed systems.
Uses•Load balancing (as in web servers) •Availability, Robustness, fail-safe (DNS serves)•Throughput increase (database clusters)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 13
Grid Computing Framework – Discussion on distributed computing
Distributed parallelism – GRID COMPUTERSParallel Processing (Solution Approaches)
• Most diffused, heterogeneous and independent form of multi node alliance. • Components of a grid computer can be anything from a commodity PC to clusters of supercomputers.• Components can be located al over the globe.• Internet is the basic networking medium between the components.
Requires• Special programming technique• Specially written software• Extensive GRID framework
11/11/2010 Abhishek Ganguly | Apoorv Pandey 14
Grid Computing Framework – Discussion on distributed computing
Grid Computer Architecture
Grid User(Programmer) Grid Coordinator Server
Workers
11/11/2010 Abhishek Ganguly | Apoorv Pandey 15
Grid Computing Framework – Discussion on distributed computing
Grid Computer Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 16
Grid Computing Framework – Discussion on distributed computing
Grid Computer Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 17
Grid Computing Framework – What are we doing (new?)
Project Proposal
To implement such a distributed framework in java
To adapt java multi-threading concepts for scalable distributed computing
11/11/2010 Abhishek Ganguly | Apoorv Pandey 18
Grid Computing Framework – What are we doing (new?)
Implementing Distributed Framework
Network
MachineMachine MachineMachine MachineMachine
JVMJVM JVMJVM JVMJVM JVMJVM
Grid Frame Work
UserProgram User
Sub-ProgramUser
Sub-Program
Grid CodeGrid Code Grid CodeGrid Code Grid CodeGrid Code Grid CodeGrid Code
Grid APIGrid API Grid APIGrid API Grid APIGrid API
11/11/2010 Abhishek Ganguly | Apoorv Pandey 19
Grid Computing Framework – What are we doing (new?)
Implementing Distributed Framework
Network
Machine Machine Machine
JVM JVM JVM JVM
Grid Frame Work
UserProgram User
Sub-ProgramUser
Sub-Program
Grid Code Grid Code Grid Code Grid Code
Grid APIGrid API Grid APIGrid API Grid APIGrid API
11/11/2010 Abhishek Ganguly | Apoorv Pandey 20
Grid Computing Framework – What are we doing (new?)
Adapting Multi Threading Concepts
Common Java Code
Common Java Code
11/11/2010 Abhishek Ganguly | Apoorv Pandey 21
Grid Computing Framework – What are we doing (new?)
Adapting Multi Threading Concepts
Grid Aware java
Code
11/11/2010 Abhishek Ganguly | Apoorv Pandey 22
Grid Computing Framework – Who is doing what, Technical details etc
Functional Objectives
Research
Design
Coding
Testing by using
11/11/2010 Abhishek Ganguly | Apoorv Pandey 23
Grid Computing Framework – Who is doing what, Technical details etc
Research on existing work
Basic understanding of parallel computingAbhishek & Apoorv
MPI, Java RMI , Directory and Naming servicesApoorv
Object Serialization, CORBA, Dot net RemotingApoorv
Existing grid computing frameworks. BOINC, OPEN-Grid, Abhishek
Distributed file systems, Distributed SynchronizationAbhishek
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 24
Grid Computing Framework – Who is doing what, Technical details etc
Client
Job ServerWorker
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 25
Grid Computing Framework – Who is doing what, Technical details etc
Client
Job ServerWorker
Client uses Grid APIorg.accurate.grid
org.accurate.grid.Job
Job
Runnable SerializableThread
void start()void Run()void join()
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 26
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 27
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin
JobServer.submitJob( Job j );>>
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 28
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin Job j=JobServer.getJob()>>
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 29
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin JobServer.doneJob( Job j )>>
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 30
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin
Job j =JobServer.getCompleted(this)>>
Software DesignData Flow Diagram 0
11/11/2010 Abhishek Ganguly | Apoorv Pandey 31
Grid Computing Framework – Who is doing what, Technical details etc
Grid FrameworkGrid Framework
Jobs
Completed Results
Software DesignData Flow Diagram 1
11/11/2010 Abhishek Ganguly | Apoorv Pandey 32
Grid Computing Framework – Who is doing what, Technical details etc
1.2Job Queuing
and dispatching
1.2Job Queuing
and dispatching
1.1Job Creation
1.1Job Creation
1.3Job Execution
1.3Job Execution
Code
blockUser Program
User Program
Job Queue Completed Job Store`` ` `
User Program
User Program
Child job modules
All that remains to be done
11/11/2010 Abhishek Ganguly | Apoorv Pandey 33
Grid Computing Framework – Future Scope
Virtual distributed file system.A distributed file system for high volume data interchange
Dedicated application servers.So that long running applications do notrequire their originating PCs to stay on.
Integration with existing grid computing systems.So that jobs can be interchanged between them.
Automatic parallelization.So that code can be parallelized withoutexplicit programmer effort.
All that remains to be done…
11/11/2010 Abhishek Ganguly | Apoorv Pandey 34
Grid Computing Framework – Future Scope
Wrapper to precompiled codeSo that precompiled modules can bereused without reengineering
Worker credit trackingSo that workers can be gifted benefit a/c to work they do
Standardization of APISo that programmers of different platformscan feel comfortable.
11/11/2010 Abhishek Ganguly | Apoorv Pandey 35
Grid Computing Framework – References
90% of the research material has been obtained from the internetAnd rest 10% from text books
Text Books:
Herbert Schildt, Java 2- The complete reference, Osborne.Client Server, Most of practical java
Bruce Eckel, thinking in java, PearsonMost of theoretical java
Galvin, Operating system conceptsScheduling, multitasking, synchronization
11/11/2010 Abhishek Ganguly | Apoorv Pandey 36
Grid Computing Framework – References
90% of the research material has been obtained from the internetAnd rest 10% from text books
Websites :
WikipediaAbout everything has been consulted on Wikipedia
Oracle sun tutorial-trailjava code samples, in-depth class structure
boinc.berkeley.eduBasic understanding of one of the most successful grid.
11/11/2010 Abhishek Ganguly | Apoorv Pandey 37
Grid Computing Framework – Rebuttle
Questions
11/11/2010 Abhishek Ganguly | Apoorv Pandey 38
Grid Computing Framework – Thank you
Thank You