Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization
description
Transcript of Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization
![Page 1: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/1.jpg)
Parallel Job Submission In Grid Environment Using
Parallel Particle Swarm Optimization
Dr. G. Sudha SadhasivamAsst. ProfessorDept. of CSE.PSG College Of Technology.
D. Komagal Meenakshi (07MW05)
PSG College Of Technology.
![Page 2: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/2.jpg)
Outline• Scheduling in Grid.• Problem Statement• Need For Job Grouping in Scheduling• Previous Work Done in Job Grouping• Proposed System• Trust Based Filtering of jobs• Particle Swarm Optimization• Parallel PSO• Model for PPSO• Dynamic jobs• Results• Conclusion and Future work• Bibliography
![Page 3: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/3.jpg)
Scheduling in Grid.• Grid computing is a high performance computing
environment to solve large scale computational demands.
• Task scheduling is a fundamental issue in achieving high performance in grid computing systems.
• Reason: Large numbers of tasks are computed on the
geographically distributed resources, a reasonable scheduling algorithm must be adopted order to minimize job completion time with uniform load distribution.
![Page 4: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/4.jpg)
Need
An unorganized deployment of grid applications with a large amount of fine-grain jobs
Leads to
communication overhead dominate the overall processing time
Low computation-communication ratio.
Results
![Page 5: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/5.jpg)
Need For Job Grouping in Scheduling
• Efficient job grouping-based scheduling system is required.
• A Grid Scheduler shouldReduce the total transmission of user jobs to/from
the resources.Reduce the overhead processing time of the jobs
at the resources.
![Page 6: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/6.jpg)
Job Grouping
Dynamically assemble
Transmit
Grid resources
job groups [ coarse grained ]
Jobs of an application [ fine grained ]
![Page 7: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/7.jpg)
Previous Work Done in Job Grouping
• Comparison of Scheduling algorithms with and without job grouping.
• In the context of DAG scheduling, grouping of jobs into clusters to reduce inter-job communication.
• Job Grouping strategy, adaptive to run time environment
• Job Grouping with PSO.
![Page 8: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/8.jpg)
Proposed System• A novel job grouping method using Parallel PSO
• To reduce the communication overhead.
• Enhance the speed of completion of processes.
• Improve resource utilization.
• Improve parallel efficiency.
• Uses PPSO to select the resources to minimize the make span.
• Trust level and dynamism of jobs is considered
• Tool Used - Gridsim-4.2-beta.
![Page 9: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/9.jpg)
The Project aims at …
• Job Grouping based on trust Using PPSO• Parallel Job Submission• Enhancing Computation-communication Ratio• Reducing The Overall Processing Time Of Jobs Using
Parallelization • Improving Resource Utilization In The Grid Environment.• Trust based job filtering• Dynamic job submission
![Page 10: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/10.jpg)
Dynamically assemble Using PPSO
Transmit to
Grid resources
job groups [ coarse grained ]
Filtered Jobs of an application [ fine grain] based on Trust
Grid resources Grid resources
In Parallel
1. Job Grouping
![Page 11: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/11.jpg)
Total number of jobs
Average MI rate of job
MI deviation Percentage
Overhead processing time
Granularity time
Grid Resource
Grid resource 0
Grid resource 1
Grid resource N
Grid Resource File
User Input
GridletsGrid resources’ characteristics
Gridlet MI Resource MIPS Granularity time
Total MIPS
Grid resource 0
Gridlet group 0
Grid resource 1
Gridlet group 1
Grid resource 2
Gridlet group 2
Gridlet groups Resource IDs
…..
Gridlet Scheduler
(1)
(3)
(4)
(5)
(6)
(7)(2)
Trust level
In parallel
Filter jobs based on trust
Job Grouping
![Page 12: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/12.jpg)
2. Trust Based Filtering of jobs• The Grid Information Service GIS gives the information
about all the trust level of the resources .
• The user submits the jobs with different trust values.
• From this, the jobs that have trust values greater than the resource's trust value are filtered out.
• Trust aware resource management and scheduling offer Quality of Service at application layer in grid environment.
![Page 13: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/13.jpg)
3. Particle Swarm Optimization• If large numbers of tasks are computed on the
geographically distributed resources, a reasonable scheduling approach must be adopted in order to get the minimum completion time.
• Task scheduling is a NP-Complete problem
• Heuristic optimization algorithm can be used to solve NP-complete problems.
![Page 14: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/14.jpg)
• Particle Swarm Optimization (PSO) is an evolutionary optimization technique inspired by nature.
• It simulates the process of a swarm of birds preying.
• Its global searching ability can be used for neural network training, control system analysis and design, structural optimization.
• It also has fewer algorithm parameters than genetic algorithm.
• PSO algorithm works well on most global optimal problems.
![Page 15: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/15.jpg)
PSO Concept
• A swarm intelligence based algorithm finds a solution to an optimization problem in a search space.
• Proposed solution exists in the form of a fitness function. • The swarm is typically modeled by particles in
multidimensional space that have a position and a velocity.
• A Particle is a candidate solution in the population and represents a task.
![Page 16: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/16.jpg)
• Particles fly through hyperspace .
• An iterative process to improve candidate solutions is set in motion. The particles iteratively evaluate the fitness of the candidate solutions.
• Particles posses two essential reasoning capabilities– Memory of their own best position and – knowledge of the global best of the swarm.
• As the swarm iterates, the fitness of the global best solution improves.
• All particles being influenced by the global best eventually approach the global best. This phenomenon is called 'convergence'.
![Page 17: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/17.jpg)
PSO Algorithm• Initialize parameters
• Initialize population randomly
• Initialize each particle position vector and velocity vector
Do {• Update each particle’s velocity and position;• Find a permutation according to the updated each particle’s
position;• Evaluate each particle and update the personal best and the global
best;• Apply the local search;• } While (!Stop criterion)
![Page 18: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/18.jpg)
![Page 19: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/19.jpg)
Parallel PSO
Recent advances in computer and network technologies led to parallel optimization algorithms.
Parallel PSO (parallel implementation of stochastic optimization alg)
![Page 20: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/20.jpg)
Parallel PSO design
Intialize
f(x) f(x) f(x)
Check Convergence
Update
# of particles#
of it
erat
ion
s
![Page 21: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/21.jpg)
Model for PPSO
Master
Slave Slave Slave
SEND GLOBAL VALUE RECEIVE INDIVIDUAL VALUE
![Page 22: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/22.jpg)
Gridlet Grouping
Scheduler
Trust based filtered Gridlet list
Resource list
Call PPSO to assign Gridlet To Resources
Create new grouped GridletWith length= Total length
Assign to resources
![Page 23: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/23.jpg)
4. Dynamic jobs
• Dynamic submission of jobs is considered.• User can submit jobs when other jobs are being
processed.• The unused MIPS rating of the resources can be
utilized in a efficient way such that grouping is done by considering the unused MIPS as total MIPS and the jobs are processed.
• Then Parallel Submission of grouped Gridlets to resources is done
![Page 24: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/24.jpg)
![Page 25: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/25.jpg)
Simulation Time for Job Grouping using PSO vs. Parallel PSO
90
100
110
120
130
140
150
20 40 60 80
No of Gridlets
Sim
ula
tio
n T
ime
PPSO
PSO
![Page 26: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/26.jpg)
Total number of processed gridlets for different granularity time and resources
0
20
40
60
80
100
R1 R1-R2 R1-R3 R1-R4 R1-R5
Resources
No
of G
rid
lets
co
mp
lete
d in
gra
n
time
10
20
30
40
50
![Page 27: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/27.jpg)
Load at resources during job grouping with PPSO
0100200300400500600700800900
1000
R1 R2 R3 R4 R5
Resources
loa
d
50 gridlets
60 gridlets
70 gridlets
80 gridlets
90 gridlets
![Page 28: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/28.jpg)
Difference in submission time of gridlets with PSO and PPSO
![Page 29: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/29.jpg)
• Add load balancing feature graph here
![Page 30: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/30.jpg)
Conclusion• The proposed framework using PPSO has less simulation
time compared to job scheduling framework using PSO as the simulation time is reduced.
• Resource selection based on PPSO is used to generate an optimal schedule so as to complete the tasks in a minimum time than PSO as well as utilizing the resources in an efficient way.
• Simulated results demonstrates load balanced resource selection.
• Simulation results demonstrate that PPSO algorithm can get better effect for a large scale optimization problem.
![Page 31: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/31.jpg)
Future Work• Future work would involve developing a more
comprehensive job grouping-based scheduling system that takes into account QoS (Quality of Service) requirements of each user job before performing the grouping method.
• Resource utilization can be done according to the capacity of the resource.
![Page 32: Parallel Job Submission In Grid Environment Using Parallel Particle Swarm Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062304/568144e1550346895db1afc8/html5/thumbnails/32.jpg)
THANK YOU