Time Utility Functions for Modeling andEvaluating Resource Allocations in a

Heterogeneous Computing SystemLuis Diego Briceno1, Bhavesh Khemka1, Howard Jay Siegel1,2, Anthony A. Maciejewski1,

Christopher Groer3, Gregory Koenig3, Gene Okonski4, and Steve Poole3,4

Colorado State University 3Oak Ridge National Laboratory 4Department of Defense1 Department of Electrical and Oak Ridge, TN 37830 Washington, DC 20001

Computer Engineering2 Department of Computer Science

Fort Collins, CO 80523

email: LDBricen, Bhavesh.Khemka, HJ, AAM@colostate.eduGroerCS, Koenig, SPoole@ornl.gov

okonskitg@verizon.net

Abstract—This study considers a heterogeneous comput-ing system and corresponding workload being investigatedby the Extreme Scale Systems Center (ESSC) at OakRidge National Laboratory (ORNL). The ESSC is part ofa collaborative effort between the Department of Energy(DOE) and the Department of Defense (DoD) to deliverresearch, tools, software, and technologies that can beintegrated, deployed, and used in both DOE and DoDenvironments. The heterogeneous system and workload de-scribed here are representative of a prototypical computingenvironment being studied as part of this collaboration.Each task can exhibit a time-varying importance or utilityto the overall enterprise. In this system, an arriving taskhas an associated priority and precedence. The priority isused to describe the importance of a task, and precedenceis used to describe how soon the task must be executed.These two metrics are combined to create a utility functioncurve that indicates how valuable it is for the systemto complete a task at any given moment. This researchfocuses on using time-utility functions to generate a metricthat can be used to compare the performance of differentresource schedulers in a heterogeneous computing system.The contributions of this paper are: (a) a mathematicalmodel of a heterogeneous computing system where tasksarrive dynamically and need to be assigned based on theirpriority, precedence, utility characteristic class, and taskexecution type, (b) the use of priority and precedence togenerate time-utility functions that describe the value atask has at any given time, (c) the derivation of a metricbased on the total utility gained from completing tasks tomeasure the performance of the computing environment,and (d) a comparison of the performance of resourceallocation heuristics in this environment.

This work was supported by the United States Department ofDefense and used resources of the Extreme Scale Systems Centerat Oak Ridge National Laboratory. This research also used the CSUISTeC HPC System supported by NSF Grant CNS-0923386.

I. INTRODUCTION

In certain High Performance Computing systems, acommon metric for evaluating the performance of thesystem and the scheduler is observing the system uti-lization. In a homogeneous computing system (wheretask execution times do not vary across machines), thismetric intuitively makes sense because the executiontime of a task will be the same on any machine towhich it is assigned. However, a high utilization of aheterogeneous computing system (where task executiontimes may vary across machines) does not necessarilyimply the system is being used in the most effective way.The question that needs to be answered in some hetero-geneous environments becomes: How do we measuresystem performance?

This study presents a simulation of an oversubscribedcomputing environment with system characteristics andworkload parameters based on the expectations forfuture environments of DOE and DoD interest. Theperformance of the target system is dependent on howmany tasks are executed, how much useful work is done,and how timely this information is to the customer.In the target system, tasks can have several differentlevels of priority, i.e., how important is it for the taskto be computed. Additionally, the precedence of a taskindicates how quickly it loses utility. The value of higherprecedence tasks decay more rapidly with time. Thepriority and precedence are used to create a time-utilityfunction that calculates the utility a task will earn ifit is completed at a specific time (e.g., [17], [33]).In our simulation studies, each dynamically arriving

2011 IEEE International Parallel & Distributed Processing Symposium

1530-2075/11 $26.00 © 2011 IEEE

DOI 10.1109/IPDPS.2011.123

7

2011 IEEE International Parallel & Distributed Processing Symposium

1530-2075/11 $26.00 © 2011 IEEE

DOI 10.1109/IPDPS.2011.123

7

2011 IEEE International Parallel & Distributed Processing Symposium

1530-2075/11 $26.00 © 2011 IEEE

DOI 10.1109/IPDPS.2011.123

7

task will belong to one of a fixed number of “utilitycharacteristic classes,” and to one of a fixed number of“task execution types.” The utility characteristic classis used in conjunction with the priority and precedenceof a task to generate a time-utility function [17] thatspecifies the utility a task will achieve based on when itcompletes. Task execution types are groups of tasks withsimilar computational characteristics. In our model, theworkload consists of serial tasks (i.e., non-parallelizable)that are independent (i.e., there is no inter-task com-munication). A more detailed description of these taskfeatures are discussed in Section II-D.

In this paper, we use time-utility functions to measureand quantify the performance of the resource allocationheuristics in the system. In the target system, tasksarrive dynamically and the resource allocation heuristicdoes not know ahead of time when the next task willarrive, what utility characteristic class it will have, whatthe task’s priority will be, what task execution type itwill have, and what the task’s precedence level will be.Therefore, a utility function for a specific task is onlyknown after it arrives. In this study, the goal of theheuristics is to maximize the total system utility, whichis the sum over all tasks of each task’s accrued utility(described in detail in Section II).

The contributions of this paper are: (a) a mathematicalmodel of a heterogeneous computing system where tasksarrive dynamically and need to be assigned based ontheir priority, precedence, utility characteristic class, andtask execution type, (b) the use of priority and prece-dence to generate time-utility functions that describe thevalue a task has at any given time, (c) the derivation ofa performance metric based on the total utility gainedfrom completing tasks, and (d) a comparison of theperformance of resource allocation heuristics in thisenvironment.

This paper is organized as follows. The next sectionis a description of the time utility functions. SectionIII explains the problem statement. Section IV gives adescription of the resource allocation heuristics we used.A comparison of our work with other related work ontime-utility functions is given in Section V. In SectionVI, we describe how the simulations are structured. Theresults are shown in Section VII, and the conclusionsare in Section VIII.

II. TIME UTILITY FUNCTIONS

A. Overview

Time-utility functions are created to quantify theutility of completing each task at a given time. Thistime utility function is based on priority, precedence,and utility characteristic class. These utility functionsmodel the importance of tasks to the overall enterprise

and are based on discussions with system designers.All the utility functions we consider are monotonicallydecreasing. However, in general, these utility functionscan have any arbitrary shape. We do not use arbitraryshapes in an effort to avoid overwhelming the users byhaving them specify a utility value at any given pointin time. Therefore, the goal is to use a small set ofparameters that the users understand and that enablesthem to obtain the desired utility curve.

We designed the time-utility function framework usedin this simulation environment to be flexible. Given thatthe characteristics of the tasks used in real systems areunknown, it is important to be able to create a largenumber of possible utility characteristic classes. Whilethe precedence and priority determine the basic informa-tion about the utility function, the utility characteristicclass allows us to fine tune the basic utility function toobtain the final utility function. This provides flexibilityin our model.

B. Priority

The priority of a task is based on the importance ithas to the organization. It is assigned to each task bythe system owner in collaboration with the customersubmitting the task. Each priority level has an associatedmaximum “utility” level that is used to compare tasksagainst others in different priority groups. The maximum“utility” of tasks with the same priority level is the same.We assume there are four priority levels (critical, high,medium, and low). The critical tasks are uncommon (lessthan 5% of tasks in our simulation study); however, theoverall utility of the system is heavily influenced by thetimely execution of these tasks. The other priorities aremore common.

In our model, π1 represents the maximum utility ofcritical priority tasks, π2 the maximum utility of highpriority tasks, π3 the maximum utility of medium leveltasks, and π4 the maximum utility of low priority tasks.The difference among the maximum utility of prioritylevels is exponential, e.g., high priority can have a utilityof 100 and medium priority a utility of 10.

C. Precedence

The precedence of a task is based on how soonthe output of the task is needed. The precedence isrepresented by the “shape” of the “utility” function. Thesystem uses four levels of precedence, where precedencelevel i indicates a faster decline in utility over time thanlevel i+1. The precedence of a task is determined by thedesigner/programmer. In this study, we assume exponen-tial decay rates, where β(L) represents the exponentialdecay rate of tasks with a precedence level of L.

888

400

600

800

1,000

1,200

uti

lity

i1t

i2t i

3t i4t i

5t

0

200

0 20 40 60 80

completion time from arrivalprecedence 1 precedence 2 precedence 3 precedence 4

(a) Utility function that is composed of five different time intervalsis illustrated.

200

300

400

500

600

700

800

900

1,000

uti

lity

0

100

200

0 10 20 30 40 50completion time from arrival

precedence 1 precedence 2 precedence 3 precedence 4

(b) Utility function that consist of one time interval is illustrated.

Fig. 1. Examples of time-utility functions for different precedence levels for a utility characteristic class i are shown.

D. Utility Characteristic Class

The utility function associated with a task is based onthe task’s precedence, priority, and utility characteristicclass. For our study, there are four different utilitycharacteristic classes.

Each utility characteristic class i is composed of threedifferent elements. The first element is the set of τi timeintervals used to partition the time axis of the utilityfunction. The start time for the kth interval, t(k, i) isrelative to the arrival time of that task.

The second element of a utility characteristic class iis the percentage of maximum utility at the beginningof a time interval k, u(k, i), where 0 < u(k, i) ≤ 1. Themaximum utility for a utility characteristic class i withpriority x in time interval k is

u(k, i, x) = u(k, i) · π(x) (1)

The last element is a modifier Γ(k, i) for the expo-nential decay in each time interval k (1 ≤ k ≤ τi);in our studies, 0.8 ≤ Γ(k, i) ≤ 1.2. The precedencedecay modifier for a utility characteristic class i with aprecedence level of ρ at an interval k is therefore

α(k, i, ρ) = β(ρ) · Γ(k, i). (2)

For task h, let x ∈ critical, high,medium, lowrepresent the priority level, ρ ∈ 1, 2, 3, 4 repre-sent the precedence level, and i represents the utilitycharacteristic class. The utility of this task at time t(which is relative to the arrival time of task h), wheret(k, i) ≤ t < t(k + 1, i) is

Ui,ρ,x(t) =u(k, i, x)− u(k + 1, i, x)

eα(k,i,ρ)·(t−t(k,i)) + u(k + 1, i, x).

(3)An example of the utility functions for different prece-

dence levels is shown in Figure 1. The utility functionshown in Figure 1(a) has five intervals, while Figure 1(b)only has one interval.

E. Model of Environment

The target system simulated here has M heteroge-neous machines (M = 100) and 10,000 tasks. In oursimulation study, we consider two types of machines:general purpose and special purpose machines. The gen-eral purpose machines are capable of running most tasksbut do not offer significant advantages in execution time.In contrast, the special purpose machines can execute afew tasks with a significant improvement in executiontime. In this simulation environment, we assume thattasks cannot be preempted.

For each task execution type i, we assume that anEstimated Time to Compute on each machine j has beenprovided, denoted ETC(i, j). The generation of ETCvalues for our simulation is explained in Section VI-B.The relative values of the entries in the ETC matrixrepresent how “heterogeneous” the environment is. Thisassumption is common in the literature (e.g., [3], [10],[12], [19], [22], [37], [42]).

In this environment, the workload consists of a smallset of task execution types. Therefore, it is reasonableto collect information about the running times of thesetask execution types. We assume that the initial ETCvalues can be determined by doing experiments, and thenrefined as the scheduler accumulates historical data.

Due to the nature of special purpose machines in ourstudy, certain tasks cannot execute on some machines.In our simulation model, if task h cannot execute onmachine j then ETC(h, j) =∞.

999

F. Performance Metric

Let tfinished(h) represent the time when task h fin-ishes executing, Uh→i,ρ,x(tfinished(h)) represent theutility that task h obtained, and Ω(t) represent the set oftasks that have finished execution by time t. The measureof performance (system utility) in this heterogeneouscomputing system is defined as

usystem(t) =∑

h∈Ω(t)

Uh→i,ρ,x(tfinished(h)). (4)

III. PROBLEM STATEMENT

In this study, we evaluate the performance of differentresource allocation heuristics in the computing systemenvironment described in Sections I and II. The goal ofeach heuristic is to maximize the performance metricdescribed in Equation 4. We evaluate the heuristicsconsidering two types of heterogeneous environments:inconsistent and partially-consistent (defined in SectionVI-B), and two different arrival rates.

IV. HEURISTICS

A. Types of Dynamic Mode Heuristics

There are several types of dynamic mode (also knownas online [25]) heuristics in the literature. The twotypes we consider in this study are the immediate modeand batch mode heuristics [30]. The immediate modeheuristics assign incoming tasks to machines as soon asthey arrive, and their assignment is never changed.

For batch mode heuristics, there is a virtual queueof tasks that is composed of tasks that have arrivedbut are not pending or currently executing (as shown inFigure 2). This virtual queue is stored at the scheduler.Tasks that are in the virtual queue can be reassignedin subsequent mapping events. This contrasts with tasksthat are pending or executing that cannot be reassignedin subsequent mapping events. These batch mode heuris-tics execute periodically after a given time interval (i.e.,one minute in this simulation study). The batch modeheuristics gather incoming tasks before allocating thesetasks and any tasks in the virtual queue to machines.

In this study, we do not allow a task to be preempted.The optimal use of preemption adds to the complexityof the scheduling system. Additionally, the use of batchmode heuristics will minimize the advantage gained bypreempting tasks. In future studies, it may be possibleto incorporate preemption to maximize the total systemutility.

B. Immediate Mode Heuristics

1) Round RobinThe Round Robin assigns the first task that arrives for

a mapping event to the first machine, the second task to

m1

m2

m3

m4

executing

tasks

pending

tasksvirtual queue

in the scheduler

t1

t4

t5

t6

t3

t2

t7

t8

t10

t13

t14

t11

t15

t16

t9

t12

t18

t17

tasks in the virtual queue

can be reassigned in

subsequent mapping events

pending tasks and

executing tasks cannot be

reassigned

Fig. 2. Illustration of the virtual queue used in the batch modeheuristics. Row i corresponds to tasks on machine i

the second machine, and so on. The algorithm for thisheuristic is given in Figure 3.2) Random

As the name suggests, the Random heuristic randomlyassigns an incoming task to some machine. The proce-dure used for the Random heuristic is shown in Figure4. The Round Robin and the Random heuristic are usedas baseline heuristics for comparison purposes.3) Maximum Utility

The Maximum Utility heuristic is a greedy heuristicthat picks the machine that will obtain the highest utilityfor each task. For this task, the machine that obtains themaximum utility will also have the earliest completiontime. Therefore, this heuristic is similar to the MinimumCompletion Time (MCT) heuristic [5]–[7], [16], [30],[31], [35], [38]. The procedure used to assign tasks withthe Maximum Utility heuristic is shown in Figure 5.

The Maximum Utility heuristic takes into considera-tion not only the execution times of the task h on themachines but also the ready times of the machines (whena machine will complete all tasks already assigned toit). Once an assignment has been made, the machineready time of that machine is updated. The tasks areconsidered for mapping in the order of their arrival. Thedrawback of this heuristic is that there is no guaranteethat tasks will be assigned to the machines with theirfastest execution times (versus earliest completion time).4) Minimum Execution Time (MET)

The Minimum Execution Time heuristic [5]–[7], [16],[30], [31] is a greedy heuristic that assigns the currenttask to the machine that executes it the fastest. Thetasks are considered for assignment in the order of theirarrival, and each task is assigned to the machine thatcan execute it the quickest. This heuristic is sometimesreferred to as “user directed assignment” because theusers will want their program to run on the fastest

101010

(1) Let count be a counter using for assigning tasks to machines, and set its initial value to one.(2) Consider next task h that just arrived.(3) If task h can execute on machine (count mod M ) + 1, then assign task h to (count mod M ) + 1.(4) If task h cannot execute on machine (count mod M ) + 1, then increase count until task h is able to

execute. Thus, machines that are skipped in this step must wait until the next pass through the list ofmachines to have a task assigned to it.

(5) count = count+ 1, for the next mapping event.(6) Go to step (2).

Fig. 3. Procedure for using the Round Robin heuristic to generate a resource allocation.

(1) Consider task h that just arrived and the set of machines that can execute h.(2) Assign task h to a randomly selected machine from the set of machines that can execute it.

Fig. 4. Procedure for using the Random heuristic to generate a resource allocation.

(1) Consider task h that just arrived.(2) Assign task h to the machine that can obtain the highest utility.(3) Update the machine ready time.

Fig. 5. Procedure for using Maximum Utility heuristic to generate a resource allocation.

machine.The procedure for the Minimum Execution Time

heuristic is shown in Figure 6. The Minimum ExecutionTime heuristic only considers the execution times forthe task on the machines and ignores the ready timesof the machines (earliest time when that task can beginexecution on that machine). The drawback of MinimumExecution Time is that it can lead to only a few machinesbeing used, and this can lead to long queues on fastmachines.5) K-percent Best

The K-percent Best algorithm [1], [6], [7], [13], [23],[30], [31] combines both the MET and the MaximumUtility heuristics by choosing machines based on boththese heuristics. When a new task arrives, the K-percentBest algorithm picks a machine in two stages. In thefirst screening process, it picks K-percent of the bestmachines (M · K/100) that give the lowest executiontimes among all the machines. In the next step, amongthe machines chosen in the first step, it picks themachine that gives the maximum utility. In this way, K-percent Best picks a machine that is partially an METmachine and partially a Maximum Utility machine aswell. The K parameter controls the extent to which theK-percent Best algorithm is biased toward the MET orthe Maximum Utility machine. For example, if K is setto 100 then in the first stage all the machines would beselected and the result of K-percent Best would be tochoose the maximum utility machine. However, if K isset to 100/M , then in the first stage only one machineis picked and that is the MET machine. The procedurefor the K-percent Best heuristic is shown in Figure 7.In our implementation, the K parameter was empiricallydetermined to be 50, that is, 50% of the MET machineswere chosen in the first phase.

C. Batch Mode Heuristics

1) Min-Min Completion TimeThe Min-Min heuristic [6], [7], [11], [13], [16], [18],

[21], [23], [24], [30], [35], [36], [38], [41] is a two phasegreedy heuristic. In the first phase, it independently findsthe minimum completion time machine for every task.In the next phase, for every task-machine pair foundin the first phase, it picks the pair that has the earliestcompletion time (second Min). It then assigns that taskto that machine. The procedure used to implement Min-Min Completion Time is shown in Figure 8.2) Max-Max Utility

The Max-Max heuristic [7], [23], [31], [38] is similarin principle to the Min-Min heuristic. The only differ-ence is that we maximize the utility in each of the twophases. The procedure for Max-Max Utility is shown inFigure 9.3) Sufferage

The Sufferage heuristic [4], [6], [8], [11], [14], [18],[21], [23], [24], [30], [32], [34], [35], [40] attemptsto maximize the utility earned by assigning every taskto the machine that would give the highest utility, butwhen multiple tasks want the same machine it givespreference to the task that would “suffer” the most if itwere not assigned to that machine. If maxutility(h) isthe best obtainable utility of task h, and smaxutility(h)is the second highest utility obtainable for task h, thenthe sufferage value (suff ) is suff = maxutility(h) −smaxutility(h). The procedure of the Sufferage heuristicis described in Figure 10.

V. RELATED WORK

The authors in [17] present a system where processesare running on a simulated, symmetric, shared-memory

111111

(1) Consider task h that just arrived.(2) Assign task h to the machine with the smallest estimated time to compute.

Fig. 6. Procedure for using the Minimum Execution Time (MET) heuristic to generate a resource allocation.

(1) Consider task h that just arrived.(2) Determine the K-percent fastest execution time machines for task h.(3) From this set of K-percent best machines, assign h to the machine that can obtain the highest utility.(4) Update the machine ready time.

Fig. 7. Procedure for using the K-percent Best heuristic to generate a resource allocation.

(1) Make a set composed of the unmapped tasks.(2) For each task h in the set of unmapped tasks, determine the machine that has the minimum completion

time.(3) From the task/machine pairs found in (2), determine the task/machine pair (task h, machine j) with

the overall smallest completion time.(4) Assign task h to machine j, and update the machine ready time.(5) Remove h from set of unmapped tasks.(6) Repeat steps 2 through 5 until all tasks are mapped.

Fig. 8. Procedure for using the Min-Min Completion Time heuristic to generate a resource allocation.

(1) Make a set composed of the unmapped tasks.(2) For each task h in the set of unmapped tasks, determine the task/machine pair that obtains the maximum

utility time.(3) From the machine found in (2), determine the task/machine pair (task h, machine j) with the overall

maximum utility time.(4) Assign task h to machine j, and update the machine ready times.(5) Remove h from set of unmapped tasks(6) Repeat steps 2 through 5 until all tasks are mapped.

Fig. 9. Procedure for using the Max-Max Utility heuristic to generate a resource allocation.

multi-processor (SMP) with one to four processing el-ements. Examples of the systems using the time-utilityfunction shapes defined in [17] can be found in [27]and [28]. Similar to our model, [17] associates a valuefunction with every process that specifies the value thatwill be earned by the system if it completes executionof that process at any given moment. Every process hasa request time associated with it that gives the timefrom which the process is available for execution. Therequest time can be in the future, which makes thatprocess unschedulable at this instant, but the schedulercan consider these times to make current schedulingdecisions. This is different from our model, where weassume that the scheduler has no prior knowledge ofthe arrival time of the tasks. Moreover, in [17] theprocesses can be either periodic or aperiodic, whereasin our case we only model aperiodic tasks. The au-thors in [17] use four different shapes for the valuefunctions, including an exponential decay model. Theypresent two new algorithms that make decisions basedon value density (value divided by processing time).Their simulations show that these algorithms did betterthan basic scheduling algorithms that consider eitheronly deadline or only execution times. The authors in

[17] consider homogeneous processing elements, whilewe consider a heterogeneous system. We do not usevalue density for guiding decisions (as done in [17])of our heuristics because this metric makes sense onlyin the homogeneous environment of [17] as opposed toour heterogeneous environment.

In [9], the authors consider the problem of schedulinga set of non-preemptive and mutually independent taskson a single processor. Associated with each task is aTime Value Function, that gives the contribution eachtask will provide at its completion time. Each task hasa soft and a hard deadline. The contribution of a taskstays constant at its highest value up to its soft deadline.Beyond its hard deadline, the contribution of a taskis zero. Between the soft and the hard deadline, thecontribution value of a task falls from its highest valueto zero. The aim of the problem is to sequence theset of tasks to the single processor in an order thatmaximizes the cumulative contribution from the wholetask set. Thus, while [9] maximizes the total utilityearned, the environment of a single processor versusour environment of a heterogeneous distributed systemmakes solution techniques quite different for the twocases.

121212

(1) Make a set composed of the unmapped tasks.(2) While there are still unmapped tasks:

(a) For each machine find the set of tasks that have their maximum utility on this machine.(i) If the set of tasks is size one, then assign the corresponding task to the machine and remove

the task from the set of unmapped tasks.(ii) If the size of the set of tasks is greater than one, then assign task with the highest suff, and

remove that task from the set of unmapped tasks.(b) The ready times for all machines are updated.

Fig. 10. Procedure for using the Sufferage heuristic to generate a resource allocation.

In [20], the authors consider a single-processor real-time system. They use analytical methods to createperformance features that they then optimize. In theirmodel, they represent the utility that a completed jobwill earn by using time-utility functions. The time-utilityfunctions have non-zero values between the arrival timeand the deadline of the jobs. Unlike our model, whereevery task can have a different shape for its time-utility function, [20] models a system wherein all jobshave the same shape for their time-utility functions.As mentioned before, solution techniques become sig-nificantly different when working with single-processorsystems (as considered in [20]) as opposed to workingwith heterogeneous systems with multiple processors (asconsidered in our model).

Having fixed time-utility function shapes for variousjobs might not be an accurate representation of reality.The authors of [26] point out that every user has theirknown agenda and as a result, every job that theysubmit should accurately have a time-utility functiontailor-made for their needs. This study considers ahomogeneous computing system. They propose a modelwherein, when High Performance Computing users sub-mit their jobs, they draw out a time-utility function basedon their needs. It is up to the users to decide howaccurately they want to model the time-utility functions.Thus, similar to our model, the time-utility functions in[26] usually have time intervals (of varying lengths) anddifferent decay rates within each of these intervals. Theauthors then propose a genetic algorithm for schedulingwith the objective of maximizing the total accrued utility.This genetic algorithm has an average execution time of8,900 seconds. For the purposes of our work in whichscheduling decisions must be made at much smallerintervals (e.g., 30 seconds or less), the direct approachemployed in the paper is not suitable. Furthermore,in our study, we assume a heterogeneous computingsystem, and we incorporate the concept of precedenceand priority as scale and shape factors to modify theutility function.

The authors in [39] apply time-utility functions to rep-resent the utility that is earned based on the transmission

duration for packets between hosts. Each host can send apacket from a given set of possible packets. Each packethas a time-utility function associated with it, and a utilityis earned based on the time duration of the transmis-sion. The goal is to maximize the aggregate accruedutility over the arrival of all packets to their respectivedestinations. Similar to our model, they consider onlyuni-modal (non-increasing) time-utility functions. Theyconsider six time-utility functions in their study: step,soft-step, linear, exponential, quadratic, and composite.In the study, the authors define a concept that theycall “critical times” that is similar to our definitionof time intervals. However, before the first “criticaltime” the utility is at its maximum, and then it dropsto zero at the second time interval. In our study, weconsider a High Performance Computing system (notpacket transmission). Additionally, we have a systemwhere the completion of some tasks is significantly moreimportant than other tasks. We incorporated this priorityand precedence information as scale and shape modifiersof general time utility functions.

In [29], the authors apply time-utility functions tomodel the utility obtained when a thread successfullycompletes execution on a QNX Neutrino real-time op-erating system (with a single processor). In our study, weapply utility functions to a heterogeneous High Perfor-mance Computing system with thousands of processors.A “task” in our paper is equivalent to a “thread” in [29].In contrast to our model, they incorporate preemption ofthreads. In addition, unlike our model, where the utilityof function of a task may decrease over time, they havenon-real-time threads whose time-utility function is setto a constant value in order to represent threads that donot have time-constrained activity.

There are also utility functions that vary as a functionof the Quality of Service (QoS) associated with theoutput. For example, a video conferencing task can havemultiple utilities associated with the different levels ofQoS. The utility gained from executing this task withless quality, e.g., lower bandwidth, would be less thanthe utility gained from executing it with a higher quality,e.g., higher bandwidth. In [15], the authors explore the

131313

use of utility functions for highly configurable tasks indynamic real-time systems, i.e., phased-array radars. Inthis research, the utility a task gains upon completionis not dependent on the completion time. In [15], theutility of a task is based on “QoS dimensions.” Thesedimensions represent an aspect of task quality that isof direct relevance to the user, e.g., a level of audiocompression. This type of QoS based utility functionis significantly different from our time-utility basedfunctions.

VI. SIMULATION TRIAL SETUP

A. Generation of Simulation Trials

The target system simulated here has two differenttypes of heterogeneity (partially-consistent and inconsis-tent), and two different arrival rates (normal and fast).For each of these arrival rates and heterogeneities, 100different trials are generated.

Each simulation trial has M = 100 heterogeneousmachines and 10,000 dynamically arriving tasks (fromten different task execution types and four differentutility characteristic classes). The ETC matrix, for eachtrial, will be composed of 100 columns to represent themachines and ten rows for each different task executiontype. The procedure for generating the ETC matrices isdescribed in Section VI-B

The priority, precedence, task execution type, andutility characteristic class for the ith task will in generalvary from one trial to another, i.e., the properties areassigned independently for each trial. All the simulationtrials (both inconsistent and partially-consistent) use thesame four utility characteristic classes. The method usedto assign arrival times, precedence, and priorities to tasksis described in Section VI-C.

B. Generation of Estimated Time to Compute Matrices

The ETC values are generated using a modified ver-sion of the procedure described in [2]. The differencebetween [2] and how we generate the ETC values is theuse of the trimodal and bimodal distributions insteadof a single gamma distribution. We use a trimodaldistribution (as shown in Figure 11(a)) to generate ETCvalues for a task that executes significantly faster onspecial purpose machines, and a bimodal distribution (asshown in Figure 11(b)) for a task that can execute wellacross multiple machines.

Consider tasks that can make use of some of thespecial purpose machines. In Figure 11(a), the first modeis a gamma distribution that represents the executiontime of a task on special purpose machines. The secondmode is a gamma distribution used to represent theexecution time of a task on most machines. In the thirdmode, a pulse is used to represent the execution time of

a task on machines that cannot execute it (i.e., a machinenot capable of executing that particular task).

A bimodal distribution is used to generate an ETCvalue for tasks that do not benefit from executing onspecial purpose machinesThe first mode of the bimodaldistribution in Figure 11(b) is used to generate the ETCvalue of a task on a machine that can execute it (similarto the second mode of Figure 11(a)). While, the secondmode is used to represent the execution time of a taskon a machine that cannot execute it (similar to thethird mode of Figure 11(a)). That is, the second moderepresents special purpose machines for which thesetasks cannot execute (e.g., a C++ program executing ona Java machine).

Approximately one-tenth of the task execution typescan use special purpose machines that can execute themvery quickly. For these task execution types that executesignificantly quicker on special purpose machines, theprobability of using the first mode in Figure 11(a) is5%, the probability of the second mode is 85%, and theprobability that a machine cannot execute that task is10%. For the remaining task execution types, the firstmode has a probability of 90%, and the second modehas a probability of 10%.

ETC matrices generated for this simulation representtwo different types of actual heterogeneity: inconsistentand partially-consistent. An inconsistent matrix is onewhere if any task ti has a lower execution time onmachine x than machine y, then nothing can be inferredabout the relative performance of a task tj on machinex versus machine y. For a partially-consistent ETCmatrix within a consistent sub-matrix, if ti has a lowerexecution time on machine x than machine y (both x andy are part of the consistent sub-matrix), then the same istrue for any tk within the sub-matrix. For the partially-consistent matrices simulated here, the consistent sub-matrix was 50% of the task execution types and 50% ofthe machines. For this study, 100 different ETCs weregenerated for each type of consistency.

In our study, the following procedure and parametersare used to generate the ETC values. The methodologyfor generating the value is similar to [2]; however, thereare a few differences. For each task execution type usinga mean estimated computation time of ten minutes andtask coefficient of variation of 0.1, we sample a gammadistribution to obtain a mean for a specific task executiontype.

C. Generation of Arrival Rate, Utility CharacteristicClass, Precedence, and Priority for each Task

The arriving task has several characteristics that aredetermined stochastically for the purpose of our simu-lation. The first characteristic is the arrival time. The

141414

pro

babili

ty f

unction

10%

0.1 10

execution time (minutes)

infinity

pro

babili

ty f

unction

0

0

5% 85%

(a)

pro

babili

ty f

unction

10%

10 infinity

pro

babili

ty f

unction

0

0

90%

execution time (minutes)

(b)

Fig. 11. The illustrations represent the trimodal and bimodal distributions used to generate the ETC matrices for our simulations. Thepercentages for the different modes represent the probability of sampling that mode. (a) A trimodal distribution is used to generate the ETCvalues for the tasks that can run faster on special purpose machines. (b) A bimodal distribution is used to generate the ETC values for thetasks that do not benefit from special purpose machines.

3

4

5

6

7

8

9

10

uti

lity

fixed precedence (4)

and priority (medium)

0

1

2

3

0 20 40 60completion time from arrival

utility class 1 utility class 2 utility class 3 utility class 4

Fig. 12. The four different utility characteristic classes in mediumpriority and precedence level four (lowest).

TABLE ITHE JOINT PROBABILITY MATRIX SHOWING THE DISTRIBUTION OFTASKS ACROSS THE PAIRS OF POSSIBLE PRECEDENCE LEVELS AND

PRIORITY LEVELS.

precedencepriority 1 2 3 4critical 2% 2% 0.05% 0%high 3.45% 5% 1.5% 3%

medium 0% 10% 10% 10%low 0% 0% 20% 33%

interarrival times of these tasks follow a Poisson distri-bution. Therefore, the length of the time interval betweenthe arrival of tasks is an exponential random variable.The mean of this random variable is 5.4 seconds for thenormal arrival rate and 4.2 seconds for the fast arrivalrate. These arrival rates were selected in conjunction

with the mean execution time to ensure the system isoversubscribed.

The next characteristic that needs to be determined isthe utility characteristic class. There are four differentclasses in this study, and each class has an equal prob-ability of being selected. The corresponding parametersof a utility characteristic class (time intervals, utility atthe beginning of the interval, and decay rate modifier)are representative of workloads anticipated in futurecomputing environments of DOE and DoD interest.Figure 12 shows the utility functions for four differentutility characteristic classes with a critical priority anda precedence level four.

The precedence and priority of a task were selectedusing a matrix that represents joint probabilities of aprecedence at a given priority level. An example of thistable is shown in Table I. In this figure, for example, atask has a 2% probability of having a “critical” levelpriority and precedence level of one (p(priority =“critical′′ ∩ precedence = 1) = 2%). For this study,the maximum utility values for each priority level areπ1 = 1000, π2 = 100, π3 = 10, and π4 = 1 . The valuesused for precedence decay are β1 = 0.6, β2 = 0.2,β3 = 0.1, β4 = 0.01.

VII. RESULTS

A comparison between the simulation results with anarrival rate of 11.1 tasks per minute (Figure 13) and14.3 tasks per minute (Figure 14) show the advantage ofusing batch mode heuristics in a highly oversubscribedenvironment. The main difference between the resultsshown in Figures 13 and 14 is that the performanceof the immediate mode heuristics degrades much more

151515

0

50,000

100,000

150,000

200,000

250,000

300,000

350,000

400,000

450,000

Random Round Robin

Max. Utility MET KPB Min-Min CT

Max-Max Utility

Sufferage

tota

l u

tili

ty o

bta

ined

heuristics

batch mode heuristicsimmediate mode heuristics

Fig. 13. The averaged results of 100 trials for the inconsistent machine heterogeneity with arrival rate of 11.1 tasks per minute. The errorbars represent the 95% confidence intervals.

than the performance of the utility-based batch modeheuristics.

The inconsistent heterogeneity results show that thebest performing heuristics are Sufferage and Max-MaxUtility. Results from the partially-consistent hetero-geneity show very similar relative performance amongheuristics compared to the inconsistent heterogeneityresults with both arrival rates. In all the results, the Suf-ferage has overlapping confidence intervals with Max-Max Utility. The Sufferage heuristic runs slower than theother heuristics (see Table II). In general, as shown inTable II, the immediate mode heuristics execute in lessthan one millisecond, while the batch mode heuristicsutility based heuristics can take from 200ms to 500ms.This difference could become more significant if weconsider a larger system.

The performance of the best immediate mode heuris-tics is not as good as the utility-based batch mode heuris-tics. The best performing immediate mode heuristicsare the Maximum Utility and K-percent Best heuristics,which have an average utility of approximately 135,000units for the results with an arrival rate of 11.1 tasksper minute. The best performing batch heuristics are onaverage 23% of the maximum possible system utility ifthe system is not oversubscribed, i.e., the sum of themaximum utility value of all 10,000 tasks. The Mini-mum Execution Time heuristic performs poorly becausethere are only 10 task execution types; therefore, thesystem only has 10 minimum execution time machines

and the other M − 10 = 90 machines are never used.The simulation results with an arrival of 14.3 tasks per

minute (Figure 14) show how important it is to keeptasks in the virtual queue if the system is very over-subscribed. The performance at 14.3 tasks per minutedegrade for all heuristics; however, the performance ofutility-based batch mode degrades by approximately 7%while the performance of immediate mode heuristicsdegrade by approximately 50%. The utility-based batchmode heuristics are better at scheduling critical andhigh priority tasks when the system is oversubscribed,because they can ignore lower priority tasks in favor ofhigher priority tasks. Therefore, they are able to outper-form the immediate mode heuristics that are forced toallocate all tasks as they arrive.

VIII. CONCLUSIONS

The primary goal of this study is to define a generalmodel of task-based time-utility functions and overallsystem performance. Although our model is based pri-marily on guidance from ESSC customers, we believethe model is sufficiently flexible to be useful in otherenvironments as well. The second goal is to design andevaluate heuristics that can be used to derive resourceallocations that attempt to maximize this system perfor-mance measure. The results show that the performanceof the the Sufferage and Max-Max heuristics are betterthan the other heuristics considered in this study. In thissimulation environment, heuristic execution times are

161616

0

50,000

100,000

150,000

200,000

250,000

300,000

350,000

400,000

450,000

Random Round Robin

Max. Utility MET KPB Min-Min CT

Max-Max Utility

Sufferage

tota

l u

tili

ty o

bta

ined

heuristics

batch mode heuristicsimmediate mode heuristics

Fig. 14. The averaged results of 100 trials for the inconsistent machine heterogeneity with arrival rate of 14.3 tasks per minute. The errorbars represent the 95% confidence intervals.

TABLE IITABLE SHOWING THE AVERAGE EXECUTION TIME FOR HEURISTICS FOR ALL 10,000 TASKS.

heuristicsarrival Random Round Robin Max. Utility MET KPB Min-Min CT Max-Max Utility Sufferagenormal 0.46ms 0.56ms 0.67ms 0.74ms 1.34ms 0.1ms 0.3s 0.4s

fast 0.45ms 0.54ms 0.67ms 0.736ms 1.28ms 0.12ms 0.34s 0.5s

significantly higher for the batch mode heuristics thanfor immediate mode heuristics. We intend to observehow these times change as the system scales.

Possible future work includes using stochastic esti-mates of execution time, avoiding task starvation byimplementing a penalty function, varying the size ofthe partially consistent sub-matrix, incorporating harddeadlines, considering additional heuristic approaches,incorporating energy consumption in the system model,using wall clock completion times in addition to com-pletion times that are relative to arrival times, allowingparallel and dependent tasks (i.e., tasks modeled asdirected acyclic graphs), permitting preemption, andimplementing these heuristics in an existing schedulerwithin the ESSC.

Acknowledgments: The authors thank Abdulla Al-Qawasmeh and Dalton Young for their valuable com-ments.

REFERENCES

[1] I. Al-Azzoni and D. G. Down, “Linear programming basedaffinity scheduling for heterogeneous computing systems,” inInternational Conference on Parallel and Distributed ProcessingTechniques and Applications (PDPTA ’07), Jun. 2007.

[2] S. Ali, H. J. Siegel, M. Maheswaran, D. Hensgen, and S. Ali,“Representing task and machine heterogeneities for heteroge-neous computing systems,” Tamkang Journal of Science andEngineering, Special 50th Anniversary Issue, vol. 3, no. 3, pp.195–207, Nov. 2000.

[3] H. Barada, S. M. Sait, and N. Baig, “Task matching andscheduling in heterogeneous systems using simulated evolution,”in 10th IEEE Heterogeneous Computing Workshop (HCW ’01),Apr. 2001, pp. 875–882.

[4] F. Berman, R. Wolski, H. Casanova, W. Cirne, H. Dail, M. Faer-man, S. Figueira, J. Hayes, G. Obertelli, J. Schopf, G. Shao,S. Smallen, S. Spring, A. Su, and D. Zagorodnov, “Adaptivecomputing on the grid using AppLeS,” IEEE Trans. on Paralleland Distributed Systems (TPDS), vol. 14, no. 4, pp. 369–382,Apr. 2003.

[5] T. D. Braun, H. J. Siegel, N. Beck, L. Boloni, R. F. Freund,D. Hensgen, M. Maheswaran, A. I. Reuther, J. P. Robertson,M. D. Theys, and B. Yao, “A comparison of eleven staticheuristics for mapping a class of independent tasks onto het-erogeneous distributed computing systems,” Journal of Paralleland Distributed Computing, vol. 61, no. 6, pp. 810–837, Jun.2001.

[6] L. D. Briceno, M. Oltikar, H. J. Siegel, and A. A. Maciejewski,“Study of an iterative technique to minimize completion timesof non-makespan machines,” in 16th Heterogeneous ComputingWorkshop (HCW 2007), in the proceedings of the 21st Inter-national Parallel and Distributed Processing Symposium, Mar.2007.

[7] L. D. Briceno, H. J. Siegel, A. Maciejewski, M. Oltikar, J. Brate-man, J. White, J. Martin, and K. Knapp, “Heuristics for robustresource allocation of satellite weather data processing onto a

171717

heterogeneous parallel system,” IEEE Transactions on Paralleland Distributed Systems, accepted to appear.

[8] H. Casanova, A. Legrand, D. Zagorodnov, and F. Berman,“Heuristics for scheduling parameter sweep applications in gridenvironments,” in 9th IEEE Heterogeneous Computing Workshop(HCW 2000), Mar. 2000, pp. 349–363.

[9] K. Chen and P. Muhlethaler, “A scheduling algorithm for tasksdescribed by time value function,” Journal of Real-Time Systems,vol. 10, no. 3, pp. 293–312, May 1996.

[10] M. K. Dhodhi, I. Ahmad, and A. Yatama, “An integrated tech-nique for task matching and scheduling onto distributed hetero-geneous computing systems,” Journal of Parallel and DistributedComputing, vol. 62, no. 9, pp. 1338–1361, Sep. 2002.

[11] Q. Ding and G. Chen, “A benefit function mapping heuristic fora class of meta-tasks in grid environments,” in CCGRID ’01: 1stInternational Symposium on Cluster Computing and the Grid,May 2001.

[12] A. Ghafoor and J. Yang, “A distributed heterogeneous supercom-puting management system,” IEEE Computer, vol. 26, no. 6, pp.78–86, Jun. 1993.

[13] S. Ghanbari and M. R. Meybodi, “On-line mapping algorithmsin highly heterogeneous computational grids: A learning au-tomata approach,” in International Conference on Informationand Knowledge Technology (IKT ’05), May 2005.

[14] ——, “Learning automata based algorithms for mapping of aclass of independent tasks over highly heterogeneous grids,” inEuropean Grid Conference (EGC ’05), Feb 2005, pp. 681–690.

[15] S. Ghosh, J. P. Hansen, R. Rajkumar, and J. Lehoczky, “Inte-grated resource management and scheduling with multi-resourceconstraints,” in 25th Real-Time Systems Symposium, Dec. 2004.

[16] O. H. Ibarra and C. E. Kim, “Heuristic algorithms for schedulingindependent tasks on non-identical processors,” Journal of theACM, vol. 24, no. 2, pp. 280–289, Apr. 1977.

[17] E. Jensen, C. Locke, and H. Tokuda, “A time-driven schedul-ing model for real-time systems,” in IEEE Real-Time SystemsSymposium, Dec. 1985, pp. 112–122.

[18] Z. Jinquan, N. Lina, and J. Changjun, “A heuristic schedulingstrategy for independent tasks on grid,” in Eighth InternationalConference on High-Performance Computing in Asia-PacificRegion 2005, Nov. 2005.

[19] M. Kafil and I. Ahmad, “Optimal task assignment in heteroge-neous distributed computing systems,” IEEE Concurrency, vol. 6,no. 3, pp. 42–51, Jul. 1998.

[20] M. Kargahi and A. Movaghar, “Performance optimization basedon analytical modeling in a real-time system with constrainedtime/utility functions,” IEEE Transactions on Computers, Pre-Print 2011.

[21] K. Kaya, B. Ucar, and C. Aykanat, “Heuristics for schedulingfile-sharing tasks on heterogeneous systems with distributedrepositories,” Journal of Parallel and Distributed Computing,vol. 67, no. 3, pp. 271–285, Mar. 2007.

[22] A. Khokhar, V. K. Prasanna, M. E. Shaaban, and C. Wang,“Heterogeneous computing: Challenges and opportunities,” IEEEComputer, vol. 26, no. 6, pp. 18–27, Jun. 1993.

[23] J.-K. Kim, S. Shivle, H. J. Siegel, A. A. Maciejewski, T. Braun,M. Schneider, S. Tideman, R. Chitta, R. B. Dilmaghani, R. Joshi,A. Kaul, A. Sharma, S. Sripada, P. Vangari, and S. S. Yellam-palli, “Dynamically mapping tasks with priorities and multipledeadlines in a heterogeneous environment,” Journal of Paralleland Distributed Computing, vol. 67, no. 2, pp. 154–169, Feb.2007.

[24] J.-K. Kim, H. J. Siegel, A. A. Maciejewski, and R. Eigenmann,“Dynamic resource management in energy constrained hetero-geneous computing systems using voltage scaling,” IEEE Trans-actions on Parallel and Distributed Systems, Special Issue onPower-Aware Parallel and Distributed Systems, vol. 19, no. 11,pp. 1445–1457, Nov 2008.

[25] C. M. Krishna and K. G. Shin, Real-Time Systems. McGraw-Hill, 1997.

[26] C. B. Lee and A. E. Snavely, “Precise and realistic utilityfunctions for user-centric performance analysis of schedulers,” in

16th International Symposium on High Performance DistributedComputing (HPDC ’07). New York, NY, USA: ACM, 2007,pp. 107–116.

[27] P. Li, B. Ravindran, H. Cho, and E. D. Jensen, “Schedulingdistributable real-time threads in tempus middleware,” in TenthInternational Conference on Parallel and Distributed Systems(ICPADS ’04), 2004, pp. 187–194.

[28] P. Li, B. Ravindran, S. Suhaib, and S. Feizabadi, “A formallyverified application-level framework for real-time schedulingon posix real-time operating systems,” IEEE Transactions onSoftware Engineering, vol. 30, no. 9, pp. 613–629, Sep. 2004.

[29] P. Li, H. Wu, B. Ravindran, and E. D. Jensen, “A utility accrualscheduling algorithm for real-time activities with mutual exclu-sion resource constraints,” IEEE Transactions on Computers,vol. 55, no. 4, pp. 454–469, Apr. 2006.

[30] M. Maheswaran, S. Ali, H. J. Siegel, D. Hensgen, and R. F.Freund, “Dynamic mapping of a class of independent tasksonto heterogeneous computing systems,” Journal of Parallel andDistributed Computing, vol. 59, no. 2, pp. 107–121, Nov. 1999.

[31] A. M. Mehta, J. Smith, H. J. Siegel, A. A. Maciejewski,A. Jayaseelan, and B. Ye, “Dynamic resource allocation heuris-tics that manage tradeoff between makespan and robustness,”Journal of Supercomputing, Special Issue on Grid Technology,vol. 42, no. 1, pp. 33–58, Jan. 2007.

[32] D. Paranhos, W. Cirne, and F. Brasileiro, “Trading cycles forinformation: Using replication to schedule bag-of-tasks applica-tions on computational grids,” in International Conference onParallel and Distributed Computing, Aug. 2003.

[33] B. Ravindran, E. D. Jensen, and P. Li, “On recent advancesin time/utility function real-time scheduling and resource man-agement,” in Eighth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC 2005), May2005, pp. 55–60.

[34] P. SaiRanga and S. Baskiyar, “A low complexity algorithm fordynamic scheduling of independent tasks onto heterogeneouscomputing systems,” in 43rd annual Southeast Regional Con-ference - Volume 1, Mar. 2005, pp. 63–68.

[35] V. Shestak, J. Smith, H. J. Siegel, and A. A. Maciejewski,“Stochastic robustness metric and its use for static resourceallocations,” Journal of Parallel and Distributed Computing,vol. 68, no. 8, pp. 1157–1173, Aug. 2008.

[36] S. Shivle, H. J. Siegel, A. A. Maciejewski, P. Sugavanam,T. Banka, R. Castain, K. Chindam, S. Dussinger, P. Pichumani,P. Satyasekaran, W. Saylor, D. Sendek, J. Sousa, J. Sridharan,and J. Velazco, “Static allocation of resources to communicatingsubtasks in a heterogeneous ad hoc grid environment,” Journal ofParallel and Distributed Computing, Special Issue on Algorithmsfor Wireless and Ad-hoc Networks, vol. 66, no. 4, pp. 600–611,Apr. 2006.

[37] H. Singh and A. Youssef, “Mapping and scheduling hetero-geneous task graphs using genetic algorithms,” in 5th IEEEHeterogeneous Computing Workshop (HCW 1996), Apr. 1996,pp. 86–97.

[38] P. Sugavanam, H. J. Siegel, A. A. Maciejewski, M. Oltikar,A. Mehta, R. Pichel, A. Horiuchi, V. Shestak, M. Al-Otaibi,Y. Krishnamurthy, S. Ali, J. Zhang, M. Aydin, P. Lee, K. Guru,M. Raskey, and A. Pippin, “Robust static allocation of resourcesfor independent tasks under makespan and dollar cost con-straints,” Journal of Parallel and Distributed Computing, vol. 67,no. 4, pp. 400–416, Apr. 2007.

[39] J. Wang and B. Ravindran, “Time-utility function-drivenswitched ethernet: Packet scheduling algorithm, implementation,and feasibility analysis,” IEEE Transactions on Parallel andDistributed Systems, vol. 15, no. 2, pp. 119–133, Feb. 2004.

[40] B. Wei, G. Fedak, and F. Cappello, “Scheduling independenttasks sharing large data distributed with bittorrent,” in The 6thIEEE/ACM International Workshop on Grid Computing, Nov.2005.

[41] M. Wu and W. Shu, “Segmented min-min: A static mappingalgorithm for meta-tasks on heterogeneous computing systems,”

181818

in 9th Heterogeneous Computing Workshop (HCW ’00), Mar.2000, pp. 375–385.

[42] D. Xu, K. Nahrstedt, and D. Wichadakul, “QoS and contention-aware multi-resource reservation,” Cluster Computing, vol. 4,no. 2, pp. 95–107, Apr. 2001.

BIOGRAPHIES

Luis D. Briceno received his Ph.D. degree inElectrical and Computer Engineering from ColoradoState University, and his B.S. degree in Electricaland Electronic Engineering from the University ofCosta Rica. He is currently a post doctoral scholar atColorado State University. His research interests includeheterogeneous parallel and distributed computing.

Bhavesh Khemka is a Ph.D. student and researchassistant in the Electrical and Computer EngineeringDepartment at Colorado State University. He receivedhis B.E. degree in Electrical and Electronics Engineeringfrom Hindustan College of Engineering affiliated withAnna University, India. His research interests includerobust resource allocations in heterogeneous anddistributed computing environments.

H. J. Siegel was appointed the Abell Endowed ChairDistinguished Professor of Electrical and ComputerEngineering at Colorado State University (CSU) in2001, where he is also a Professor of Computer Scienceand Director of the CSU Information Science andTechnology Center (ISTeC). From 1976 to 2001, hewas a professor at Purdue University. Prof. Siegel isa Fellow of the IEEE and a Fellow of the ACM. Hereceived a B.S. degree in electrical engineering and aB.S. degree in management from the MassachusettsInstitute of Technology (MIT), and the M.A., M.S.E.,and Ph.D. degrees from the Department of ElectricalEngineering and Computer Science at PrincetonUniversity. He has co-authored over 380 technicalpapers. His research interests include robust computingsystems, resource allocation in computing systems,heterogeneous parallel and distributed computing andcommunications, parallel algorithms, and parallelmachine interconnection networks. He was a Coeditor-in-Chief of the Journal of Parallel and DistributedComputing, and has been on the Editorial Boards ofboth the IEEE Transactions on Parallel and DistributedSystems and the IEEE Transactions on Computers.He has been an international keynote speaker andtutorial lecturer, and has consulted for industryand government. For more information, please seewww.engr.colostate.edu/∼hj.

Anthony A. Maciejewski received the B.S.,M.S., and Ph.D. degrees in Electrical Engineering

in 1982, 1984,and 1987, respectively, all from TheOhio State University. From 1988 to 2001, he wasa Professor of Electrical and Computer Engineeringat Purdue University. In 2001, he joined ColoradoState University where he is currently the Head of theDepartment of Electrical and Computer Engineering.He is a Fellow of IEEE. A complete vita is available atwww.engr.colostate.edu/∼aam.

Christopher Groer is a research staff scientist inthe Computational Mathematics Group at Oak RidgeNational Lab. His research interests include large scalecombinatorial optimization problems, job scheduling,and High Performance Computing. He worked as anapplied research mathematician for the Departmentof Defense from 2002 to 2008 and received a B.A.from Vanderbilt (math and economics), M.A. fromUniversity of Georgia (mathematics), and a Ph.D. fromthe University of Maryland (applied math and scientificcomputation).

Gregory Koenig is an R & D Associate at OakRidge National Laboratory where his work involvesdeveloping scalable system software and parallel toolsfor ultrascale-class parallel computers. His researchinterests include middleware for grid, cloud, andon-demand/utility computing incorporating technologiessuch as virtualization, fault detection and avoidance,and resource scheduling. He holds a Ph.D. (2007) andM.S. (2003) in Computer Science from the Universityof Illinois at Urbana-Champaign as well as three B.S.degrees (Mathematics, 1996; Electrical EngineeringTechnology, 1995; Computer Science, 1993) fromIndiana University-Purdue University Fort Wayne.

Gene Okonski is a Systems Architect for theDepartment of Defense. His areas of focus are ondistributed computing, data processing architectures,and information technology efficiency. He has beeninvolved as a technologist and leader of a numberof large scale development activities spanning sometwenty years. He received his B.S. Degree in ElectricalEngineering with a minor in Economics from ColoradoState University (1988). He received an M.S. inEngineering from Johns Hopkins University Schoolof Engineering with a concentration in SystemsEngineering (1993).

Steve Poole currently divides his time between du-ties at the Department of Defense and Oak RidgeNational Laboratory. He is the Chief Scientist andDirector of Special Programs. He can be reached atspoole@ornl.gov.

191919