Modeling the Complexities of Human Performance

Brian F. GoreSan Jose State University/

NASA Ames Research CenterMS 262-12

Moffett Field, CA, USAbgore@mail.arc.nasa.gov

Peter JarvisQSS Inc./

NASA Ames Research CenterMS 262-12

Moffett Field, CA, U.S.A.pjarvis@mail.arc.nasa.gov

Abstract – The Man-machine Integrated Design andAnalysis System (MIDAS) is an integrated humanperformance modeling software tool that symbolicallyrepresents a human’s perceptual, cognitive and motorsystems in an integrated fashion to produce emergent,high level behavioral predictions characteristic of actualhuman performance. MIDAS has been augmented in anumber of significant ways to simulate even more realisticsimulations of human behavior in various aerospaceoperational contexts. The effort undertaken in the currentproject culminated in an agent-based sub model (e.g.slots) that can be used by a variety of models to predictand recreate short- and long-term effects of stressors(fatigue, stress, time pressure, inadequate situationawareness, etc.) on performance in aerospaceaccidents/incidents causation. A computational simulationdemonstrated performance influences brought to taskperformance by fatigue (as characterized by the Yerkes-Dodson theoretical threshold model) that is incurredwhile undertaking activities required to complete a goalbehavior, and the impact of performance-influencingfactors (PIFs) on human performance output by combiningthis with a primitive based action error vulnerability.This paper discusses the computational development effortundertaken in creating the conceptual relationship andPIF implementation in MIDAS.

Keywords: Human performance modeling, MIDAS,performance influencing factors, fatigue modeling, errormodeling.

1 IntroductionHuman performance models (HPMs) and the

human performance modeling process have attempted tointegrate operator characteristics (cognitive, attentional, andphysical) with environmental characteristics to moreaccurately represent vulnerabilities brought to the system’sperformance by the human and enable emergent outputs.Some dynamic, integrated HPMs utilize the cognitive taskanalysis (CTA) process to create their procedural models.The CTA is an ideal way to begin uncovering the cognitiveand performance demands associated with aspects ofcomplex human behavior. The integrated representation andassociation among the various static and dynamic HPMs,includes theoretical, pragmatic, physical and task networkmodels [1]. The output of HPMs has traditionally takenthe form of workload predictions, situation awareness

predictions and procedural and task timelines.1 A key tothe HPM simulation methodology is that the humanoperator is simulated and is not physically present in thesimulation environment. Only the characteristicsassociated with a representative human operator’sperformance are contained within the environment.Workload predictions, timing considerations and systemstate information are produced as output from the dynamicintegrated models. Since the human operator responsiblefor interacting in these systems is not present in the actualsystem evaluation, the risks to the human operator and thecosts associated with system experimentation are greatlyreduced: no experimenters, no subjects, and no testingtime. The framework integrates many aspects of humanperformance allowing each micro model component tobehave as designed, the integration of which replicates ahuman.

2 The Man-machine Integration Designand Analysis System (MIDAS)The Man-machine Integration Design and Analysis

System (MIDAS) is an integrated human performancemodeling tool that symbolically represents manymechanisms that underlie and cause human behavior.MIDAS integrates computational representations of humanperceptual, cognitive and motor systems to enable high-level behavioral predictions characteristic of actual humanbehavior. MIDAS, one of the more comprehensive of thesemodels, has been used for procedural analysis and designsince 19862. MIDAS has proven useful for identifyinggeneral human-system vulnerabilities and cross-domainerror classes and for recommending mitigation strategiesand job re-designs to account for the vulnerable areas, orrisks, in system design [1,3]. MIDAS also possesses acomplex visualization environment that can demonstratethe integration of human/system elements. MIDASrepresents a "first principles" approach, based oncomputational models of the mechanisms that underlie andcause human behavior within the context of human-systemperformance.

1 For a review of the types of modeling tools that operatein this manner [1,2].2 NASA Ames Research Center, and the US Army co-developed MIDAS for military-related applications, whileNASA and SJSU augmented MIDAS within the complex,multi-crew aviation-related environment.

MIDAS’ agent architecture is made up of physicalcomponent agents and human operator agents. Physicalcomponent agents use commercially available computer-aided design (CAD) databases to graphically representphysical entities in an environment. Physical componentagents are the external environmental influences such asterrain and aeronautical equipment. Human operator agentsrepresent models of cognitive, perceptual and motoroperations of a task that describe within their limits ofaccuracy the responses that can be expected of the humanoperator for safe operation of advanced automatedtechnologies. Attention demands are represented byWickens’ Multiple Resource Principle and incorporate atask-loading index initially created by McCracken andAldrich for quantifying attention [4,5]. This scale wasmodified to include a six-channel representation of taskload. Combining attention demands along the input(visual, auditory), central cognitive processing (spatial,verbal), and output (psychomotor, visual) resourcesaccomplish the goal of developing a measure of attentiondemands. In addition, MIDAS incorporates functions thatsimulate the effects of stressors on skilled performancethrough workload and timing exceedances. When thecumulative demands of concurrent tasks exceed an arbitrarythreshold of seven, the operator is assumed to be at greaterrisk for shedding tasks or reduced performance levels,thereby leaving the operator vulnerable to error.

Internal (e.g., intelligence, expertise, personality,emotion, attitudes) and external (e.g., physiologicalstressors such as fatigue and time stress) moderators ofbehavior impact human performance in a variety of waysthat are very important for the aerospace community. Forexample, fatigue, stress and other attention decrementshave been found to be precursors to operator errors [6]. Inaviation, it has been estimated that flight crews’ alertnesslevels are degraded approximately 15% of the time thatthey are on duty, possibly because they are approaching theboundaries or limitations of their performance capabilities.Excessive time on task has been found to negativelyimpact a human operator’s vigilance, accuracy, grammaticalreasoning ability and simple sensory experiences [6]. Aninverse relationship has been found between hours ofwakefulness and performance on a critical task [6].Accurately representing human behavior computationally,therefore, requires that many aspects internal to an operatorthat might impact his performance capability be accuratelyrepresented. This fostered the need for models of erroneousperformance and performance influencing factors (PIFs) inMIDAS as MIDAS contained no representation of PIF-likedegradation effects on performance. This report willsummarize the recent efforts undertaken to generate aprimitive-based error model that becomes triggered by afatigue-related PIF.

2.1 Primitive Action Error Model

The primitive action error model is one that simulatesthe variance in the quality and error probability of anoperator’s primitive actions (reach, grasp, etc) as workloadincreases. The primitive-level fatigue model is based on aperformance degradation model from aviation performance

literature in which erroneous performance occurred [7]. TheMIDAS model displays error vulnerability informationbased on the likelihood of error given an operator’spredicted workloads. It was determined that designing asimulation environment in this manner would permitcreating procedures that could be iteratively refined throughexamination of resource loaded, simulated operators.

The initial boundary of performance capabilities usedthe Yerkes-Dodson arousal model of operator performanceas the theoretical underpinning for MIDAS behavioralpredictions [8]. The Yerkes-Dodson characterization ofarousal predicts an inverted U-shaped relationship betweenarousal and performance. A certain amount of arousal canbe a motivator toward change (e.g., learning). Too much ortoo little change will certainly work against the performer(learner). The desired state of arousal is likely at some mid-level to provide the motivation to change (learn). Too littlearousal has an inerting effect while too much has ahyperactive effect. There are optimal levels of arousal foreach task. These are:

• Lower for more difficult or intellectually(cognitive) challenging tasks (the learners need toconcentrate on the material)

• Higher for tasks requiring endurance andpersistence (the learners need more motivation).

This relationship is consistent with the MIDASrepresentation of operator loads, resource availability, andperformance decrements, and serves to augment thepredictive capabilities of MIDAS. The decrement is set tooccur whenever any of MIDAS primitives are “performed”by the simulated operator (a subset of the MIDASprimitive error table is identified in Table 1) AND whenconcurrent workload levels exceed the boundary level.Table 1 demonstrates the mapping that was developed tolink the primitive actions, the likelihood of error (asidentified by the Information Processing Model’sprobability of failure prediction [9] and the Human FactorsAnalysis Classification System (HFACS) errorclassification) [7]), the additional time taken if an erroroccurs [10], and the quality of the operator’s performanceas a function of the operator’s performance/reliability level[11]. This linkage is a necessary component in generatingrealistic human behaviors in MIDAS.

Table 1: Fragment of the primitive error table

It is important to note that an operator’s performancein MIDAS is set to 100% and is degraded when the load onany workload channel exceeds the theoretical limit of seven

MIDAS Primitive Action PerformanceRange1

ErrorPercentage2

TimePenalty3

QualityPenalty4

fixate-alphanum-object 0 – 20 2.15 150 0fixate-alphanum-object 21 – 45 .72 150 25fixate-alphanum-object 46 – 80 .72 150 50fixate-alphanum-object 81 – 100 2.15 150 0fixate-spatial-object 0 – 20 2.15 150 0fixate-spatial-object 21 – 45 .72 150 50fixate-spatial-object 46 – 80 .72 150 80fixate-spatial-object 81 – 100 2.15 150 0

1 Yerkes-Dodson breakdown analogous breakdown of total workload2 Wickens and Flach’s (1988), and Weigmann and Shappell’s (1987) IP Error Model implementation3 Boff and Lincoln (1988)4 Hollnagel (1998) – MIDAS’ initial conceptualization of reduced reliability in operator’s performance.This is a very preliminary conceptual approach and care needs to be exercised when using quality as aperformance degradation function

(based on [5]). The error classification was broken into threecategories - low, medium, and high. This division wascreated so that the modeling tool would have a trigger rangeof behaviors that parallels theoretical boundaries of operatorperformance that are associated with workload levels [12].These boundaries were determined from the error potentialfor the behavior primitives as outlined by HFACSsummary of error research [7]. For example, the HFACSclassification system outlines that of the total errorpotential attributable to internal human performance was91%, only 2.84% of these were classified as the specificerrors type within the MIDAS primitive structure (sensoryerrors), and therefore resulted in 2.58% error rate(0.91*0.0284*100) in terms of the MIDAS primitive basedtask performance level. There were three performance rangesbased on the Yerkes-Dodson characterization of arousal andperformance that were implemented based on HFACS data. Range 1, considered the low range, will call the low end ofthe error scale (0.0082 or 0.82%). Range 2, considered themedium range, will call a mid level error rate (0.82%*2 =1.64%). Range 3, considered the high range, will call ahigh level of error rate (0.82% * 3 = 2.47%).

This primitive action error model links theoperator’s attention loads to the likelihood that an error willoccur and becomes activated by the performance influencingmodels of the simulated operator. One model thatincorporates a two-dimensional potential for impacting theperformance of the simulated operator is the MIDAS fatiguemodel.

2 . 2 Developing the MIDAS FatigueBehavioral Model

Fatigue is a multidimensional construct that requiresappropriate consideration of the fatigue that an operatorbrings to an activity (pre-task fatigue) and the fatigue thatdevelops as time-on-task extends (within-task fatigue).Two paths were taken to develop the fatigue models inMIDAS – one fatigue model that is triggered at a global,or state, level and one that is triggered at a primitive level.The global level is termed Pre-task fatigue and theprimitive level is termed Within-task fatigue.

2.3 Pre-task fatigue

Fatigue that an operator brings to a task based onhis/her previous activities is modeled as a penalty that isapplied to the MIDAS performance level for the operator.This feature also serves as a placeholder for future linkingto other fatigue models (e.g., the Sleep, Activity, Fatigue,and Task Effectiveness (SAFTE) circadian rhythm modelof sleep deprivation [13] or the two-process model of sleepregulation that interacts the homeostatic regulation processwith the circadian process to generate the timing of sleep-wake cycles [14]). The performance level approachdovetails cleanly with the primitive action error. The morefatigued an operator, the lower his/her performance leveland the greater the likelihood of errors occurring.

2.4 Within-task Fatigue

Fatigue can also develop as a function of the numberof times that an operator repeats a task (15), The fatigueaccumulated during the execution of a task, or within taskfatigue, is modeled as a function of the number of times astep is repeated. Table 2 shows a fragment of the table usedto compute the effects of in task fatigue. This tabledemonstrates the implementation of the fatigue-basedmodel of degradation in MIDAS. This table outlines the‘from count’, the ‘to count’, the error probability, the timepenalty and the quality penalty.

Table 2. Fragment of the Within-Task Fatigue Table.

When a new action is instantiated, the number ofprevious instances executed is retrieved and the errorprobability, time penalty, and quality penalty values arelooked up from the appropriate row in the fatigue table.The within-task values listed in the table are placeholdersfor more accurate numbers grounded in research. The “fromcount” and “to count” in Table 2 is a hypothetical point inthe model that outlines the time in the simulation when theMIDAS will perform according to a specific errorprobability (percentage). In the first row of the table, it canbe seen that from 0 to 99 ticks in the simulation, theoperator will engage in behavior with a 1% likelihood oferroneous performance. When the error occurs, the timepenalty associated with this is 150% (i.e., the task willneed to be re-started). The hypothetical quality constructindicates that during periods of low error probability, therewill be a very low likelihood of a quality penalty, whileduring periods of higher error probability, there will be ahigher likelihood of a quality penalty.

The process of integrating the PIFs into MIDAS hasalso fostered the need to collect data on error likelihood.To collect data an Error Summation and PresentationFunction was created that aggregates the total errorpredicted by the Pre-task and Within-Task fatigue modelsfor a given simulation clock tick.

2 . 5 Error Summation and PresentationFunction

In calculating the error summation and presentationfunctions, MIDAS calculates two sets of performancevalues each time an action is executed; one set for Pre-taskfatigue (1) that operates as a function of the operator’sperformance level and a second set for the Within-taskfatigue (2) that operates as a function of the number oftimes that actions of this class have been repeated. Thesenumbers are defined mathematically in MIDAS as:

Action FromCount

ToCount

ErrorProbability

TimePenalty

QualityPenalty

Fixate-alphanum-object 0 99 1 150 0Fixate-alphanum-object 100 ∞ 20 150 25Fixate-spatial-object 0 99 1 150 0Fixate-spatial-object 100 ∞ 20 150 25Fixate-symbol-object 0 99 1 150 0Fixate-symbol-object 100 ∞ 20 150 25

€

ProbabilityError(task)primitivetask,TimePenalty(task)primitivetask, (1)QualityPenalty(task)primitivetask

€

ProbabilityError(task) fatigue,TimePenalty(task) fatigue,QualityPenalty(task) fatigue (2)

The total error values for an action are calculated inMIDAS by taking the maximum value of the numbers ofthe probability of error, the time penalty and the qualitypenalty. The maximum value, as opposed to an averagevalue or a minimum value, was selected as the mostconservative value in determining when an errorvulnerability existed in the MIDAS performanceprediction. The formulas that were encoded in MIDAS canbe found in equations 3, 4, and 5.

€

ProbabilityError(task)total =max(ProbabilityErrorprimitivetask,ProbabilityError(task) fatigue ) (3)

€

TimePenalty(task)total =max(TimePenalty(task)primitivetask,TimePenalty(task) fatigue ) (4)

€

QualityPenalty(task)total =max(QualityPenalty(task)primitivetask,Quality(task) fatigue ) (5)

Several actions are typically executing at any givenclock tick in a simulation. The overall error value for aclock tick is calculated in MIDAS by taking the maximumof the total for each task. Given a simulation with N tasksrunning at a clock tick T, MIDAS will calculate and collectthe data according to the formulae in equations 6 and 7.

€

ProbabilityError(T) = Maxi=1

i=N(ProbabilityError(i)total ) (6)

€

ProbabilityPenalty(T) = Maxi=1

i=N(TimePenalty(i)total ) (7)

The top portion of Figure 1 shows the MIDASruntime display graph (visualization) of the probability oferror, time penalty, and quality penalty for each tick of thesimulation clock while the lower portion of Figure 1demonstrates the runtime prediction of operator workloadfor each of the six attention channels in MIDAS. Thesepredictions are generated from the cumulative summationof the loads associated with the activity that is called bythe MIDAS software in response to the environment.

Figure 1. Timeline output (notional) produced by MIDASintegrating the new error and fatigue models.

2.6 Discussion

It is important that integrated HPMs accuratelyaccount for the impact of relevant human conditions onhuman-system performance. Developing an error predictivecapability becomes particularly important in the aviationcommunity when attempting to predict operatorperformance in the face of advanced display designs or newrules of operation and procedural specification associatedwith passenger travel. HPMs that do not includeappropriate PIFs may produce different results than humanin the loop performance. Accepting the data output fromhuman performance models that do not account for theeffects of various performance influences increases the risksof selecting inappropriate technologies or developingunrealistic procedures.

Creating PIFs within the context of existing,validated HPM software tools enables predictions ofperformance when humans are operating near theirmaximum capacity as well as when they are more likely tomiss the onset of critical events (e.g., missing flight pathdeviation as time on task increases). Understanding whenthe human operator is most vulnerable permits thedevelopment and evaluation of mitigation strategies.Testing such advanced system concepts in the relativesafety of a HPM is both cost- and time-efficient and, whenused in concert with empirical research, is a system designconcept that is likely to achieve maximum humanperformance. Creating PIFs also enables developing

technologies and methods to augment the human operator’svigilance and attention to critical events during vulnerableperiods. Adopting a system perspective enables thedevelopment of mitigation strategies based on performancepredictions from a human performance modelingenvironment.

The Yerkes-Dodson theoretical threshold-modelthat was implemented in the current modeling effort isconsistent with the MIDAS representation of operatorloads. It augments the capabilities of MIDAS with avalidated model of the impact of arousal (as estimated bythe overall level of workload computed within MIDAS) onoperator performance. The MIDAS model displays errorvulnerability information based on error likelihood given anoperator’s predicted workloads. This capability allows aprocedure designer to create and refine procedures based onpredictions generated by the performance of a simulatedoperator in conditions where performance is suboptimal orwhere performance vulnerabilities are predicted.

It is important to highlight that the PIF modelthat has been developed as part of the current effort has notbeen rigorously validated against empirical data; the modelsfunction as designed but may need fine-tuning. The dataprogrammed into the MIDAS primitive-based model oferror are valid human performance data that wereempirically collected and tied to the primitive behaviorlevel within MIDAS. At this time, the fatigue model isstill very simple, but offers “hooks” to insert a more refinedmodel.

2.7 Limitations and Directions for FutureResearch on PIF Implementation

The development of the fatigue conceptualization ascompleted in the current effort is limited as it considersfatigue within each action type separately. It does notmodel fatigue across action types. It is anticipated that areservoir approach, or a two- or three-stage model of theimpact of fatigue on operator performance is warrantedgiven that the current “proof of concept” has shown that themodel is responsive to PIFs such as fatigue and humanerror.

3 ConclusionsThe effort undertaken in the current project

culminated in an agent-based model (e.g. slots) that can beused by a variety of models to predict and recreate short-and long-term effects of stressors (fatigue, stress, timepressure, inadequate situation awareness, workloadexceedances, etc) on performance in creating a situation inwhich incidents or accidents would be more likely. ThePIF model demonstrated performance influences brought totask performance by fatigue (in a pre-task fatiguealgorithm), by fatigue that is incurred while undertakingactivities required to complete a goal behavior, and theimpact of these PIFs on human performance output bycombining this with a primitive based action errorvulnerability. Further research is required to validate the

hooks that have been implemented in the MIDAS PIFrepresentation.

References[1] B.F. Gore, “The study of distributed cognition in freeflight: A human performance modeling tool structuralcomparison,” Proceedings of the Third Annual SAEInternational Conference and Exposition - Digital HumanModeling for Design and Engineering, Dearborn,Michigan, Paper # 2000-01-2181, pp. 1-6, June 2000.

[2] B.F. Gore and J.D. Smith, “Risk assessment andhuman performance modeling: the need for an integratedapproach,” in K.A. Malek (ed.) International journal ofhuman factors of modeling and simulation, Vol. 1(1),ISSN (Online): 1742-5557 - ISSN (Paper): 1742-5549, (inpress).

[3] B.F. Gore, “Human performance cognitive-behavioralmodeling: A benefit for occupational safety,” in B. Chase& W. Karwowski (Eds.), International journal ofoccupational safety and ergonomics (JOSE), Vol. 8, No.3, pp. 339-351, 2002.

[4] C.D. Wickens, “The multiple resource model ofhuman performance: implications for display design” InProceedings of AGARD/NATO Report, Williamsburg,VA, AGARD/NATO, 1984.

[5] J.H. McCracken and T.B Aldrich, “Analysis ofselected LHX mission functions: Implications for operatorworkload and system automation goals,” Technical noteASI 479-024-84(b), Anacapa Sciences, Inc., Fort Rucker,AL, 1984.

[6] M.M. Mallis, S. Mejdal, T.T. Nguyen3, and D.T.Dinges, “Summary of the key features of sevenbiomathematical models of human fatigue andperformance,” Aviation, space, and environmentalmedicine, Vol. 75, No. 3, Sect. 2, pp. A4-A14, March2004.

[7] D.A. Wiegmann and S.A. Shappell, “Applying thehuman factors analysis and classification system (hfacs) tothe analysis of commercial aviation accident data,” 11th

International Symposium on Aviation Psychology,Columbus, OH, The Ohio State University, 2001.

[8] R.M. Yerkes and J.D. Dodson, “The relation ofstrength of stimuli to rapidity of habit-information,”Journal of comparative neurology and psychology, Vol 18,pp. 459-482, 1908.

[9] C.D. Wickens and J.M. Flach, “Informationprocessing”, in E.L. Wiener & D.C. Nagel (Eds.), Humanfactors in aviation, pp. 111-155, Academic Press, SanDiego, 1988. 3 Posthumously.

[10] K.R. Boff and J.E. Lincoln, Engineering datacompendium: human perception and performance, JohnWiley & Sons, NY: NY, 1988.

[11] E. Hollnagel, Cognitive reliability and error analysismethod, Elsevier Science, NY, 1998.

[12] S.G. Hart, “Pilot workload coping strategies”,AIAA/NASA/FAA/HFS Conference on Challenges inAviation Human Factors: The National Plan, Tyson’sCorner, VA., 1991.

[13] S.R Hursh, D.P. Redmond, M.L. Johnson, D.R.Thorne, G. Belenky, T.J. Balkin, W.F. Storm, J.C.Miller, and D.R. Eddy, “Fatigue model for appliedresearch in warfighting,” Aviation, space, andenvironmental medicine, Vol. 75, No. 3, Sect. 2, pp. A44-A54, March 2004.

[14] A.A. Borbély, “A two process model of sleepregulation,” Human neurobiology, Vol. 1, pp. 195-204,1982.

[15] S.R. Rodgers, “Work physiology – fatigue andrecovery,” In G. Salvendy (ed.) Handbook of humanfactors and ergonomics, 2nd Edition, pp. 269-297, 1997.

4 AcknowledgementsThe research presented herein is the result of NASA

Office of Biological and Physical Research funding sourceUPN 107-07-07. The technical monitor of NASA AmesResearch Center Grant # NCC2-1302 is Dr. JessicaNowinski. The authors would like to thank K. MichaelDalal from QSS Group, for his programming assistance onthis task and for all reviewers for their insightful feedback.Sandra Hart and Faith Chandler were instrumental inidentifying various opportunities within the Spaceapplication domain for the current MIDAS applications andin promoting the refinement of the MIDAS tool.