DOCUMENT RESUME 235 Semple, Clarence A.; rd 0 the ... · PD 171 242. AUTHOR TITLE. TNSTITUTON....

PD 171 242

AUTHORTITLE

TNSTITUTON

SPANS AGENCYREPORT g0PUB DATECONTRACTNOTF

FARS PRICFCESCRIPTOFS

DOCUMENT RESUME

tsE 235

Semple, Clarence A.; rd 0 theFtinctional Design of ar Autcma ed InstructionalSupport System for Operational Flight Trainers,Report, June 1976 through Seertember 1977.Canyon Research lroup, Inc., Westialie Village,Calif.Naval Training Elquipmrt Center, Orlaric,NAVTRATNUTFO:N-76-C-0096-1Jan 79N61339-76-1-Q096119p.

nal

MF01/PC05 Plus Postag=.*Automation; Fdacaticral Fla rIlng; *R1-ght Training;Tnstructional Ai as; *Thstractional Systems; *MilitaryT'-aining; *Simulators; Trairirg Techniques

ABSTRACTFn-ctional requirements for a highly automated,

flexible, instructional support systal for aircrew trainingsimulators are presented. Automat(ad support modes and associatedfeatures ana capabilities are desoribod, along with hardware andsoftware func4.ional requirements for implementing a baseline sysin an operational flight training context. The importance of aneffective man-machine interface is .1 soused as it relates toi nskructor acceptance and S1/6m utility. (Author/JVF)

Reproductions supplied by ERRS are the pest that can be madefrom the original dccuaent.

*********** ************************w************

is

1./ S OEPART'MRNTCiF HEALTH.tpuCAT 10,4 L WELPARNATIONAL NSTIloTE og

Ti os oocuNIENT HAS Decry REPRowee() EXACTLY AS RECEIV ED FROMTHE PERSON oR ORGAN? ZA TION ON/GIN.ATINIc IT POINTS OF vigW oR OPINIONSSrAftc) DO Nor NECESSARILY REPRE-SENT oFF iCiAL NATIoNaLiNsTiTuTE pFtOuchitoN POSITION OR POLICY

nchnical Report t VT EQUIPC 76-C- 96-1

FUNCTIO DESIGN OF AN AUTOMATED INSTRUCTIONIAL

S1JPFORT FOR OFEJRATIONAL FLIGHT TRAINERS

Semple, Donald Vr uls, John C, C ttonJ.Ttel Hooks, E wsrd A. flutter

Canyon R search Croup, fee.741 Lakefield Roth, Suite BWestlake Village, California 91361

Report June 1976 - September 1977

usty L979

TRAINING EQUIP NT CENTERA, FLORIDA 32813

*

SECUFIITV CL.ASSI PI CATION OF THIS PAGE (Kell Dein Filtered)

REPORT DOCUMENTATION PAGE_T NUM, R

NAVTRANUIPCEN 76=C-0096-1TITLE nd SON(10FUNCTIONAL DESIGN OF AN AUTOMATEDINSTRUCTIONAL SUPPORT SYSTEM FOROPERATIONAL PLIGHT TRAINERS

GOVT AC

READ i TIDEPORE CO pl..ETINC

IQN NO. 3REcirai NTs CATALOG NO

7. AU T

Clarence A. Semple, Donald Vreuls, JohnC. Cotton, D. R. Durfee, J. Thel Rooks,Edward A. I utter

FTGANIZATION NAME AND Ao0REss

Canyon Research Group, Inc.741 Lakefield Road, Suite BWestlake Village, California 91

il. cos.)rAOL.LIN(IOFfICE NAME ANO ACHO

5. TYPE OF REPORT A PER104 COVERED

Final ReportJune 1976 - Sep 1977

6. PERFORFAING c7RG. REPORT Numn

NTRACT OR GRANT NU

N61339-76-C-0096

PROGRAM ELEMENT, PROJECT, TOAREA & W O R R U N I T N I I M N E F I S

NAVTRAEQUIPCENTask No, 2753-5P1

7a1 Training Equipment CenterLando, Florida 32813

ERORT DATEJanuary 1979

ONITORING AGENCY 0 RES Ctlarroll

t71 T 47i11JTIOF.T9TAT T lot rKl. Report)

ER

116SECURITY CL A3,

ES

Unclassified

F

150. DECL-ASSIFICATION/ DOWNORACHNqSCHEDULE

Approved for public release; distribut n unlimited .

nISTNIRLI TtgrJ STATE no Aho I k 20, I d ow from

PLEFAENTAFV NOTES

9 ?ter woR os (continue un ee)ereo .Nile ery end ldarillly by

Automated Instructor SupportAdaptive TrainingComputer Resident SyllabusModular Flight- training Software

0 AOSTRACT n on rava. aide 11 n end ldnllly by block nun

Functional requirements for a highly automated, flexible,instructional support system for aircrew training simulatorsare presented. Automated support modes and associated featuresand capabilities are described. Hardware and software functionalrequirenents for implementing a baseline system in an operationalMight training context are presented. The importance of arleffective man-machine interface is discussed as it relates

tructcr fceptance and system utility.

D F"O I JAN 14 EDITION OF t NOV AS is ossoLETE UNCLASSIFIEDSECURITY CLA IFICATION OF THIS RAGE (tthet, atom Filtered)

MOT _WUIPCgli 77-C-0096-1

FOREWORD

This report ocuments the plaan g stage of an effort toextend automa on for flying training to support for the flightinstructor. edback is necessary for such development, and theoperational training of fleet replacement pilots for the f' -14Aaircraft at NAS Miramar will be the context for the evaluationof software designed for instructor support. Logicon, Inc. iscurrently producing the described system and the first stage ofimplementation should be installed on Device 2P95 In February1980.

Gilbert L, RicardScientific Officer

1/2

NA UI C N 76-C-0096-1

PREFACE

The authors to acknowledge the assistance and coopera-tion of the Navy personnel who directly. contributed to the study.The success of any study of this type hinges strongly on theeftorts of personnel at both the program direction and operation-al levels who devote time and effort to provide guidance andinformation required. In particular, the authors wish to ackno

ledge the efforts of the following:

1r. ira Goldstein, who served as the NavalmTraining EquipmentCenter Technical Representative for the study and was instru-mental in providing both short term and long term directionfor the I5S development, including the identification offuthex automated instructional research.

Mr. James BolwerX, who served as Commander Naval Air Force,U. S. Pacific Fleet liaison and provided guidance regardingoperational training factors to be considered for the struc-

turing of instructional support capabilities.

LCDR William Gilchrist, UCDR George Cagell, and Mr. VictorSreddell, Jr. of Fleet Aviation Specialized OperationalTraining Group, Pacific Fleet,,whose cooperation facilitatedthe observation of instruction on Device 2F95 and the inter-viewing of training device personnel at VF-124, NAS Miramar

California.LT Scott bauney, LT Jeffrey Punches and LCDR John Snyder,together With other Instructional Systems Development per-sonnel and instructors at VF-124 and the authors gratefullyacknowledge these individuals for making available theirscarce time to answer the myriads of questions asked of them.

3/4

Sec ion

NAVTRAEQUIPCEN 76 -C- -009

INTRODUCTION

TABLE OF CONTENTS

pa e

9

Overview of the Study . . . . . . . ..

9

The Automated Instructional SupportSystem Setting.

10

Summary . . . . . . . .. . . . . .

11

Il APPROACH TO THE PROBLEM .S

Issues a,nd Considerations15

Method . . aE

16

THE TRAINING ENVIRONMENT FOR ININSTRUCTIONAL

SUPPORT SYSTEMS .

Overview . . .Simulatc,r Training ApplicationsOFT UsageRP and IP AssignmentsIUT Training . .Instructional Functions

23

232324

. .. .

25

. ..

25

4 . .26

IV INSTRUCTIONAL SUPPORTSYSTEM FEATURES AND

CHARACTERISTICS . . . .....29

Overview @ . . . . ... . . . . ..

29

Objectives . . . . . . . . . .. . .

.29

TasX Modules . . . . . . ..

31

ISS Modes . . . .34

Support Features40

Summary . . . . .. . . 54V ORIENTATION TO OPERATING

WITH AN

SUPPORT SYSTEM. . . . .

The Add-On Concept . .

Operational Features

INSTRUCTIONAL0

56

5656

Operating with the ISS . .63

VI SYSTEM DESCRIPTION65

Overview . ..65

Man-Nechine Interface .65

iardware .76

:vice 2F95 Interface .78

Software Description .79

Data Base Description .@

Si

Data Retrieval and Analysis ..

83

Data File Houskeeping .. .. .84

5 .

Section

VII

NAVTRAEQUIPCEN 76-C-0096-1

TABLE OP C NTENTS (Continued)

?Age

IMPLEMENTATION . .... 85

Overview . . a65

Stage I . , *0 85Stage II . . .. . 00 07Stage III . .6.06 6.0.0900 68

VIII

Hardware Staging Considerations

LIFE CYCLE SUPPORT COI\ISIDERATION

Maintenance ConsiderationStaffing Considerations .0

90

9090

IX SOFTWARE SUMMARY . 6 92Introduction . . .. .

92

Information Flow . 04.0.0..6 92Module Summary . .

92

USER TRAINING97..

Considerations97

Initial Training . .97

Formal Training97

BILIOGRAPHY99

APPENDIX A (Listing of Nc rttal Procedureand Flight Modules). . .

3

APPENDIX B (Listing of Failure (Emergency)107

Task Modules) . . . .. . . . . .

APPENDIX C (Examples of expanded FlightProfile Modules) s 115

Figure

1


LIST Ot" ILLUSTRATIONS

Flight Training Simulator Automated Instruc-tional Support System Functional Flow..

2 Methodological Framework for Complete ISSDevelopment and Validateion

Phase I - Analysis of Requirements and

Technologies

4 Phase II - Development of a SpecificConcept . . .

6

7

9

10

11

12

13

14

15

16

Phase III System Design

Summary of ISS Functional Capabilitiesand Modes . . . . .

F-14 OPT Training Area with 159 Equipment

Basic Information Selection Display

Parameter Selection Display

Procedures Monitoring Display .

Graphic Display Showing a Final Approach.

Side View of ISS D play Group.

158 in Relation to a Typical OFTInstructor's Console

Basic I-_formation Directory .

First gunplay of Log on Sequence

Graphic Display Exampe Showing FinalApproach Format . . . . .

17 Parameter Selection Display .

18 Procedures Monitoring Display

19 ISS Hardware Organization Assuming aTwo-Stage Implementation .

20 IS5 /2F95 Interface

7

Page

12

17

18

20

22

55

57

59

60

61

62

66

67

69

70

71

72

74

75

80

NAVT tAEQUIPCEN 76-c-0096-1

12RE

LIST OF ILLUMATIONS (Continued

12_42a9_

21 ISS Sof_ w.r ra_chy82

22 Candidate Staging of 155implement- ion 86

23 ISS Data Flow for Mission Condu't93

1 Mission Conduct Software Module Description94


SECTION I

INTRODUCTION

OVERVIEW OF STUDY

This report is the result of a study that:

o Took a global look at Instructional Support Sys e(ISS), then

o Created a functional ISS design for a specific flitraining simulator (F14A), and

o Tailored the design to fit a realistic budget andschedule.

The need for the study came about by the requirement tominimize the use of advanced Fleet aircraft for training andthe need to maintain and preferably improve the quality ofaircrew training. In spite of the minimized use of aircraftfor training purposes, the current high cost and limitedavailability of advanced Fleet aircraft for training hasgreatly increased the Navy's need to rely on training simulatorsto reduce training costs without sacrificing operationalreadiness. This then requires more use of flight simulators insuch a manner that the flight training standards are achievedin a cost effective manner. These objectives can be realizedby the continued application of the instructional systemdevelopment approach and the enhancement of the instructionalcapabilities of present future training simulators.

The technology of automated instructional support offersthe potential for reducing the instructor's workload whileproviding a means for further improving the efficiency andquality of training with simulators. In particular, the ISScan be expected to:

Standardize training

o Measure performance objectively

o Adapt the scheduling of the training tasks

Advance the student through the curriculum accordingto proficiency

Motivate the student

Aid instructional management

This can be achieved by a system design which:

9

NAVTI -QUIPCEN 76-C-0096-1

o Provides the Instructor Pisupports that can relievetional tasks.

ots (IP) with automatedhem of ancillary instruc-

o Provides automated auxiliary support by incorporatingsupport capabilities typically not found in trainingsimulator settings.

o Provides an Instructional Support System (ISS ) thatwill facilitate ongoing refinJ tg and testing.

'TIE AUTOMATED INSTRUCTIONAL SUPPORT SYSTEM SETTING

In any instructional setting, the instructor's primaryduty is to teach, i.e., to transmit knowledge and guide thestudent's learning. Flight simulators offer the potential forvery effective instruction because of the control that can beexercised over the training situation. However, instructortasks that are ancillary or secondary to teaching often reducethe efficient and effective use of flight simulators. It hasbeen estimated that approXimately 20 percent of the IP's timeduring each simulator training session must be devoted to ancillarytasks. Automated instructional support can reduce this time byproviding assistance in the performance of such antillar tasks as:

o System initialization

o Problem set up

o Note taking

o Acting as missing crew

o Mission communications

er

Automated instructional support technology also offers thepotential for enhancing the quality and for standardization ofsimulator training by providing IP's and training managementpersonnel with auxiliary capabilities not previously available.Auxiliary task support can include:

Computer-resident syllabus to facilitate planning andstandardization

o Computer-generated pre - session briefing aids

o Automated Replacement Pilot (RP) readiness testing

o Automated performance measurement and scoring

Data files for objectively establishing quantitativeperformance norms

10

NAVT1- 1JIPCEN 76-C-0096-1

o Automated prescription of subsequent training

o Automated maintenance of student progress files

Automated performance summaries for debriefing

The general functional flow of an Automated InstructionalSupport System that would accomplish the foregoing purposes isshown in Figure 1.

Projects leading toward the development of effectiveinstructional support systems have been sponsored by the NavalTraining Equipment Center for some time. Early work establishedthe feasibility of automating a syllabus for aircraft weaponsystem training (Leonard, Doe and Hofer, 1970; and Futas,Butler and Johnson, 1972). The concepts were demonstrated inthe laboratory (Charles and Johnson, 1971; Charles, Johnsonand Swink, 1972 and 1973), and later implemented in the fieldat Luke Air Force Base (Swink etal., 1975), and for the GreekAir Force (Butler, Langford and Futas, 1975; and Butler,Barber and Futas, 1975). Parallel automated performancemeasurement work was performed for the Navy during the sametime frame (Vreuls and Obermayer, 1971; Vreuls, Obermayer andGoldstein, 1976; and Wooldridge, Breaux and weinman, 1976) .

Successful demonstrations also have been made of thetechnical feasibility of computer automated problem set up,performance measurement and control of the next exercise forselected segments of instrument flight training, GroundControlled Approach (GCA) and TACAN approaches, Air-to-AirIntercept (AAI), and Ground Attack Radar (GAR). Automation ofproblem briefing, air and ground controller voice messages,problem dynamic replay, and limited instructional coaching alsohave been achieved. Computerized Speech Understanding Systems(SUS) have been installed in the laboratory and at an opera-tional flying training establishment (Fuege and Grady, 1975) .

These demonstrations show that it is possible to useexisting hardware and software technology to automate variousinstructor functions. Initial applications have been shown tooffer training benefits (Puig and Gill, 1975; and Brown,,Waagand 12ddowes, 1975) .

S do/A 11 Y

based on the current state of ISS technology this studyachieved the following objectives:

Developed an ISS structure and concept that has generalaircrew training application to any aircraft type yetis flexible enough to be programmed to meet a specificFleet requirement.

1 1

Cirriculum

Control

Training

Task

Selection

Automatic Adaptation

Housekeeping

Scheduling

Presession

Support

Training

Session

Support

Post

session

Support

Figure 1. Flight Training Simulator Automated Instructional

Support Systwn Functional now

Ltj

0

H

00

NAVTR EOUIPCEN 76-C-0096-1

Developed a function design for an automated ISS thatcould be "strapped onto" an existing P414A OperationalFlight Trainer (OPT) (Device 2Y95) which will enhancepilot training by more effective use of the IP.

The following definitions amplify program objectives andthe type of ]SS design that was pursued:

o Instructional _ is providing computer assistance,through automation, r the performance of ancillaryand auxiliary instructional tasks.

Anl-)_aryLj_Ictionalcs are those routineactivities that are required simply to iise trainingdevices in present configuration.

Auxiiiar ort is providing instructors and

bemanagement with aids and information that can

be used to more objectively and comprehensively ensuretraining quality.

Strat

NAVTFAZQ ZPQ 76-C-0096-1

An oper ional descshow its compatibilisimulators and train

m of the proposed system toth existing training

requirements,

o An outline of the hardware and software design whichis the basis for subsequent system specifications.

An implementatiaz and administration pla,1 by which theI55 can be broilght into operational use.

Because of the vide ranging disciplines involved in thisstudy it has been dif ficuit and at times impossible to qualifyand record the rationale far the numerous decisions thatprovided its directioh. To say the least,much good,judgaentand experience prevailed, each involved.nvolved realizingthat the resultant ISS will require significant flexibility toaccommodate tuning, medi ieation and validation of the basicdesign.

14

NAVTRAEQUIPCEN 76-C- 096-1

SECTION II

APP1OAC3 TO THE PROBLEM

ISSUES AND S CONSIDERATIONS

The approach was intended to ensure that the ISS functional

design would be responsive to the needs of both operational

training and research. Key elements considered were:

Instructional support needs, as defined from anoperational training reference point, wouldlargely provide the guidance for determining thebandwidth for automated support capabilities tobe incorporated in the ISS functional design.

o The 15$ design must capitalize an the strengthsand compensate for the weaknesses of presentinstructional support system technologies, in-

cluding instructional techniques, hardware and

software. Therefore, the design must incorporatecapabilities that enable it to be used in aresearch role as well as an instructional roleto optimize the capitalization/compensationrationale.

Support capabilities incorporated into ISSobjectives should not be limited by a-priori judg-

ments of implementation difficulties and constraints.Rather, a full support system should be conceivedand then any reasonable subset of capabilitiescould then be selected for implementation.

o ISS hardware and software designs should be basedon state of the art technologies to ensure aneffimienf- ant compact system. Technology that"advanced" state of the art could be used ifdevelopmental risks are modest.

The ISS design should complement, rather thanduplicate, instructional features of the hostsimulator's instructors console.

o System capabilities, as expressed through the man-machine interface, must have user acceptance andrequire minimal instructor training.

o The design must acknowledge that pilot trainingand the use of a particular flight simulator toaccomplish the training are dynamic and changing.Therefore, certain Instructional support capabilitiesMust be modifiable with relative ease.

IS

NAV QUlPCEN 76-C-009

o The designdes must be generalizable to simulatortraining applications other than those characterizinga particular instructional setting or device.

The resulting system design, when Implemented mustbe transparent to the host simulator, thus not inter-fering with or requiring modification to the simulator'ssoftware.

KVTHOD

Recognizing that ning such a system was highly innova-,

tive, research necessitated venturing beyond what has been triedand proven. The methodology used is shown in simplified form inFigure 2. In the absence of implementation and operationalfeedback, the designs process used was largely open loop and basedupon best estimates, As implied in Figure 2, feedback ultimatelywill be required to optimize features of the system, determinetheir utility and acceptance, and estaL41i4h guidelines forgeneralizing the design features to other applications.

The approach to the problem was divided into three pha

Phas

Phase II

Phase III,

Analysis of Requ-

Pevelopment of a

System Design

ants and technolog

-oifio Concept

PRASE I --AW.LYSIS- OP ITIQUIR2MEi- Figure 3 shows

the Phase I tasks. Activities in the phase centered upon d t r-mining operational instructional support regairemenLq. Faralactivities examined instructional techneaogy that could be broughtto bear on the problem, and hardware and software constraintsthat could affect TSR (qPign

Fleet Readiness Squadron (FRS VP -124, at NAS Miramar, CA.,used as the operational training environment for establishingort requirements. Device 2F95 (F-14A OFT) was establishedhe candidate host simulator for an 1SS iniplementation.

Initial activities centered uponanticipated uses of the On in the FRSat Miramar, the one selected incorporaThis visual System uses point light souvisual imagery which is limited to a refield of view.

consis 1

Both OFTs arele containingcated remotel

ablishing present -idilabus. Of the two OFTsa VITAL-2 visual system.

to display nightvely narrow forward

single seat simulators.various repeater instrumefrom the cockpit.

nstructor'sand annunciators

VRUINSTCTOR

FUNCTIONS,AUTOMATEDSUPPORTDEVEL.S

ri7

1UTPCE 76-C-009

FININGIREDS

iMPI:' NTATLON

TRAINING &DEVICE,TECHNOLOGY& RESEARCI

-- DEVELOPMENTAL PATH

_ical fame workISS Development and Valid

REQUIRED FEEDBACKPATHS

to

RESEAlicE

TECHNOLOGY


iNsTRoCTIONAL ___MkRDWARE/SoPTWARECHNOLGOI --1TECHNOLOGY

W2PLACENENT PILOTTRAINING ANALYSIS,VT -124, NAS MIRAMAR

REVIEWAUTOMATEDINSTRUCTIONAL.

SUPPoRTTECHNOLOGY

FY ANALYZE OFT

NCIL.IARY RARDWAE,INSTRUCTIONAL SOFT A.= AND

FUNCTIONS INTERFACECHARACTEPLISTIOS

SELECT CANDIDATEINSTRUCTIONALSUPPORT CONCEPTS

REv-r

IR and TO TRAINING

REVIEW HARDWAREAND

Milan TENNOLOGIESAND IDENTIFYDESIGN APPROACHES

INTEGRATE ANALYSISINFORMATION FORPREL(M1NARYDESIGN DECISIONS

ure 3. Phase I. - Analysis Of Requirements and Tech 1

18


Analysis of the OFT was done for Fleet Squadrons as well as

VP-124. Fleet personnel typically use the OFT electronicallylinked to Device 15C9, the Mission Trainer, for integrated aircrew

training and NATOPS checks. The linked mode enables training on

tactical problems. This is not possible in a stand-alone OFT.

Taken together, analysis of FRS and Fleet usage provided informa-

tion on a broad sample of weapon system training tasks to which

ISS design may ultimately have to be responsive.

Instructor interviews, Training Device Operator (TD) inter-

views, and training exercise observations were completed to

identify ancillary instructional functions and provide a focusfor prescribing meaningful automated instructional support.

This part of the analysis phase was concluded with a reviewof instructor Under Training (TUT) syllabus. materials and prac-

tices. This was done to establish the type and amount of trainingthat 'Ps typically receive on the use of the OFT in the FRS pro-

gram= This provided a baseline for estimating special trainingrequirements for the effective use of the ISS.

One parallel activity examined on-going and recently com-pleted instructional support research to assess the potentialutility of recent advances in:

Computer-resident syllabus structures

Automated performance measurement and scoring

o Instructional man-machine interfaces

o Automated adaptive training

The main purpose of the examJ-nation was to evaluate th

status of these technologies for field application. An adjunctpurpose was to examine and, hopefully, benefit from lessonslearned in the laboratory and from the field as they relate tosystem utility and system design for user acceptance.

A second parallel activity involved interviews withmaintenance personnel and a review of OFT hardware and softwaredocumentation. The primary objective was to establish the feasi-

bility of a strap-on, transparent interface. A second objective

was to establish the adequacy of the technical documentation ofthe OFT for use in later design work.

The culmination of Phase I was the integration of a.n. lysis

information to establish goals and bounds for Phase II.

PEASE II - DEVELOPMENT OF A SPECIFIC CONCEPT. The goalof Phase II was to draw upon available information and experience,

and develop an integrated ISS concept. The tasks involved in

this iterative and largely inventive process are shown in Figure4and were create& to

19

NA'VTWQUIPCEN 76-e-0096-1

ESTABLISHSYLLABUS IMPACTS

TRAININGOBJECTIVESINFORMATION

SYSTEMGLOBALCONCEPT

DEFINE ISSFUNCTIONALREQUIREMENTS

'SS INITIALREQUIREMENTS

INTERFACEHARDWARE

*SIMULATIONMODELS

SOFTWARE MODULES

DEVELOPTOP LEVELSYSTEMDESIGN

INSTRUCTIONALBENEFITS

NO

EXPECTED

YES

FINALIZE DESIGNCONCEPT

Figure 4. Phase II_ -Development ©f a Specific Concept

202


o Refine candidate automated support capabilitiesinto a set of easily understandable modes of operation.

o Evolve an operational system structure that organi-support capabilities hierarchically.

o Refine requirements of research as they relate to theanticipated needs of ISS development in the field.

Evaluate the requirements of research on hardware andsoftware design.

o Evaluate alternative man-machine interface designsin terms of system utility, user acceptance, andIUT training.

Determine alternative programming methods toaccommodate ISS software.

Evaluate alternative hardware configurations,considering hardware/software interdependenciesand relative costs.

o Weigh each concept against anticipated instructionalsupport benefits and developmental risk factors.

o Result in a "full - support" concept that eventually couldbe implemented.

The foregoing activities were not accomplished independentof those of Phase III. Indeed, several candidate designs hadto be examined to evaluate their effectivity as an 'SS.

PHASE III - SYSTEM DESIGN: Figure 5 summarizes the-

activites performed during the design phase. Activitiesin this phase centered upon translating operational system re-quirements into a functional system design which is the centralsubject of the remainder of this report.

Recommendation for the IUT syllabus were also finalizedduring Phase III. These recommendations could not be finalizedearlier because the relative merits of system concepts incorporatingresident computer assisted instruction and operator job perfor-mance aids had to be resolved first.

21


Following development of hardware andsoftware requirements,

a phasedimplementation plan was developed.The strategy, pre-

sented later, was developed to provide ameans of obtaining user

feedback reasonably early whilesimultaneously providing a

realistic schedule for the productionof the more complex soft-

ware modules. Early feedback wasfelt to be of particular impor

tance since many of the featuresof the ISS have not been subjected

to the realities of operational training

FINALIZE IUTSYLLABUSRECOMMENDATIONS

DEVELOPHARDWARE/SOFTWAREFUNCT/ONALREQUIREMENTS

DEVELOP AMINIMIJM RISKIMPLEMENTATION

LAN

Figure 5. Phase III¢System Design

22

NAVTRAEQUIPCEN 76-O-0096-1

SECTION III

THE TRAINING ENVIRONMENT FORINSTRUCTIONAL SUPPORT SYSTEMS

OVERVIEW

The purpose cf this section is

o To briefly describe the instructional environmentused as the baseline for the analysis of therequirements .for ISS.

Provide an understanding of several specific featuresof the ISS design which were dictated by the assumptionthat the prototype field implementation would be in thesame training environment as that analyzed.

FRS VF -124, NAS Miramar, California, provided the trainingprogram analyzed. Replacement Pilots (RP) and Replacement NavalFlight Officer (NFO) training at VF-124 is intended to providethe skills, knowledges and attitudes necessary for an F14A Fleetassignment where aircrew members must perform proficiently inall aspects of aircraft operation, weapon utilization and missionaccomplishment. Training toward these goals is provided througha 26-week course of instruction involving academic, simulatorand inflight training.

SIMULATOR TRAININING APPLICATIONS

RP simulator training at VF-124 focuses almost exclusivelyon the use of the OFT. A procedures trainer was not available.The aircrew mission System Trainer (Device 2F112) had notbeen installed during the analysis timeframe.

The simulator portion of the RP syllabus has three purposeswhich were addressed in a minimum of eleven structured exercises.The content of each exercise is largely described in theInstructor's Briefing Guide document. Each exercise lasts -fr.one to one and one-half hours. The three purposes are discussedbelow.

FLIGHT FAMILIARI ATION involved performanceof normal procedures, takeoff and Standard instrument Departures(STD's) from NAS Miramar. Simulated flight can be continued toAn over-the-water controlled airspace, Whiskey -291_ Theaircraft (simulator) can then be flown and maneuvered in thewarning area to enable the RP to learn aircraft responses,control techniques associated with the F14A, the use of cockpitcontrols and displays unique to the P14A, and to practiceresponses to selected failures/emergencies. Upon completion oftraining within the dedicated area, the aircraft (simulator) isflown to a marshal point, and thence to NAS Miramar, where

23

NAVTRAE P EN 76-C-0096-1

various instrument approaches, final approaches, missed-approachesand landings are practiced. Prescribed voice procedures are re-

quired.

AIRWAYS FLIGHT. This involvesnormal procedures,

takeoffs andSidardIriStrument departures from NAS Miramar.Flight is continued on various legs of specially designated

air-

ways training routes (India routes). Instrument approaches andmissed approaches are performed at intermediate Air Force Bases.Fuel planning is emphasized and upon ccpletion of the last air-ways leg, various instrument approaches, final approaches, missedapproaches and landings are performed at NAS Miramar. A bingofuel profile flight to MCAS Yuma can also be simulated. Thistraining further enables the RP to learn control techniques uniqueto the PI4A, the use of F14A cockpit controls and displays, andto practice responses to selected failures/emergencies. Pre-scribed voice procedures are required.

CARRIER OPERATIONS. This training centers uponlaunch, departure, flight to a Marshal point, holding pattern,instrument approach, various final approaches, missed approaches

and recovery. Normal procedures and carrier operations procedures

are incorporated. Prescribed voice procedures alsoare re-

quired. Failures/emergencies are not incorporated.

The OFT is also used for fleet defense. This involvedintegrated aircrew team performance, standoff missile and short

range missile launches, and tactical and electronic responses to

various threats. Prescribed tactical communications are re-

quired. This application is almost exclusively for Fleet air-

crew refresher training and NATOPS checks, and requires the OFT

to be electronically linked with Device 15C9, the rear-seatMission Trainer. The VF-124 syllabus does not require linked

mode Fleet defense training. However, since this is likely tochange with the introduction of Device 2F112, the need to incor-

porate aircrew training exercises into the ISS was taken into

account.

OFT USAGE

Each of the two OFTs at NAS iramar is normally scheduled

to be available approximately 50 hours per week for training.A review of six weeks of OFT usage showed a relatively high

overall utilization. Significantly:

o 23 percent of OFT simulator sessions were logged as

scheduled RP training

o 17 percent of OFT sessions .ere logged as Fleet use

and other rniscellaneou

55 percent were loggedRPs use the OFT outside

es

"extra training," whereinthe normal syllabus.

24


These findings were of particular importance in terms ofincorporating ISS features that could enhance quality and stan-dardization of instruction while making maximum productive useof "extra training" time. Typically, an IP is not presentduring extra training; however, with the assistance of a train-ing device-man (TD), the RP can practice flight control, navigaLion, fuel management, some communication procedures, mostnormal procedures and a plethora of emergency procedures withno formal instruction, and with performance feedback derivedalmost solely from cockpit cues. It was felt that a "full support"ISS should be capable, to some extent, of compensating for theabsence of an IP by providing the RP and TD with information andsystem control capabilities that would make possible efficientuse of the extra time sessions.

RP AND IF ASSTGNMENTS

As in most FRSs, a particular student is not assigned to aparticular instructor, even for sub-phases of instruction. Thisreflected the instructional philosophy of exposing students toboth varying instructional styles and different approaches toproblem solving.

One consequence of this philosophy is that a heavy adminis-trative burden is placed on the use of simulation grade sheetsto ensure continuity of training in all required areas. Althoughinadvertent, it occasionally happens that continuity of instructionbreaks down and an RP does not receive prescribed training orachieve acceptable levels of performance on all objectives. Thisaspect of the instructional context is not unique to VF-124. Itwas felt that a full support ISS could provide useful adminis-trative assistance to overcome this problem.

IUT TRAINING

VF-124 is not unique in that IPs generally lack training inboth simulator operation and instructional methods (Charles,Willard and Healy, 1975). The training analysis performed as apart of this study showed the following. Instructors UnderTraining (IUT) receive rather informal training in use of theOFT, consisting largely of flying the simulator missions severaltimes, followed by on-the-job training in console operation. Anexpanded IUT syllabus recently was developed at VF-124 but, todate of this document, has not been formally implemented. Evenwhen implemented, it can be expected that heavy instructor work-loads will litit expansion and formalization of IUT training.

The latter point is important in that it directly affectsthe conceptualization of the ISS. The assumption is that it isnot realistic to count on IP training to compensate for designinadequacies of the ISS. On the contrary, an operational ISSmust be designed to maximize ease and orderliness of operationif it is to be used effectively by the IP.

25

NAVTRAEQUIPCEN 76-C- 096-1

INSTRUCTIONAL FUNCTIO S

A significant aspect of the analysis was to establish IPand TD ancillary and auxiliary instructional functions, and todetermine which of these would be candidates for automated support.The following listing summarizes the identifiable functions per-formed by the IP during training of an RP on the simulator. Anasterisk appears by those determined to be candidates for automa-tion.

*Review and evaluate the RP's progress to date.

*Decide upon training content of the simulator trainingexercise to be undertaken.

*Present a pre-session briefing coveting the instructionalobjectives to be met and the mission plan to be used asthe training medium.

*Interrogate the RP on flight control, system and opera-tional knowledges required to enable him to benefitfrom the simulator exercise.

*Provide instruction in areas of RP pre - exercise knowledgeweaknesses.

Provide over the shoulder instruction on cockpit controls,displays, an procedures.

Perform plane captain's role by giving hand and arm signalsto RP during performance of engine start and post-startsystem checks.

*Perform missing crewmember role by reading NFO checklistsand monitoring RP responses.

Perform missing (NFO) crewmember role by participatingin problem diagnosis following system failures.

*Provide training problem control in keeping with contentof the instructor's Briefing Guide.

*Adjust training problem content inobserved performance.

Communicate system, procedural andto the RP.

keeping with RP's

operational knowledge

*Provide coaching, cueing and performance feedback to theRP.

*Insert and/or remove, or command TD to insert and/orremove system failures.

26

NAVTRAEQUIPCEN 7 6- 0-0096 -1

*Vector the RP within a simulated warning area.

Interrogate the RP on system knowledge, procedures, causesand consequences of failures, flight operations, communi7cation procedures, aircraft operations and operating limits,and NFO tasks during the training exercise

*Take notes for performance evaluation, grading/scoring,learning problem diagnosis, and post-exercise debriefingof the RP.

*Sample and evaluate the RP's performance in the areassystem knowledge, normal procedures, emergency procedures,flight control, navigation, flight operations and voicecommunication procedures.

*Complete and annotate grade sheets.

*Perform the following com unications:

Mission communicationsan Diego departure controlSan Diego approach controlAirport Terminal Information Service (ATIS)Beaver (search and rescue) controlNA S MiramarClearance deliveryGround controlTowerApproach controlGCA controllerMissed approach controller

Intermediate landing sitesLocal area approach controlLocal area departure controlLocal area missed approach controllerLocal area GCA controller

Los Angeles Center (appropriate FAA sector controlleCarrier

Cari.ier Air Traffic Control Center controllerMarshall controllerCarrier controlled approach controllerBolter controlLanding Signal Officer controller

InstructionalCueingCoachingPerformance feedbackMission instructions (e.g., NFO °slow to 320 knots.")

*Debrief the RP, summarizing strengths and weaknesses inhis performance, and ascribe possible reasons for them.

27


*Prescribe remedial, extra and next training content.

*Perform post - exercise instructional management record

keeping.

The TD assists the IP to operate the device and also per-

forms maintenance as required. TDs do not perform instructionalfunctions, but respond to RP requests for system initializationsand failure insertions or removals during "extra time" trainingexercises. The functions that they pe-form in operating the

device are listed below. An asterisk appears by functions which

are candidates for automation.

Set simulator cockpit controls to appropriate positionsprior to the RP's accomplishment of checks precedingtakeoff.

Activate the OFT's computers, load the program tapes,and perform system readiness and safety checks.

Complete the system initialization by entering datain keeping with the Instructor's Briefing Guide forthe exercise or request by the IP which programs the:

Emergency manual insertion /removal controlsReset controlCarrier site dataGround site dataAircraft environmental data

*Activate and initialize the VITAL-2 visual syst

*Respond to IP and/or RP commands to accomplish:

Ground power on/offGround air on/offRemove/insert (simulated) wheel chocksInsert/remove emergencies/failuresOperate slewing control to reposition the simulator'sgeographic position

Close down the simulator following training or performmaintenance.

The asterisked IP and TD functions, in conjunction withpreviously discussed auxiliary functions, provided the basisfor deriving the organization and structure for the design of

the ISS.

2


SECTION IV

INSTRUCTIONAL SUPPORT SYSTEMFEATURES AND CHARACTERISTICS

OVERVIEW

This section covers:

o the development of the design concei=t for a "fullsupport" ISS, and

o the translation of the concept into a Fl4A OFT ISS design.

OBJECTIVES

Identifying and characterizing features of a full supportISS was not a challenging task. However, developing anintegrated ISS conceptual framework to organize and implementsupport features proved very challenging because several designgoals being pursued frequently countered each other.

A primary objective was to provide for instructionalflexibility. One need for flexibility centers on being able tomodify computer-resident syllabi with relative ease and no majordisruption to system software, so as to keep pace with changinginstructional requirements. This translated into a requirementto organize the computer-resident syllabus content in a highlymodular framework. Such that, syllabus modules could be changedwithout disruption of the resident syllabus.

A second related objective was to develop a system structurethat would accept the broadest possible spectrum of instructionaldecision making. One end of this continuum is characterized bythe highly standardized, or canned check ride. The structurealso had to accommodate automated decision making characterizingadaptive training. Finally, IP decision prerogatives at thebeginning of each simulator session and during the session hadto be accommodated in a manner that would enable IP's to tailorautomated instructional support to daily needs as they arise.

Finally, the system structure had to be flexible enough torespond to the total spectrum of training tasks that any RPmight specify for extra session training. These could cover thespectrum from canned missions through self-paced automatedadaptive training, through practice on particular flight controlor procedural tasks.

Concept transportability also had to be taken into account.This required anticipating the extension of ISS applicationsbeyond the analysis reference point established in the trainingenvironment. Thus, other mission categories, crew positiontraining devices, anticipated instructional support requirements,

29

NAVT EQUIPCEN 76-C-0096-1

and integrated aircrew training applications had to be con-sidered. Feedback will have to provide the extent to whichconcept transportability will have been achieved.

The requirement that the ISS also support instructionalresearch did not prove difficult conceptually. The need tomaintain automatically, data files identifying the utilization(or lack thereof) of ISS features was established to provide onetype of measure of user acceptance. It also is anticipated thatthe utilization files will provide information of value fordeveloping and refining adaptive training logics. Thisinformation will be based upon modeling of IP instructionalbehaviors as reflected through the manners in which IP's callupon ISS in relation to student progress and performance.

The need to support automated performance measurement andscoring research also was recognized, Automated systems can andshould measure RP performance at a finer level of task definitionand, typically, using a broader spectrum of measures than ispractical with a manual measurement and grading system. Creativemeasure analysis and definition can go a long way toward definingmeaningful automated measurement. However, the need was recog-nized for empirical measurement research to develop minimummeasure sets for various tasks, and particularly to developscoring algorithms for evaluating task performance at finerlevels than is commonly practiced in training simulations.

The need for research flexibility in automated trainingproblem development and modification also had to be accommodatedin the overall system structure. It is assumed that adaptivelogics will have to be developed empirically from logicsinitially derived by analysis. The impact of this need had tobe accounted for in software architecture. It also played arole in defining the scope and character of basic instructionalunits that comprise the computer resident syllabus.

A paramount goal was to develop system concepts that,hopefully, would be responsive to the previously described needsand would, at the same time, foster user acceptance. Thisrequires that system support features had to have a highlikelihood of providing meaningful instructional support in anoperational training environment. Thus, the selection ofsupport features and the characteristics of their operation hadto consider the IP and the TD as the system user(s). In thiscontent it was obvious that research capability of the ISS,although necessary, could not pace the system's conceptualization.

Finally, evolving a highly straightforward, logicallyorganized, relatively easy to use man-system interface for IP'sand TJJ's was considered absolutely essential for user acceptanceand, therefore, 'SS utility. It was recognized that the inter-face would have to enable IPIr, to obtain the type and amount ofinstructional support desired with considerable ease. A

30

NAVTKAEQUIPCEN 76- C- 0096 -1

control-display or procedural wilderness simply had to beavoided. As a result, all candidate features, characteristicsand system operational concepts had to be evaluated repeatedlyfor impacts on the type ©f man-system interfaces that might berequired to exercise them.

A focus was needed from which to initiate development andanalytical evaluation of an automated adaptive ISS. One pointof focus that proved workable was to define and scope themodules of instruction to be accomplished and supported. Theresulting "task modules" are discussed subsequently. A secondpoint of focus was to develop a basic structure within which toorganize support teatures and capabilities in a hierarchicalmanner. These subsequently became system operating modes, whichare discussed below.

TASK NODULES

Task modules (TM) are individual programming units whichdefine the training objectives in the computer-resident syllabus.They are the lowest common denominators with which instructors,system algorithms or adaptive logics can create trainingexercises.

The TM concept is central to ISS flexibility and growthpotential. Training exercises are built by selecting andorganizing TM's which represent training objectives that astudent has yet to achieve. This provides flexibility forefficient self-pacing. A master listing of all TM's ismaintained in a system file. Content of the file can be changedto reflect changing instructional requirements which the systemis to support. Existing TM's can be modified to add or deleteperformance measures, change proficiency standards, or makemodifications to any of the elements that define a TM. RPprogress in achieving criterion performance on TM's is maintainedin the system to allow for identification of objectives yet to beachieved, for planning purposes.

Three types of TM's are required:

Plight profile TM's, which are mission flight profilesegments.

o Procedural TM's, which are the various normal andemergency procedures that can be simulated.

o Tactical TM's, which are tactical engagement trainingobjectives for use in a Lull mission ISS.

A further characteristic of TM's is that procedural TM's,such as emergency/failure modules, can occur simultaneously witha flight profile TM or tactical TM. This can be done byassociating procedural TM's to "hooks" defined in flight or

31


tactical modules. Procedural modules also can be exercised whenpredetermined initiating conditions have been met. They also maybe manually initiated.

Module sizing was a significant consideration. Trainingobjectives can be specified at almost any level, ranging frommacro to microscopic. Procedural TM's sized themselves; eachprocedure comprises a module. Flight profile modules requiredsizing decisions. It was decided that TM's would be sized inkeeping with the manner that pilots typ:ally view a mission.Examples of TM's sized in this manner are: takeoff, departure,individual airways legs, approach, final approach and landing.Separate modules were then established for each type of takeoff,various departures, a spectrum of airways legs, differentapproaches, final approaches and landings. For trainingexercise development based on current training practices atVF -124, a total of 88 flight profile modules were defined.Normal procedure and flight profile PM's are presented inAppendix A. Emergency/failure modules are presented inAppendix B.

Tactical modules were not defined. Conceptually, however,each tactical module would consist of an engagement. Themodules would define threat types, numbers, altitudes, speeds,and additional information characterizing the threat to be copedwith during the engagement. This approach to sizing is idealfor within-session adaptive problem control because subsequentthreats can be selected and instituted based upon crew perform-ance in coping with prior threats, within the session as well asduring previous sessions.

The approach taken to TM sizing further defined characteris-tics of the modules. For example, a single module may incorpo-rate more than one performance measure or proficiency standard.This is particularly true'of flight modules, because many aremade up of more than one flight segment. For example, aSeawolf-Seven standard instrument departure from NAS Miramarcontains four discrete flight segments:

o Climbing right turn to 300° magnetic to interceptNKX TACAN Radial 280

Intercept 280 radial at 2,000 ft.

o Fly outbound 280 radial at 2,000 ft. to Seaw lf(NKX DME = 7 mi.)

Climb outbound on 280 radial to W-291 boundary(NKX DME = 31 mi.

Certainly, such fine cut flight segments can be construedas performance objectives and, hence, as training objectives.However, doing so would have extracted them from any meaningful

3

32

NAVTRAEQUIFCEN 76-_ 0096-1

operational context. Further, assembling training flightprofiles out of such small units would be an unnecessarilyburdensome human task as well as a complex machine task.

Furthermore, it was recognized that task modules also couldhe defined in ways that would make them too encompassing. For ex-ample, a flight module could be defined to consist of a particulardeparture in combination with a particular airways route. Ifthis were the case, IP's and system algorithms could only callupon the "combined" module, even if training was required onlyon a portion of the module, such as the departure.

iAccordingly, the following information elements were established

to define a task module. The information elements were derivedfrom an analysis of information the ISS would require to supportinstruction at the task module level. The resulting elementsorganize easily within the definition of a training objective,i.e., specification of behaviors to be trained, conditions ofperformance, and standards of proficiency. The additionalcategory of system information was added to accommodate infer a-tion requirements unique to automated instruction.

o System information

Module entry conditionsModule termination condi t ionsBase data, shoreBase data, shipVisual system data

T-Graphic display dataHook definition data, for associating failuremodules with flight or tactical modules

Controller models to be usedCueing message datainstructor alert message data

o Conditions Informati

Environmental data, atmospheric and oceanographicRequisite aircraft configuration dataRequisite aircraft systems failedAircraft initialization data

o Task Information

Failure module designationFailure insertion conditionsFailure removal conditionsProcedural steps for coping with failureNormal procedure module designationNormal procedure procedural stepsFlight module designationFlight segment(s) definition dataTactical module designation

NAVTRA QUIPCEN 76-C-0096-1

Task Information tied.)

Threat characteristics dataMission communicationCommunication originator - ISS, IP or RP

Communication initiating conditionsCommunication message contentCommunication delivery medium, display for IP

or voice generation

Measurement Information

Performance dimension designation (s)Performance measures - measure start/stop conditions,parameters, desired values, transforms

Proficiency scoring criteria data - ,,.1gorithm forconverting performance measures to a scoreconsisting of one of five categories

Algorithm designation for combining scores within a

module

ISS monrg

ISS operating modes evolved out of the recognition that

total system capabilities had to be organized- initial attempts

Lo identify ISS modes centered upon clustering variousinstructional support features into categories. It soon becameevident, however, that this strategy was resulting in system

operational structures that bore little direct relationship to

operational training.

A further strategy resulted in the operating modes that are

presented below. This strategy involved definition of system

modes that encompassed the continuum of instructional decision

making, ranging from detailed planning of the instructionalcontent of a simulator session through relegation of training

content decisions to the computer. A second element of the

strategy was to conceive of modes in a manner that would provide

instructors with considerable latitude to change to modes

different from the one selected at the beginning of the simulat

session. This was felt necessary to ensurethat ISS would be a

flexible, responsive support system. The following modes

resulted.

COHPUTER_ASSISTED MANUAL MODE. The ComputerAssisted Manual (CAM) mode will require the greatest instructorinvolvement in planning the content of a simulator session. As

with all modes, instructional content is selected from a

computer resident syllabus containing all task modules.

In the CAM mode, the IP (or TD) selects individual task

modules for which he desires automated instructional support.

This capability allows the IP to draw upon automated support

while tailoring the content presented to the RP. CAM is

34


expected to have research utility in that automatic recordingIP selection of this (or any other) mode, along with the trainingcontent (modules) selected, should provide valuable informationfor subsequent use in developing and refining adaptive traininglogics.

From a training standpoint, for example, if an RP is having_difficulty adequately performing TACAN approaches, the IP cancall various TACAN approach modules resident in the syllabus.This, in turn, would enable him to obtain automated measurementsupport for use in diagnosing the RP's performance problem.

In a different instance, an RP may wish to practice carrierlandings during an "extra session." In this case, the TD couldcall appropriate approach or final approach modules with relatedvoice controllers and "enable" reset to module entry conditionsfor repeated trials.

The CAA mode also enables the IP to request automatedsupport in the area of procedures monitoring. For example, themode enables him to select automated support for monitoring RPperformance relative to one or more normal procedure modules.Similarly, it provides him with total flexibility for selectingindividual and multiple emergencies/failures to be inserted,monitored and removed.

The CAM mode should have considerable utility for tacticaltraining applications. Tactical engagements usually arerelatively brief, and a number of engagements can be incorporatedinto a unit of simulator training time. The CAM mode providesthe instructor with ,?ery convenient and responsive means ofselecting and initializing subsequent engagements based onstudent prior performance.

INSTRUCTOR SELECT KODE. The instructor Select (I5P1-.1mode is much like the CAM mode in terms of training contentdecision making. ISEL, however, will allow the IP to selectfrom a list of total mission profiles, each profile consistingof a pre-determined structure of flight modules organized andsequenced into a meaningful mission context. Selection of theISEL mode allows the IP to quickly and easily draw upon automatedsupport for the flight portion of the mission, while retainingdecision prerogative on, for example, emergencies that he wishesto have the student cope with.

ISEL incorporates two ways of inserting or removingemergencies. Firstly, to select emergencies from a computergenerated display listing, and to specify the conditions forautomatic insertion and removal. Secondly, to manually insertand remove failures vi the ISS console.

Similarly, automated support in monitoring and evaluatingthe performance of normal procedures can be obtained by selectingnormal procedure modules from a computer generated display listing.


For training in tactics, the ISEL mode would allow the in-structor to select from alternatives, pre-determined sets oftactical 5eenarios, each scenario consisting of a series of

engagements made from tactical task modules.

CAIT4ED MISSION MO L. The CA1J1VED mode will build uponISEL by providing for totally pre-programmed missions and events.Thus, flight, normal procedure, emergency and tactical modulescomprising a total training exercise will be specified in advanceas the instructional, decisions of this ni-,de center exclusivelyan selecting the canned mission to be used.

In practice, it is anticipated that two categories of cannednissions will be required. The first category will be missionsdesigned to emulate present simulator training exercises and thesecond category will be NATOPS evaluation missions, which, shouldbe hicrhly standardized and objective.

A canned mission capability is desirable for several reasonsas follows:

o It provides instructional personnel with easily accessiblefull system support to accomplish training in a highly

4-1.4111-11-1 is

structured.

o It provides instructional users, with a minimum traininginvolvement opportunity, to observe a spectrum of ISSinstructional support capabilities and features. Thiscould be particularly beneficial to instructional personnelwhen they are first learning how to use ISS.

It enables the RP, with the assistance of a TD, toutilize the spectrum of ISS capabilities during "extratraining" sessions.

o It provides the student with the opportunity to self-testhis performance capabilities, at least within the trainingcontent of the mission selected.

When various adaptive training logics become operational,their effectiveness likely will be enhanced if trainingis at least started from a standardized baseline.

Note that ISS avoids the rigidity often associated with cannednission scenarios by building each mission from computer-resident

task modules. As discussed previously, the content of any module

can be altered, and new modules can be created . similarly, algo-

rithms that draw upon the modules to create canned training exercises

can be modified. Thus, content of canned missions can be modified

with relative ease. Additionally, the capability to createaddi-

tional canned missions exists.

36


willOPERATION MODES. Adaptive operation modes

will be developed empirically, after sufficient data and ex-perience have been obtainod on all lower order modes. However,at this time, the envisioned functional characteristics ofadaptive operations can be described. Basically, adaptive modeoperations will audit the passing and failing of specific learn-ing objectives, defined by task modules, and attempt to createmissions from structured lists of objectives not yet passed.

The information structure that is needed by adaptive modeoperations is illustrated below. Each learning objective islisted in a prioritized order, with the most important or mostdifficult objectives occurring generally at the top. This listwill point to the task modules that addresses the objective.Each learning objective will point to a companion criteria filethat will indicate, as a minimum, the score required for passingand the minimum number of passing trials required to meet eachobjective.

LEARNINGOBJECTIVES1

SPECIFIC PEREORKUIC CRITERTASK MASUREMEN_ FORMODULE-7S i PASSING

STUDENTLEARNINGOBJECTIVE

TORY

#1 . 1

X1.2X1.3

The minimum information set for flight and tactical moduleswill be similar to procedure modules, except that an additionalfile that organizes groupings of modules may be required forlogical flight continutiy. Also, each mission organization musthave associated with it a set of emergency insertion criteria thatare permissible. That information set is illustrated as follows:

37


LEARNINGOBJECT IVES

MISSIONLOGICALORGANI ZA-TI0tJ

SPECIFICFLIGHTTAS ICMODULES

_ MERGENC TUDENTINSERT I EARNINGCRITERIA iBJECTIVE

STORY

ti

CRITERIAFORPASSING

Thus, at this point, the system knows what needs to be learned,

a number of ways it can be organized, and the criteria forachieving

the learning objectives. The various adaptivemodes are described

below:

ADAPTIVE-BETWEEN MODE. The adaptive-between (ADBET) mode

will use adaptive training logic that operates between

simulator sessions only. It accesses information identify-

ing which training objectives have and have notbeen met

by each RP. It derives a list of objectives that have

not been passed. Next, it constructs a simulator flight

profile or series of tactical engagements from lists of

logical flight or tactical module organizations which

reflect unsatisfied training objectives. Failure modules

are attached to the resulting flight events. ADBET is

hierarchical: e.g., satisfaction of failure objectives

can be made more important than satisfaction of flight

objectives, within limits.

A fairly sophisticated set of rules will be required to

optimize the achievement of different categories of

objectives and associated task modules. For example,

some flight modules (e.g., carrier operations modules) are

not amenable to emergency procedure training.Thus, ADBET

logic may be required to create "composite missions,"

such as half of a mission in a simulated warning area,followed by transition to carrier operations. This would

be the case, for example, where flight objectives werebeing met at a rate faster than emergency objectives.

38

NAVTRAEQUIPCEN 76---0096-1

Development of rules to accomplish between sessionadaptation were not addressed during this study.However, the student's position in the syllabus, therequired procedures which precede flight training,the number of objectives passed (or yet to go), and thelimits of "normal" student progression will have to beconsidered in order to determine even a rudimentaryset of rules.

ADBET also will require rudimentary algorithms whichaddress alternate training time-optimal pathways throughthe syllabus. It is possible, for example, that someemergency or flight training objectives may be "post-poned" until a later time if achievement of them isimpeding student progress.

ADBET logic is not intended to be infinitely adaptive.For example, only logical groups of flight modules willneecito be considered for constructing mission profiles.These groups will likely fall into the categories offlights in warning area, India routes, departures,approaches and final approaches, carrier operations andtactical exercises.

o ADAPTIVE-WITHIN MODE. The adaptive within simulatorsession (ADWIN) mode will do everything the ADBET modedoes, only it will operate within a simulator session.Thus, real-timg, or very near-real time decisions willbe required. ADWIN is conceptualized to re-organi zethe simulator mission (if necessary) as a function ofthe performance achieved on the prior completed emergency,flight or tactical module. In reality, it is probablethat ADWIN will operate most effectively in relation toemergency procedure and tactical objectives.

Given prioritized lists of learning objectives, ADWINinitiates a simulator mission using ADBET logics. Ifa higher priority objective is not passed, it is broughtup again in place of a lower priority objective laterin the mission. A set of contingencies is required toreasonably distribute objectives of like priority sothat one objective does not repeat to the exclusion ofeverything else.

Due consideration will have to be given to the within-session change of flight modules. Only limited flightmodule adaptation should be permitted, within the Contextof the mission. For example, a substandard take-offand departure may be worthwhile repeating, at least aftera certain stage of training. Certainly, approaches maybe repeated in favor of some other flight modules thatcan be postponed until later or the next simulator.session. Finally, flight maneuvers that are importantfor the next aircraft flight may be repeated in favorof less critical maneuvers.

39


ADWIN's adaptatio- for tactics appears quite natural,because one may, change the nature of the next tacticalexercise, providing the exercise remains within areasonable mission context.

o SELF-ORGANIZING MODE. The self-organizing (SELFORG)mode is envisioned to be upward compatible with bothadaptive modes. SELFORG will perform audit trails onthe operations of ADBET and ADWIN, and adjust controlalgorithms (algorithms which control the selection ofexercises) in accordance with the probability of successthat has been achieved by students passing through eachof the specific exercises or individual modules. SELFORGwill eventually preclude unproductive pathways and inessence, it will perform housekeeping, but cannot addnew exercises.

A SELFORG capability will have to be implemented throughoffline data analyses performed by a training researchspecialist. However, as a result of current research,ways may emerge to implement controller algorithms thatcan perform all or part of a normal audit trail operation.

Given the current state of the art in automated instructionalsupport, it is likely that a first generation ISS would incorpo-rate only CAM, ISEL, CANNED and a preliminary ADBET capability.ADWIN and SELFORG mode capabilities require additional developmentbefore meaningful operational applications can be made. In par-ticular, methods of joint problem adaptation of flight and proce-dural task combinations and procedural and tactical task com-binations require considerable ISS data collection before modelscan be developed.

SUPPORT FEATURES

The balance of this section describes other categories ofISS features many of which will be independent of mode, sincesupport capabilities such as performance measurement and scoring,RP cueing, and use of automated voice controllers, are specifiedat the task module level.

The material herein is intended to simplify the presentationof support feature information and is not intended to dictatesoftware organization of a full support ISS.

DATA FILES. A number of different data files willbe required for efficient planning and accomplishment of asimulator training session. These are:

40


STUDENT HISTORY PILE. A master computer-resident modulefile will contain a listing of all task modules that areassociated with the learning objectives that RP classesare to accomplish through training in the simulation. Itdefines the computer-resident syllabus, and can bemodified as required. As performance measurement andscoring data indicate that individual RPs have achievedcriterion performance on a task module, note is automa-tically made of this in the master file. What resultsis an identification of modules on which criterionperformance remains to be demonstrated. This informationis maintained separately for each RP. Content of the file,when accessed by the instructor or adaptive logics,provides a basis for efficiently structuring the contentof simulator sessions by providing an ability to focus onobjectives yet to be mastered. The date of achievementof each module by each student also is noted to providefor assessing rate of progress.

STUDENT AND CLASS FILE. This file will contain individualstudent background data and is designed to provide in-structors with ready access to relevant student back-ground information which can be useful in planningsimulator training session content. Content of the filecan be accessed and displayed at the class level, inaddition. Summarized at the class level, content of thefile provides instructional management and researcherswith diagnostic information to assist in assessingdifferential group performance at the class level.

o MEASURES COLLECTION FILL.. An automated, adaptiveinstructional support system will require the use ofvalid criteria for acceptable and unacceptable levelsof student performance. Additionally, valid diagnosticmeasures that can pinpoint student performance defi-ciencies in manners that allow for attributing probablecauses also are required. These requirements exist bothfor human interpretation of student proficiency anddiagnosis of performance problems, and for automatedadaptive training logics designed to accomplish the same

goals.

The many advantages previously described for sizing taskmodules bring with them a requirement for empiricalmeasurement research and development. The various taskmodules are of a rather precise nature. Each will requiremeasuring a number of performance dimensions. Analytic,or best guess, measure analyses must provide the startingpoint. Empirical work will be required to modify andvalidate the initial best guesses, both with respect toproficiency assessment and performance problem diagnosis.

41


A considerable spectrum of performance standards will

be required. For example, some task modules, such as

flight TMs, contain more than one measurement segment,

as described previously. Others, such as procedural

TMs, require measurement and assessment of highlydifferent dimensions of performance. The assessment of

performance of a procedure, for example, requiresmeasurement and scoring of responses, together with

measurement and assessment of the occurrence of required

procedural steps and the sequence c: occurrence of at

least highly critical stages.

The purpose of the ISS measure file is to collect data

to establish objective standards of acceptable performance

as well as diagnostic information at the task level.

The value of the research and development made possible

by the file falls within several areas. One is to

replace initial best guess performance standards with

empirically derived standards. This should enhance user

acceptance of ISS proficiency assessments because the

number of scores that instructors or students would

question should diminish, at least in theory. Results

of the research hopefully should guide measure analyses

for aircrew training applications beyond aninitial ISS

installation. The functioning of automated adaptive

training logics should be enhanced through use of data

provided by the measures file.

Measures data will be voluminous and will have to be

recorded for off-line analyses to develop performance

norms and diagnostic measure sets. Each measure will

have to be highly qualified, at least in terms of the

following dimensions:

Task module designation

Task(s) (measure segments) within the module

Measure source (student or IP Identification)

Date and time of day, to allow for relatingperformance to prior executions by the same

measure source

PERFORMANCE NORMS FILE. This file is designed to contain

a current listing of performancenormative data for use

by scoring algorithms in scoring proficiencyat the task

module level. Initially, the file will contain best

guess data Content of the file can be updated with

empirically derived normative data following empirical

measurement research and development.

42

NAVTRAQUIPCEN 7 -C- 0096 -1

The file will have to contain two classes of normativedata. One class will be quantitative breakpoints forscoring individual performance dimensions within atask module. The second class will be quantitativebreakpoints for a use in developing an overall modulescore for use in assessing proficiency at the modulelevel.

An intriguing alternative to developing normative datathrough off-line analyses and human judgements is auto-matic generation of norms by system software. Automaticgeneration of norms often is assumed in higher order,self organizing automated adaptive training structures.

The 155 concepts and functions set forth in this reportdo not incorporate automated norm generation for tworeasons. First, the ability to do so in a system asinherently complex as 155 remains a researchable issue.As automated instructional support system applicationsin operational settings are in their infancy, initialuser acceptancp of primary imp7,rtance. The technologythat can be brought to bear in operational trainingsettings through such systems has yet to be demonstratedto instructional and training management personnel. Inthis context, incorporating a competitive, computer-basedcapability with NATOPS or other Navy standards of per-formance may be premature.

o INSTRUCTOR ACTIONS FILE. Its purpose is designed toautomatically record significant instructor actionsinvolving 155. These are: modes initially selected,mode changes made during the course of a simulator sessionoptional support features selected, and support featuresthat normally would come into use automatically but werede-selected by the instructor. This information is ampli-fied with instructor and student identification informa-tion, together with date and time of day. These data willprovide the mechanism for relating significant instructoractions to student performance and history information.

Over time, analysis of significant instruction actionswill serve four purposes: firstly, it will objectivelyidentify relative frequencies of ISS feature usage;secondly, it will provide a means for taking instructoractions relative to student performance into account forbuilding or refining adaptive logics; thirdly, it willprovide a quantitative source of feedback on the effec-tiveness of instructor training in 155 usage, and forthly,it will provide a source of feedback on 155 acceptance byinstructors.

PRE-SESSION SUPPORT. ISS is conceived to provide bothancillary and auxiliary instructional support in preparationfor the conduct of a simulator training session. Pre-session

NAVTRAQUIPCEN 76-C-0096-1

support centers in three areas: firstly, mission planning;secondly, computer-generated pre-session student briefing contentthat is based on session instructional content identified duringplanning, and thirdly, computer based pre-session readiness test-ing. The latter is intended largely to replace the pre-sessioninterrogation of students by instructors, and is expected tohave direct application during "extra session" training when anIP may not be present.

MISSION PLANNING. From the ins'-ructional user's stand-point, ISS is organized to provide him with studentperformance history information, including objectivesyet to be met. Displaying a list of modules on which"criterion performance is yet to be achieved".is designedto provide objective focus for session planning.

Following a review of student performance history infor-mation, the next planning decision required will be theselection of an ISS mode of operation, ranging from CAMthrough various adaptive modes. In CAM, the instructionaluser selects one or more task modules for which instruc-tional support is required. These are not limited tomodules on which criterion performance has not beenachieved. They can be any modules. In ISEL, the userselects a total flight profile or series of tacticalengagements as a starting point. If he has used avail-able planning information optimally, the selected profileor engagement will maximize opportunities to achievecriterion performance on yet unmastered task modules.The basic ISEL unit can then be amplified with proce-dural and/or emergency modules at the discretion of theuser. CANNED mode exercises are selected in a similarmanner to ISEL. If an adaptive mode is selected, sys-tem logics select modules to be incorporated into thetraining session.

A mission builder function will then be called upon.The mission builder takes mission planning inputs fromthe instructional user (CAM and ISEL modes), algorithmsfor the CANNED mission plus adaptive logic which mayhave been selected, and creates the training exercisefrom the modules that have been specified.

A mission editor function is then called upon. The missioneditor checks for module-to-module compatibility. In doingso, it determines whether existing conditions for a preced-ing flight module, such as altitude, heading for speed, arecompatible with entry conditions for the next flight module,and any discontinuities are displayed. For. example if anairways leg flight module and a final approach flight modulewere selected, a discontinuity may exist, as the termina-tion of the first would not necessarily have to coincidein time and space with the beginning of the second. In

444

NAVT_ -QUITCEN 76-C-0096-I

the event of incompatibility, the IP is required eitherto accept the incompatibility or modify his initial plan-ning to eliminate it.

A second type of mission editor compatiblity check is to as-sesscompatibilities of procedure modules, both normal pro-cedures and emergency procedures, with flight or tacticalmodules in which they are to occur or which follow them.Logical inconsistencies are identified. For example, itis inconsistent to plan an engine hot-start emergencyfor occurrence during a flight module when both enginesare anticipated to be running normally. Similarly, a no-flaps landing requires prior failing of systems that areused for operating the flaps.

In this respect, it must be noted that flight task modulesidentified in Appendix A as having instructional meaningin the VF-124 context, incorporate different flightmodule designations for situations where certain prioremergency conditions exist. Final approach is an example!different final approach modules are identified for flapversus no-flap. This is necessary because standards ofacceptable flight control performance may be differentfor the two types of landings and such standards are con=tained in individual module definitions. Measurement andscoring research will also require maintaining performancemeasure data separately for such modules.

The mission editor also can be viewed as a training toolas it will sensitize the new ISS user to potential prob-lems he can create while planning a mission if insufficientattention is not paid to details in the use of CAM, ISELoperating modes.

Furthermore, the mission editor function also will bebeneficial in developing CANNED missions which must becompletely debugged before they are made available toinstructional users on a day-to-day basis. Similarly,adaptive logics may draw upon the power of this editingfunction.

The planning activity, which identifies the content andthe sequence of the planned simulator session, provides the basisfor two optional, pre-sesssion instructional supports: a computergenerated pre-session briefing, and a computer-generated pre-sessionreadiness test.

PRE-SESSION BRIEFING. At the ISS user's option, a computer-generated pre-session mission briefing can be requested.Content of the briefing is patterned after present instructorbriefing formats, which outline the mission scenario. Thecomputer-generated briefing identifies the normal proce-dure, flight and tactical task modules, in sequence, that

45


comprise the mission plan. Emergency modules incorporatedinto the plan are listed separately and in alphabeticalsequence to minimize student anticipation of when theyare apt to occur. It is expected that the briefing willbe electronically displayed and a hard copy made.

o PRE-SESSION READINESS TEST. It is a common and usefulpractice for instructors to interrogate students on air-craft operational limits, effects of failures and malfunc-tions, stimulus patterns associ ed with various aircraftstates, operational rules, and other elements importantin the context of the upcoming training session. Thiscan be done following the mission briefing. The intentis to identify mission-critical knowledge weaknesses sothat remedial instruction can be given prior to themission. The full-support ISS incorporates this functionwith the objectives of standardization and providing formission readiness testing in the absence of an instructor.

Following receipt of the mission briefing by the student,ISS can be commanded to draw upon an item pool and develop

a pre-mission test based upon the briefed mission that the

student is to fly. The test is administered and scored by

ISS console. A multiple choice format is anticipated.

After the test has been taken, the instructor, if present,

can review the results and provide necessary remediationprior to commencing the mission demonstration.

The number of test items and the logic for their selecld.onis not addressed here. It is unlikely, however, that c%-le

test item should be administered for each task module that

has been planned for a simulator session. The resultingitems could be both excessive and redundant.

The question of the utility of providing a pre-session demon-stration of planned mission events, either in the cockpit or atan ISS console, should be answered with respect to specific ISSapplications. Demonstration capabilities are not common intraining simulators presently in use. Thus, there is a dearthof operational feedback on the utility of demonstrations as afunction of mission task characteristics and requirements. Addi-

tionally, no directly applicable basic research is known to exist.

The utility dimensions of pre-session demonstrations involv-ing an ISS or a training simulator must be questioned. Having

a computer operate cockpit controls to demonstrate procedures haslittle apparent training value and would require controls thatcould be operated remotely. Furthermore, in the tactical train-ing area, it must be remembered that tactical maneuvering andweapon delivery decisions must be influenced by adversary offensive

446

NAVTPAEQUIPCEN 76-C-0096-1

and defensive actions. Demonstrating one possible way of com-pleting a tactical exercise in a simulation could be of littlepractical training value. However, a capability to quicklyre-set to defined initial conditions, as ISS enables, providesthe opportunity to increase the number of missions conductedduring a session. For these reasons, a demonstration capabilityis not recommended for initial ISS applications.

TRAINIW; SESSION SUPPORT. Many of the teatures andcharacteristics of ISS support the conduct of instruction duringa simulator training session. They are intended to be of instruc-tional value to both instructors and students. Features andcharacteristics directly relating to the conduct of training aredescribed below.

Automatic.Problem Initializatio Following the comple-tion of planning activities, the entire training programwill automatically initialize. ISS monitors pertinenthost simulator systems and data parameters to determinewhen entry conditions for the first task module havebeen met. From this point on, system operation can betotally automatic within the design bounds of ISS.

sequencing of subsequent,task modules will be automatic. As above, module entryconditions data are monitored so that ISS can determinewhen to begin operating on the module's program.Similarly, module exit conditions are monitored so thatan on-going module can be closed out when it has beencomplteted.

Inqtruotional_140nitoring Information. Two CRT displays_ _will provide the instructor- system interface throughwhich all mission information can be called upon.

Section VI of this document addresses the physical descrip-tion of ISS, including the man-machine interface. It is impor-tant to establish here, that the instructor-system interface willbe built around two CRT displays.

One display is a graphics display that allows monitoring, inreal time, of the following: cross-country flight path history,flight path history within a simulated warning area, approach andfinal approach profiles, and tactical flight history. This capa-bility is provided to allow monitoring of mission progress rela-tive to the mission plan. Content of the display is definedlargely by the definition of the flight or tactical module thatis active at the time.

A touch panel alphanumeric display is provided for instructorcommand of ISS and for ISS display of instructional monitoringand other information. Using the display, the instructor canreview task modules comprising the session plan and receive infor-mation regarding which modules have and have not been performed

47

NAVTRAQUIPCEN 76-C-0096-1

to criterion. Also at his option, he can review performancescores on completed modules, and request_more detailed diagnosticmeasurement information on completed modules.

Instructors also will be able to participate in scoringstudent performance through the man-machine interface. Scoringof student communication protocols and performance is assumed tobe manual, pending further development of computer speech under-standing system technology. A scoring format is displayed uponinstructor request. He may then identi_y the communicationmodule and enter a score. The instructor-generated score isused to assess the student's achievement of criterion performance.

For research purposes, and to facilitate initial refinementof scoring criteria, instructors also will be provided with theoption of overriding computer-generated scores. These data willenter

DOCUMENT RESUME 235 Semple, Clarence A.; rd 0 the ... · PD 171 242. AUTHOR TITLE. TNSTITUTON....

Documents

Transcript of DOCUMENT RESUME 235 Semple, Clarence A.; rd 0 the ... · PD 171 242. AUTHOR TITLE. TNSTITUTON....