Techniques for evaluating military organizations and their equipment

TECHNIQUES FOR EVALUATING MILITARY ORGANIZATIONS AND THEIR EQUIPMENT

Paul Rrock

Stanford Research I n s t i t u t e Menlo Park . C a l i f o r n i a

Wal ter D. Correll*

U.S. Army Medical S e r v i c e Corps Hq. 6 t h Army, The P r e s i d i o San Francisco, C a l i f o r n i a

George W. Evans I1

Stanford Research I n s t i t u t e Menlo Park , C a l i f o r n i a

INTRODUCTION

The results of most theoretical endeavors in the natural sciences may be checked by

experimental evidence. The word "experiment" is used loosely to include all actions, processes,

observations, and occurrences for which data have been o r may be taken. An experiment implies

a predesigned plan for the acquisition of data and preplanning of the other phases of the processes

involved in the observations. Experiments may be controlled o r uncontrolled and produce direct

o r indirect confirmation, support, o r contradiction, or may show no relation to the parallel theo-

retical behavior.

The ingenuity and effort required both for the theory and for experimental design a re not

to be underestimated. The justification of theory by experiment may be hindered by measure-

ments and accuracies, but even this uncertainty may be pre-estimated and used in the design of

an experiment.

While natural sciences a re supported by a wealth of experimental recorded data, military

science is characterized by the lack of such support. It is true that many of the components used

by the military have had the results of their theoretical developments checked by experimental

data. Performance characteristics of communication equipments, flight characteristics of mis-

siles, and transportation characteristics of ground vehicles are examples. However, the per-

formance of such equipment when used by military personnel has not been experimentally evaluated in the environment of battle. This has been due in part to the difficulties of designing

experiments to represent battle conditions. Further, the design, for best battle performance, of

a field army, a naval or air force squadron, or even an army platoon, presents complex problems

that have resisted experimentation. (The adjective, best, is without scientific definition.) The opportunity to observe and record data in a combat environment seldom presents itself.

-

It is to this question of military experimental design, particularly for land operations, that attention is directed here.

*Lt. Colonel, MSC

211

212 PAUL BROCK, WALTER D. CORRELL, AND GEORGE W . EVANS I1

Currently, the development and performance of mili tary organizations are determined

mainly by mili tary evaluation. This consists of judgments that are neither definable nor rigor-

ous, made by mili tary personnel, often using procedures that are variable. This does not imply that evaluation methods are without value. However, such procedures require systematization

and substantiation through investigations of measurable quantities that can either support, extend,

o r contradict these evaluations. Systematized mili tary evaluations would present several distinct

points of view - i.e., an evaluation derived from one’s own forces, an enemy evaluation, and a composite evaluation. When possible, evaluations should be obtained before and after planned

testing to present evidence of the significance of the experiment. Much of the subsequent discussion will be devoted to developing criteria for the evaluation of mili tary organizations, and to

methods for obtaining these evaluations.

The evaluations should be considered as performed in peacetime for organizations to be

employed in future battles. Many of the methods may prove useful to obtain mili tary evaluations

during wartime, s o that the organizations may be continually improved during actual battle.

Another subject that will be mentioned is the transition of an organization from peacetime

to wartime. Generally, one considers that such a transition implies the developing of an aug-

mented force from an austere organization. Thus, the problems of choosing the best organization

may require prognostications of future enemies and conditions under which a war will be fought.

Although the discussion will consider the evaluation of organizations at various levels of staffing

and other transition problems, it will not be concerned with the probability of occurrence of

engagements with possible enemies.

The paper is directed to the study of organizations, equipment, and policies of the armed

land forces for possible future war s as well as towards the evaluation of present-day organiza-

tions, equipment, and policies. The philosophy behind the effort is predicated on the assumption

that i f experimentation determines flaws or weaknesses in a s t ructure , it will probably be a poor

structure for actual combat. If the s t ructure holds up experimentally i t may have a reasonable

chance of success in actual combat.

War gaming will be used in the discussion to encompass all operational r e sea rch tech-

niques that are applicable to the study of mili tary organizations. The study of mili tary organizations is predicated upon the existence of an enemy, and, when an enemy is not present, the studies

are either preliminary or of systems supporting other systems that have an opponent. Thus, both

words, war and game, are implied. ~ ~

The mili tary ground organizations of interest fall within three categories:

1. Strategic, concerned with longer range, broad planning, and execution,

2. Tactical, concerned with direct enemy contact and its immediate operational support, 3. Logistical, concerned with all aspects of troop support.

The particular techniques s t r e s sed may be classified as analytic, computer simulation,

map and CPX gaming, and field experimental. With respect to these techniques, Figure 1 indi-

cates how knowledge obtained from one technique may be expected to feed o r affect another. The ordering of boxes in this figure is not important since some investigations are s tar ted using one

technique and other investigations are s tar ted using other techniques. However, the iterative

EVALUATING MILITARY ORGANIZATIONS AND EQUIPMENT

r-

ANALYTIC METHODS

213

J.

.1 -

4t .-- COMPUTER SIMULATIONS

+

* MAP AND CPX GAMING

process displayed is important to extend the breadth of knowledge. In the following sections,

these four war gaming techniques will be discussed. Emphasis will be placed on the interrelation

of these techniques in the study of future military land organizations.

ANALYSIS

Systems

If one is asked to find the roots of a quadratic equation, a good memory, or quick refer-

ence to a high-school text wil l permit one to state a clear, precise, categorical answer. The

responder may not realize that his answer is supported by centuries of mathematical, deductive

logic.

If the question is whether o r not to pack rain gear for a three-day week end, the response

is quite clouded. Using a set of meteorological equations, a vast amount of concurrent existing

weather information, and a large computing machine, one may get an answer, yes or no, that has

a 95 percent probability of being correct. A simple, direct question, yet the mathematical cer-

tainty of an answer is lost.

If the question is one of combat force organization for optimum effectiveness, there is no hope of a rigorous definitive answer at all. Here, the question itself is ambiguous. The

methods of attack a re remote, and answers a re indeterminable. But this is the type of problem

that is critical, that has to be faced, and for which reliable answers must be supplied.

In general, the problem is to develop a system. A system must have a purpose, and it has constraints. The optimum effective combat force alone is too vague to constitute a purpose. To design a force to land on a beach whose topography, meteorology, and terrain are well known,

and for which there is knowledge of the defense, is a more defined purpose. The constraints begin to manifest themselves when one is assigned the equivalent of one marine company for the

job.

214 P A U L BROCK, W A L T E R D. CORRELL, AND G E O R G E W. EVANS I1

To design the system one studies a model. The scope of this paper covers four types of

models - analytic, computer simulative, map or CPX gaming, and field experimental. With the

model, one must establish implicit linkages - i.e., develop the ability to extrapolate from the

model to the real operation.

How much information exists for our model? The best possible intelligence gives only

limited enemy information. Realistically, our information is quite insufficient. Hence, the prob-

lem becomes “adaptive” in character; one wants to maximize the utilization of each new piece

of evidence for the puzzle [ 11.

What tools a re available? - Analysis, Logic, Statistics. A broad picture may be resolved

into the significant subpatterns. Points that a re extremely sensitive can be spotted, and those

that are invariant, about which the operation must pivot, can be determined.

The results will be at best probabilistic in nature, nonunique, and directed towards par-

ticular aspects of the questions. For example, different forces would be required for the pre-

viously mentioned landing if

1 . The action were isolated

2 . The action were to be continued by both sides

3. The action is part of a larger concurrent action.

The rationale for this effort is that it gives a pre-insight into the difficulties of military

problems. It produces probability estimates for successful operations. This probability of

success has analytical and experimental support, hence has an established confidence not present

i n guesswork or even in the experienced intuition of individual commanders under s t ress . The

latter, however, may be related to the former.

Analytical Studies

As is often the case in discussion-type papers, the methods under discussion a re not

rigorously defined. This may at times lead to some confusion. But the codification serves the

purpose of subclassifying the subject matter so that it may be more readily described and so

that interrelations between the various subclassifications may be emphasized.

The words operational gaming a re replacing the words war gaming in order that new _ . ~

techniques may be accepted in a field where preconceived ideas have inhibited progress. The

words analytical studies will be used here in place of mathematical studies. To many segments

of our society, unfortunately, anything mathematical implies much that is impractical. Analyti-

cal studies, a s used here, will refer to studies that can be carried out with the tools of mathe-

matics, sophisticated and simple.

Three general categories of investigation a re of particular interest. They are referred to as static, dynamic, and evaluative. Static studies concern the properties of organizations o r

equipment. This includes comparative analyses of the properties of two or more organizations

or equipments designed for similar purposes. Dynamic studies, on the other hand, a r e generated to determine properties that a r e time-dependent - i.e., those that are manifested over periods

of time. How an organization or equipment responds to changes in its operation or environment

is a pa6ticularly interesting and significant dynamic study. Dynamic studies usually provide

EVALUATING MILITARY ORGANIZATIONS AND EQUIPMENT 215

deeper insight into the operation of systems. However, they seldom can be performed without

previous static studies. Finally, organization and equipment systems must be evaluated. Eval-

uations a re based upon sets of criteria. The selection of proper cri teria is a primary area of

analytic investigation.

Examples of Static Studies

A comparison of the properties of two different pieces of equipment that were designed

for the Same or similar purposes is probably the simplest type of static analytical study. The

mathematics involved is probably no more than arithmetic and an understanding of the meanings

of “equal to,” “greater than, ” and “less than.’’ If the equipments a re mobile radio transceivers,

then the comparison might be based upon such properties as weight, volume, power requirements,

transmitting range, number of channels, frequency stability, bandwidth, methods of band switch-

ing, durability to exposure to various climatic conditions, and cost of the equipment.

Dynamic studies may indicate other properties that must be considered comparatively,

and may reduce the significance of properties that had been previously considered statically.

Also, some static properties may change with use and modes of operation. Static and dynamic

t ire balancing is a common illustration of the difference.

In static or dynamic comparison, a number of incommensurate properties a re measured

separately. If Item A excels Item B in each one, it is clear that A is the preferable unit. But

this is trivial, and if such a situation ar ises , the result generally would be obvious. A prime

characteristic of comparative investigations is that Item A and Item B each excel in different

properties. Then the question of which is preferable becomes meaningful.

Another simple static study is the preliminary staffing and equipping of a military force.

Assume that one knows the types of personnel and equipment available and that it is desired to

find a balance between types of personnel and equipment to obtain a specified force effectiveness.

The study also assumes that one o r more objective functions (cri teria of force effectiveness) a r e

known for the type of armed force under consideration as a function of the number and types of

equipment and personnel to be employed. In addition, constraints may be placed on the solution

by rules dictated by military needs within the armed force. For example, if a truck is assigned to the force then a driver must also be assigned.

The objective functions a re evaluated for all reasonable staffing possibilities and the desired staffing is obtained as an optimal in some sense of the word, according to these criteria,

subject to the imposed constraints. If such optimal values exist and can be determined, then the

proper balance between types of personnel and equipment of the force can be determined. Studies of this type may be extended or refined - according to the point of view established when the

investigation is initiated. For instance, the study may originate for an armed force in an attack situation and may then be reconsidered for defense, withdrawal, and reconnaissance situations.

Or, it may be originally considered for an overall posture, and then reconsidered by asking how this optimal force would be perturbed for various specified situations or missions.

The defining of the cri teria is a nontrivial problem and often the best one can do is to

assume for the criteria, functions with only some of the desired properties. In such cases, the

216 P A U L BROCK, WALTER D. CORRELL, AND GEORGE W. EVANS I1

development of methods for solution is an important by-product of such studies. In addition to providing general properties of the solutions for staffing in the above illustration, the methods

may also be used as a guide for establishing the composition of broader mili tary forces , a prob-

lem which by itself may be yet more difficult of c r i t e r i a definition.

Many other types of static studies can be performed. The determination of expected

damage and casualty assessments from various weapons or configurations of weapons, the logis-

tical requirements of scheduled and emergency supply and maintenance for specified tactical

units, the optimal communication networks and their operation in coordinating movements of

armed forces, are but a few.

Examples of Dynamic Studies

Dynamic studies a r e designed to determine how sensitive are specific properties of

equipment o r systems to changing environments and what the nature of their transient behavior

o r response is.

In reconsidering the comparison of two mobile radio t ransceivers from a dynamic point

of view, one might find that the transceiver with the fewer channels has more than an adequate

number for the system in which it is to be used. Statically, the desired number of channels may

have been g rea t e r than that possessed by the transceiver with the fewer number, but dynamic

studies may show that it is impossible to use this desired number.

Continuing to a more refined dynamic study in which breakdown ra t e s are considered, it may prove necessary to r e se rve two additional standby channels for each frequency to a s su re

reliable operation. Again, the available number of channels may become significant.

When studying staffing problems, dynamic studies often reveal facets not uncovered by

static studies. For example, the proper staffing of an armed force chosen on the basis of a static study as being optimal for attack, defense, withdrawal and reconnaissance situations may

default when studied dynamically because of transition limitations between situations. Such a result has the familiar r ing of the directions given by a service station attendant - “You can’t

get there f rom here, but I can tell you how to get there f rom the corner of Third and Main

Streets” - since the a rmed force is assumed to consist of personnel and equipment that regroup for different situations. Nevertheless, the dynamic study may show that the regrouping from an attack to a defend posture has an inherent vulnerability s o that the force may be annihilated by an enemy during the process.

Similarly, a static study may be used to choose the optimal staffing of an armed force,

considering both the missions that it will encounter, and its performance at various specified

reduced strengths. A subsequent dynamic study may show that the a rmed force s o selected is

not optimal when regrouping at reduced strength under transient battle conditions.

Most studies are performed in peacetime for future wars. The future organization of the armed force will most likely be initiated during peacetime, hence budget considerations may

be one of the criteria involved in the choice for staffing the force. Static studies may choose an optimal staffing for both an austere force for peacetime and for anaugmented force for war. Again,


the dynamic study may show that the transition from austere to augmented is impractical

because of the conditions under which it must take place. The discussion of staffing problems can and should be extended. Here, though, the pur-

pose is to point out that the assumptions made in stating a problem for study, and the restrictions

placed upon the solution by the methods of investigation a re often as important as the solution

itself. Even though it is necessary to provide, with the solution to a problem, that information

which delineates when and where the solution applies, such a complete description often leads to

misunderstandings rather than to a clarification. A negative attitude both on the part of the

scientific investigators and of their military coworkers tends to emphasize the inabilities and

deficiencies of military systems and the nonapplicability of studied solutions. This negative

attitude usually ar ises in the scientist because it is generally simpler to show when and where a system will not function than it is to investigate the regions where the system performance is

applicable, o r to extend the system so that it performs adequately where it had not functioned

originally. The negative attitude ar ises in the military when analyzing a solution to a problem,

because it is often easier to discard a solution on the basis of nonapplicability in certain regions,

than it is to understand when and how to apply the solution. It must be emphasized that a nega-

tive or a positive attitude is not to be considered as good or bad per se , but that trouble, inef-

ficiency, and wasted effort occur when the attitudes of the two groups a re considered to be

mutually exclusive. Conversely, mutual compatibility and understanding of common goals can

be tremendously effective. We need only note the development of the first atomic weapon

system.

Evaluation Criteria

To design a system such as a military organization, its purpose should be known and

the imposed restraining conditions should be known. Such a clear problem statement, as was

pointed out above, is unusual. More often, systems are designed to satisfy multiple purposes,

some of which may conflict. To determine how well a system accomplishes a purpose requires

a criterion to measure o r to evaluate.

Often in system design studies, before the problem is clearly defined, considerable

effort is expended to formulate a single criterion for evaluating a system or for choosing one

from several candidate systems. Sometimes a single criterion exists for a simple system or is justified by the study of a simplified model. Usually, one must consider several cri teria,

some of which may compete with each other. Should sufficient knowledge of the problem be gained, then these cri teria may be functionally combined with weighting functions to obtain com-

posite criterion. The weights (weighting functions) may be functions of variables appearing in the problem and, therefore, implicitly, of the cri teria themselves. This leads to a form of 2 posteriori o r feedback determination of criteria. An advantage derived from separating the cri teria is that their weights (should they be known) may be reassigned to meet new problem

specifications. A disadvantage is that the solution is less compact and may be more difficult

to understand. If, as is often the case, these weights cannot be defined, then the system must

be evaluated on the basis of multiple cri teria tabulations. The solution, then, is a function of

218 PAUL BROCK, WALTER D . CORRELL, AND GEORGE W. EVANS I1

the cr i ter ia which may be representable only in tabular form. To better understand these com-

ments, consider some rather broad criteria for logistical systems as an illustration.

Logistical systems (organizations) are required to provide mater ia l support for opera-

tional troops. Evaluation c r i t e r i a or measures of logistical systems may be of the following

types:

1. Cri ter ia pertaining to logistical performance

2. Cri ter ia pertaining to logistical efficiency

3 . Criter ia pertaining to logistical economy (generally an imposed restrictive condition)

4. Criter ia pertaining to operational effectiveness.

The cr i ter ia concerned with performance reflect how completely the system c a r r i e s out- its tasks.

Those c r i t e r i a concerned with efficiency reflect the amount of system capacity used in perform-

ing the tasks. Examples of such criteria are the ratio of the t ime used to the t ime required to

complete the tasks , the ratio of equipment and personnel hours used to those available, etc. The

criteria concerned with economy reflect the cost of operation of the system. The costs may be

subdivided into initial cost of equipment, maintenance costs, personnel salaries, operational

costs, and costs of training personnel. These costs are generally peacetime-imposed r e s t r i c -

tions. For wartime operations they have little meaning. Wartime costs are considered

relatively - i.e ., percentages of national economy, percentages of maximum or effective national

production rates, etc. The most important cr i ter ia to consider f rom a mili tary point of view are

concerned with the combat effectiveness of the units under consideration. Logistical troops and

supplies are of necessity a drag, operationally. This is similar to the initial gas load of a long

distance plane. Too much fuel and the plane won’t get off the ground.

That these classes of criteria are not easily unified into a single cri terion may be seen

by considering academically the evaluation of limiting cases for a medical system. At one limit,

the medical system has no personnel or equipment. Thus, it has minimal costs and may be con-

sidered trivially to use its zero personnel and equipment efficiently - i.e., either all that it possesses is used or none of its possessions are unused. However, such a system will do little

towards evacuating and treating casualties. Its effect on operations, while freeing the unit of a burden, imposes a crit ical morale factor, with troops performing a medical function themselves, inefficiently. Thus, this situation would probably be poor, operationally. At the other extreme,

consider a medical system staffed with at least one doctor, one ambulance and necessary medical equipment and supplies for each casualty. Such a system should show a good performance,

but certainly be inefficient and uneconomical. It would represent a heavy burden on the tactical

operation. AS one moves from the l imits into the range of real ism, the polychotomous criterion behavior remains. Conceptually, one may display the criteria effects graphically as shown in

Figure 2. A choice of positive weighting constants for a simple additive combination of c r i t e r i a

into a single composite cri terion would cause one to accept one or possibly several ca ses as a result of experimental measurements. However, this composite cri terion will not have strong

discriminating powers and is apt to change significantly with perturbations of the weights

selected. One suspects, then, that proper weightings are complicated functions of these criteria, themselves.


w 3 _I

> a

4

k a

n w

u

0

PERFORMANCE *

INCREASING MEDICAL UNIT S I Z E

* I T IS AXIOMATIC THAT PERFORMANCE DEGENERATES WITH LARGE NUMBERS OF OPERATORS.

F i g u r e 2 - Various c r i t e r i a v a l u e s as a function or' unit s ize

A judicious choice of cri teria requires and depends upon an understanding of the problem.

Should one be comparing various candidates for a system concerned with the transportation of

supplies, efficiency criteria that could be chosen should not only reflect whether or not proper

equipment has been used, but also whether proper schedules have been employed. Further, the

system should be understood so that constraints a re imposed which do not allow the considera-

tion of illegitimate shipments - e.g., combining shipments of gasoline and ammunition.

Integrated systems cannot be neglected. That is, one cannot look only at the scheduled

transportation of supplies, but one must also ascertain the interaction between scheduled and

emergency deliveries. As a further illustration, the interaction between the transportation

operation and those associated with supply depots must be ascertained.

Then, one must not fall into the trap of optimizing a partial system to the detriment of

the whole system. (As a corollary, only significant systems should be optimized.) The considerations of effectiveness cri teria for strategic and tactical operations would

differ from those suggested for logistics. Strategic cri teria certainly a re more closely inter- related with the action of the enemy. A broad plan of action must conservatively insure a posi-

tive maximizing of the force effectiveness against optimal enemy counteractions, o r calculatedly

risk a suboptimal enemy counter to increase this effectiveness maximum.

A defensive restatement, to minimize the effect of a maximum enemy effort, may be a consideration i f the nature of the military effort is to contain o r to neutralize rather than to

dominate. Classically, grand strategy has always been aggressive. It is of interest to note that the two considerations, though apparently complementary, are far from equivalent.

220 PAUL BROCK, WALTER D. CORRELL, AND GEORGE W. EVANS I1

In this discussion, the effectiveness of the enemy is considered in t e r m s of cr i ter ia

established by and for the friendly force. The enemy may very well have other cr i ter ia , hence

achieving the ends of the friendly forces may not thwart the enemy at all. If the inverse problem

is significant as it may be in the broad context of imposing one’s will on the enemy, additional

and possibly conflicting cr i ter ia must be established to determine strategic effectiveness.

Tactical operation evaluation may be more complicated as it involves human behavior

more directly. The c r i te r ia associated with performance must consider the command and con-

t ro l aspects of the operation. One is faced not only with the responsibility for choosing optimal

procedures for performing an operation within a given environment selected under stress, duress,

and within short , cri t ical t ime intervals, but also with the problem of whether o r not these pro-

cedures will be followed, and what the resultant effect of the choice will be for the subsequent

operation. Tactical cr i ter ia must be constructed to evaluate communications, f i re power, and

mobility of tactical units separately and in combination. Though the cr i ter ia may differ, the

general methods of analysis within this discussion can be used to achieve knowledge equivalent

to that discussed for logistical systems.

THE DIGITAL COMPUTER, A POWERFUL TOOL

Analysis, computer simulation, map and CPX war gaming, and field experimentation

methods of analyzing military systems overlap and interact. They should not be considered as independent techniques o r as implements for gathering data to be used in the other methods alone.

For best results, the methods must be properly mixed.

Consider a simplified tactical problem. There are two forces engaged in a battle, the

red force and the blue force. The first major simplification is to assume only two tactical pos-

tures , attack and defense. Also assume that i f the red force is on the attack, then the blue force

is on the defense, o r conversely, if the blue force is on the attack, then the red force is on the

defense. The model is further simplified by assuming that the attacking force has the initiative

in choosing each action and the defending force responds with a planned counteraction. Even

such a highly idealized model of an engagement leads to many difficult problems that have not

been satisfactorily investigated. This discussion will show how analysis, map and CPX gaming techniques, computer simulations, and field experiments can be employed together to obtain a

fuller understanding of these problems, and how the methods can be used to solve problems that are properly stated.

To initiate the investigation of the model stated above, an analytical technique is employed.

Assume that for a particular engagement, the attacking force can initiate N different actions and

the defending force can respond with any one of M different counteractions. All possible actions

a r e known to both sides. Now, consider a pay-off table for the defending force. The entries in Figure 3 indicate that we can assign some value between -1 and +1 for the desirability of employ-

ing counteraction

bounds of ineffective and effective counteractions, respectively, for defense, to an initiated action.

The model is highly unrealistic so far. Not much more is lost in assuming full force reconstitu- tion after each pair of actions.

against action E, 1 < n < N, 1 < m < M. The values -1 and +1 represent


ATTACK ACTION (Strategy)

Figure 3 - Defense counteraction as a function of attack action

In gaming theory, the various actions a re referred to as “strategies.” Thus, the counter-

actions may be referred to as counter-strategies. During the engagement between the two forces,

the attacking force will perform an ordered sequence of strategies, such a s 2, 2 , 5, 1, 11, 4, 2 ,

3, ..., which will be referred to as a policy. A counter-policy is then a sequence of counter-

strategies in response to the policy.

What questions might be asked of this simple-minded model, and what difficulties ar ise

in answering these queries ? First , one might ask, for each policy containing L strategies

(where repeats are allowed) what is the best counter-policy? The answer to this problem can

be answered by providing an automatic digital computer with the pay-off table and with a routine

that will systematically develop all policies for each value of L starting with L = 1 and proceed-

ing in sequence through the values of L = 2, 3, 4, ... . For each policy so established the com-

puter routine would evaluate a best counter-policy. By circling a best counter-strategy in each

strategy column of the table, it is clear that for any policy, one can glance at the value table and

deduce a best counter-policy. But this becomes a chore when one considers that the total num-

ber of policies for any L is NL. Here one has a glimpse of the value of the speed and computa-

tional reliability of modern large-scale digital computers.

For any policy, a best counter-policy is now at hand. As an initial difficulty, impose the

condition that the counter-policy must be planned before one knows what policy is to be used.

Our preselected counter-policy may not be the best for the policy chosen. How bad or good was

the choice? This implies a need for a measure of the value of a counter-policy against a policy. This would clearly be a function of the values in the strategy table. The analytic subject of gam-

ing deals precisely with this problem [ 21. Although the details will not be discussed here, some things are evident. First , certain counter-policies may be eliminated because others dominate,

o r a re consistently better. When N, M, and L a re large, many possible policies may be rwled

out as being absurd, others a s being impractical, etc. Thus, the actual number of policies that

can be used by the attacking force may be relatively small, compared to the total number of

possible policies. The problem is thus reduced to evaluating counter-policies only for a selec-

tive policies group.

One method of obtaining a set of policies to be used by the attacking force is to use a

gaming facility where the engagement is enacted many times by players. The frequencies with

222 P A U L BROCK, WALTER D. CORRELL, AND GEORGE W. EVANS I1

which certain policies a re repeated can be used a s a measure of their expected occurrence. The

counter-policies a re then evaluated for those policies which most probably may occur.

To complete the picture, even though not a realistic one, it is suggested that the values

entered in the pay-off table may be obtained from field experiments for each strategy pair.

Figure 3 is a pay-off table for the defending force and is used to obtain optimal counter-

policies for policies used by the attacking force. A similar pay-off table may be constructed for the

attacking force so that it might choose the optimal policy for attack against probable defending policies.

Each entry of the attackpay-off table need not be the negative of the corresponding entry of the defense

pay-off table since what is best for the attacking force need not be worst for the defending force.

The following a re ways in which this simple model of an engagement may be expanded so

that it can begin to approach reality:

1. The list of possible postures may be extended to reconnaissance, attack, defense, and

withdrawal (including subpostures for each of these). At the start of an engagement, the red

force is in one posture and the blue force is in another. As the engagement progresses, the

initiative is forced upon the side that will most benefit by a change in strategy and, thus, the

postures may change during the action.

2. It should be recognized that an action or counteraction taken at a particular time may

be related to past actions, to past counteractions, and to future planned actions and counter-

actions. The nature of these relations is often determined by the engagement. This, in turn,

may determine the method of analysis to be used.

3. The values entered into the pay-off tables may be expected to change for each suc-

cessive action a s a result of the previous action. One factor causing this is the depletion of

each force and its method of using its reserves.

4. The table entries themselves need not be single-valued. The values may be expect-

ancy distributions for each counter-strategy when applied to a given strategy. That is, the

expected pay off usually will have not only a mean value but will have also a probability distri-

bution over a range of pay-off values. With these considerations, a force may take an action or counteraction which has a poor mean value, and gamble for a higher pay-off expectancy.

These a re a few considerations, which, if included in the model, would cause it to take on more aspects of realism. However, their inclusion now takes the engagement into the realm

of exceedingly complicated problems. Computers a re now essential even to consider starting

an investigation; it is not too difficult to exceed the capacity of current equipment by including these additional complexities. All of the complications need not be important. That is, the

inclusion of a specific facet may not greatly affect the aspect of the problem being studied. This

raises an area of investigation for which a theoretical analysis can greatly affect the proper use

of computer simulation.

Analytic techniques and field experiments generally simulate continuous operation of a

system. A digital computer simulation of a map or CPX war game approximates the operation by a sequence of discrete steps. Analytic techniques a re usually restricted in the scope of

investigation of a system. Field experiment, surprisingly, introduces phasing and control e r rors .

These phasing and control e r ro r s arise from stopping certain units during the experiment so that


damage and casualty assessments may be assigned while other unit actions a re still in progress.

The time required for damage and casualty assessments is great enough that over-shooting often

occurs. Field experiments used to tes t a given system and its operation generally suffer from

insufficient observations to obtain statistically conclusive results. That is, if we pit two almost

equal organizations against one another the weaker may, by the majority of cri teria used, be

indicated as the stronger because of the relatively few times the systems are tested [3 ] . Large

field experiments cannot be repeated often enough because they a re expensive. Furthermore, a

simulated war environment seldom provides identical conditions in which to repeat the operation

of a system. An understanding of the necessity for making many repetitions of an experiment can be

gleaned from competitive sports where individual games seldom reflect the relative strengths

of the teams. For example, Team A may win the contest between Teams A and B, B may win

the contest between Teams B and C, and C may win the contest between Teams A and C. In a season, when all teams of the league have played each other, Teams A and C may win only 10

percent of their games and Team B may win 90 percent of its games. Similarly, games of luck,

such as coin tossing, provide a theoretical 50-50 chance of winning to each of two contestants.

Even though these contestants start with equal resources, the game i s usually of short duration,

ending with one contestant broke. Furthermore, such games require repeating many times for

each contestant to have won approximately 50 percent of the times played. Even i f the game is played a large number of times, the probability is very low of an even split at any particular

toss. Single experiments that provide conclusive results usually imply no contest or obvious

results. Thus, there ar ises the problem of comparing results obtained for the same operation

when studied by the different methods of analysis, and the ultimate problem of determining what

the implications a re for the real system. By itself, each method has weaknesses. In coordina-

tion they may produce more valid results. Exact results are not reasonable to expect, but quan-

titative, significant, probabilistically meaningful values may be expected. For example, the different methods of analysis may be used to obtain upper and lower bounds of performance for the

operation of a system - i.e., one method may yield a lower bound while another produces an

upper bound, and the third yields confirmatory data for each.

CLASSICAL WAR GAMING

Classical war gaming has a very old history about which much has been written [ 4 1. Its major uses have been to train military personnel, to preplay actual military engagements, and

a s a mental recreational exercise. The principal tools of the discipline a re the map (ranging from the chessboard box array to highly detailed, three-dimensional terrain boards) and the symbols representing the military elements, and their position and direction orientations.

The game usually has two sides, the friendly blue force versus the enemy red force.

Certain CPX (Command Post Exercise) games, involving a number of separate semi-autonomous

groups may be considered to be n-sided games, though they all have a common enemy. Games


range from rigid (for which all possible actions a re specified) to free (where the unit command-

e r s have freedom of action within only broad constraints).

The rules of the plays a re included in a scenario. The scenario defines a suitable

enemy red force, a battle area, and initial FEBA (forward edge of the battle area), several pos-

sible deployments of the forces, and logical objectives for each force. As will be explained, the

scenarios may be written through the use of the gaming facility.

With the complements of the red and blue forces established, attention is directed to a

map war-gaming facility that can serve the investigatory purposes. The equipment of this

facility may range from the crude to the sophisticated. The requirements of the map war-game facility a re dictated by the necessity of perform-

ing two-sided games even though there a re times when one-sided games a re desirable. A two-

sided map war game implies a team making decisions for the red force and a team making

decisions for the blue force. A one-sided game uses one team to make decisions for the blue

force, opposed by red force using predetermined actions. A team can consist of the command-

ing officer only or the commanding officer and a slice of his staff to assist him with the military

problems that a r e under study.

The information received by the red or blue team concerning its own and the enemy

forces is controlled by the scenario so that it may represent typical battle information, loaded

information, or limited information. For example, one may ask how well does a particular blue

force organization perform when the red force possesses complete knowledge of the blue force,

while the blue force obtains its knowledge of the red force only through reconnaissance missions

and direct contact with the enemy? Here, the red team possesses loaded information and the

blue force possesses typical battle information. Similarly, one may ask how well does the blue

force perform when following a preplanned-scheduled strike with communications restricted

according to a planned schedule ? Here, the blue team possesses limited information concerning

both forces.

A gaming facility may be used in conjunction with the writing of scenarios, with engage-

ments that a re to be further gamed, and with the performance of field experiments.

To initiate the writing of scenarios, critical objectives must be selected. These a re investigated through the development of individual scenes for which initial FEBA’s and initial dispositions of each force a re chosen. Then, using skeleton red and blue teams these engage-

ments a re map-played for each configuration. The object here is not to obtain performance

data but to determine parts of the engagement that a re critical or significant in the opinion of

qualified observers and evaluators, so that proper restrictions may be included in the finally

synthesized scenario to force the engagement into directions that will cause the organizations to m.eet the problems that have been selected for study.

Before field play is attempted, map play is used as a dry run to pretest field-data record-

ing forms, for the training of controllers and to catch gross incongruities that may exist in the

composite scenario. Map play of detailed scenarios should also stimulate questions that might be better answered by future field play. Thought should be given continually to field experiments

that may be designed about a map game.


FIELD EXPERIMENTATION

The laboratory to complement the other military science tools for the study of the army,

is the simulated battlefield. The simulation may range from a rifle range test to a full, two-

sided, scientifically controlled maneuver. The field experiment introduces a natural environment, real timing and real sequencing.

Though it cannot induce the motivations or emotions of a true battle, it removes considerably the abstract and academic unreality of a map game. To a certain extent, decision-making by

military personnel under combat s t resses can be studied.

Map gaming requires such fundamental quantitative information as battle rates of fire,

vehicular breakdown rates, vehicular speeds with respect to various terrain conditions, etc.

Distributions of these measures a re often derived from past histories, and from manufacturer’s

specifications, o r are recommended from experience. Field experiments, under varying degrees

of experimental control, offer opportunities to revise the distributions used for many of these

statistics, particularly with respect to differing system and operational environments.

A field experiment may be pure or mixed. A pure experiment considers a system’s

behavior as implemented in the field with only the personnel and equipment necessary to study

that system. A simple example would be to determine ambulance loading times for various

sequences of types of casualties. A mixed experiment may consider, in addition to the principal

system under study, one or more of the supporting or supported systems that constitute an armed

force. It may consider the operation of the investigated system under direct or indirect con-

tact with an enemy. A mixed experiment would be used if it is desirable to determine what

changes a re caused by the added environment in the ambulance loading times. These changes

may occur because of added confusion, additional tasks for ambulance and aid men, or because

additional precautions must be taken to protect the ambulance from the enemy.

A field experiment may be considered to be an enlarged map war game in which the map

is an actual terrain. Each team is represented by a fairly dense sample of the personnel

assigned to each force. The rules for operating a field experiment differ, however, from those

of a map war game, in that they must recognize the semicontinuous operation of the field experi-

ment rather than the discrete, time-updating of the map game. One would expect the control

room for a field experiment to be in the nature of a sophisticated control room for a map war-

gaming facility. In a map game, it is unusual for a controller to sit with either operating team. Informa-

tion to the control room is transmitted directly by the players, and instructions a re received by

them directly. In field gaming a field controller must serve as an intermediary. To maintain

the time-continuity, the players cannot be loaded with a control burden. Though necessary and

valuable, field controllers pose distinct field-experimentation problems of their own. The control room maintains contact with the field controllers.

The field experiment also produces a different type of military evaluation. Observers may see a composite picture in the control room, but the field observers have locally restricted

visibility. In combination, the two viewpoints can yield a deeper and more significant evaluation.

226 P A U L BROCK, WALTER D. CORRELL, AND GEORGE W . EVANS I1

The differing aspects of map war gaming and field-experiment gaming can and should be

exploited by the use of both methods to test systems.

Field experiments must be conducted with existing equipment. The use of existing equip-

ment to simulate future equipment introduces problems inherent in the experiment. Fo r exam-

ple, a system to be tested in the field may require a communication net involving radio equipment

that has not reached the field stage of development. However, its specifications and operating

procedures may already be known. This equipment may have pushbutton tuning, whereas the field

equipment may have dial tuning. The tes t equipment may have a frequency range permitting

twice as many channels as the field equipment yet it may require only half of the physical space

and power as that required by existing equipment.

In this case a CPX game being partially a field experiment and partially a map game

might be undertaken. Here, several units of the existing radio equipment are combined to repre-

sent the performance characterist ics of each advanced unit. Several of these combinations of

existing equipments would be t r ied to determine a best simulation of the communication system

for the future equipment. The simulating system would be tested under specified operational

procedures of the future system and then adjusted in a manner to best simulate the system in the

field experiment, considering space and channel requirements for the experiment.

Often the simulation of future equipment with current gear poses a much more complex

extrapolation problem. This is particularly t rue when one of the desired effects of the experi-

ment is to recommend rational operating procedures for the proposed equipment. To illustrate,

how does one simulate radio jammers, when it is not c lear whether the tactics of a future engage-

ment will use spot frequency jamming o r band jamming, and when the expected optimal duration

of such jamming is not known?

A valuable technique in field experimentation is to perform an experiment with simulated

equipment and then with existing equipment to help determine the effects of simulation. The

design of the field experiment must be adjusted to reflect these equipment anachronisms. In contrast to this, in a map war game, the simulation of existing o r future equipment is performed

per the stated specifications. Questions of equipment simulation do not arise.

A map war game reflects the interrelation between man and equipment in an indirect manner, i.e., through distributions for these events as acquired from other sources , whereas

the field experiment reflects the interrelation directly. Here again, we s e e a power and a weak-

ness of field gaming. If the interrelation is crit ical , the map game, using an assumed averaged

input, can completely invalidate the effort. Conversely, if the interrelationship is not critical, nor cumulatively critical, it can introduce undesirable complexities into the field action, that

can be eliminated in the map play. The trouble, of course, is that the significance is generally not known a priori . --

Field gaming is a complex operation. The synthesizing of operating scenarios from

scenes that themselves were generated to answer specific fundamental questions, was briefly

mentioned in the preceding section. Extensive detail and effort is required to design and test measurement fo rms and to make the recording of field data as automatic as possible (eliminating

as far as possible the human umpire problem). There are the final map dry-runs to optimize the


expectation of a successful field run. Finally, the execution of the experiment represents for-

midable tasks, only a few of which a r e s imilar to the tasks of the mili tary in preparing a force

for battle.

The values and distributions for events acquired from the field experiment are used in

replays of the map war games to extend the resul ts of both the map game and the field experi-

ment. The extensions can occur in several ways. The distributions for events occurring in the

field experiment must be adjusted for equipment designed for the system but not available to the

experiment, and for the consequent anomalies in tactics. More engagements can be produced in

map play based on these distributions s o that more conclusive evaluations of the cr i ter ia may be

attained. Map play is used as a post-analysis procedure for overall mili tary evaluation by

repeating the field experiment as many times as is necessary. Post play is the heart of the

mili tary evaluation of the problem. Besides direct results, a major contribution of a post-play

analysis is to determine areas for subsequent field, map, computer, and analytical efforts.

There is always the question of effect induced upon the experiment by the presence of

controllers. Map gaming can be used to estimate the magnitude of this effect.

It has been stated in this discussion that a field experiment should be pre-map gamed

a s it is to be executed in the field, and post-map gamed using field resul ts before extending the

map gaming to situations that did not exist in the field. The feedback generated can and should

be used to improve computer simulations and to extend analytical studies.

In considering computer simulation, map war gaming, and field experiment technique,

there is a basic operational problem that is seldom recognized as such, and for which there is

no a pr ior i solution. The problem is concerned with the game interval, the elapsed t ime between

the t imes that the game is updated, and the discriminating time interval that allows two events

to be simultaneous should their t imes of occurrence differ by less than this interval. The dis-

criminating time difference occurs in real life since it represents the accuracy with which time

measurements are made. It also has that meaning in field experiments and may be associated

with the assessment time interval (the time required to make an assessment of damage or casualties), since the entire experiment is not stopped each time an assessment is made.

Usually game intervals are not used explicitly in field experiments.

-- -~

Let the game interval be AT and the discriminating time difference be At. The basic operational problem involves the following two questions:

1. Should games be advanced by a constant value for AT; and if so, what value should be chosen ?

2. How large can At be made without changing the evaluation of cr i ter ia used in games under consideration ?

Many choices and methods of choosing AT and At a r e presently in use and heurist ic

and philosophic arguments are given for their validity. Analytical solutions for these questions should be sought. At the same time, one can design map games and field experiments for the sole purpose of shedding light on and possibly answering these questions. Thus, one may ask for

what values of AT and A t is a field experiment an approximation to a map war game and vice versa , o r is a map war game an approximation to a computer simulation, o r are other


combinations approximations to each other ? Here there is raised one issue of the general prob-

lem of using the proposed techniques for the study of these techniques themselves and for the

development of new techniques.

CONCLUDING REMARKS This article has discussed four techniques that may be employed by the military for more

scientific design of military systems. The methods a re analysis, computer simulation, war gam-

ing, and field experimentation. It has been stressed that the best results can be achieved by using

these tools adjunctively, rather than independently. The methods can be used to determine abso-

lute results, but a more powerful concept is to use the procedures iteratively. The results of an

investigation should lead to new, more critical, more refined investigations. The philosophy is

to continually improve structure, organizations, and tactics. The conditions under which Organi-

zation A is better than Organization B, or conversely, a r e important. But to be able to develop

a still better organization, C, is even more rewarding.

It must be emphasized that results obtained by these techniques cannot be considered

solutions in any categorical sense. When problems are stated intuitively, and the methodology

is largely heuristic, the most that can be expected of results a re attenuated statistical reliabili-

ties. These, however, can be much better than military guesses.

The methods range from the use of the abstract symbolism of analysis to the pseudo-

realism of field experimentation. The lack of imparting true realism, even to field experimenta-

tion, is often considered a failure of these methods. However, ascertaining what realism is dur-

ing a war is a nebulous task. Considering hot war engagements as investigating tools, can more

information for the development of military systems be obtained? The answer is probably a conditional affirmative - conditional according to the usage of techniques such as have been dis-

cussed to answer the question of how a hot engagement can be utilized, and conditional also

according to the use of the true engagement to complement the other tools for best results.

This a rea of investigation has not been discussed because, although essential and vital,

it is a more advanced technique and must await an understanding based upon experience with the more passive methods.

In field experimentation, the importance of gathering system performance data with minimal disturbance to the system is a prime consideration. Automation of data collection i s a means toward this end. As the necessary types of data that must be measured for the evaluation of military systems a re singled out, efforts should be made to automate the acquisition of this

data. Having both automatic methods of gathering data, and automatic procedures for evaluating it, the military will then be in the position to improve military systems during a war situation

from data that can be acquired during battles. Although it has not been stressed, the methods described allow for the vital investigation

of augmentation procedures of armed forces for or during a war. Should one increase the size of a platoon, increase the number of platoons in a company, increase the number of companies

in a division, increase the number of divisions in a field army, or revise the whole structure ?

The inverse question - the decrease of a force - might also be asked for the purpose of


establishing striking forces o r for other special missions. The military must learn to use

these techniques to develop “best” military postures against all possible enemy configurations.

They must learn what every major football team knows - how to quickly and efficiently shift

from one formation to another with their large, but still limited, resources. A most critical

shift is rapid conversion from a peacetime to a wartime force. A more subtle shift involves

anticipating national and military economy transitions.

Although it is unlikely that a best military system for war will ever be known, it is the

function of the military to provide the best systems it can devise for military conflicts any

place and at any future time. This is an awesome responsibility requiring a critical balance of

effort expenditure of basically limited personnel and natural resources. The only compensation

is in the form of the efficient and skilled use of scientific investigating techniques.

ACKNOWLEDGMENTS

The authors a re indebted to Col. Donald McB. Curtis, GS, and Col. Charles E. Tegtmeyer,

MC, for assistance and encouragement during the development of the ideas herein reported.

BIBLIOGRAPHY

Bellman, R., and Brock, P., “On the Concepts of a Problem and Problem Solving,” Am.

Math. ASSOC., 67, 2 , 119-134 (1960).

Williams, J. D., The Compleat Strategyst, Being a Pr imer on the Theory of Games of

Strategy (McGraw-Hill Book Company, Inc., New York, 1954).

Fend, A. V., Swallow, K. P., and Penick, J. J., “Measures of Combat Effectiveness in

Small Duels,” Research Memorandum RO-RM 13, Research Office, U.S. Army Combat

Development Experimentation Center, Fort Ord, California (June 1961).

Young, J. P., “A Survey of Historical Developments in War Games,” Operations Research

Office, the Johns Hopkins University, Bethesda, Maryland (August 1959).

* * *

Techniques for evaluating military organizations and their equipment

Documents

Transcript of Techniques for evaluating military organizations and their equipment