Factors Affecting DOD FM Usage of a GDSS

Jeff Bohler & Dianne Hall, PhDAuburn University

Pre-ICIS 2006 SIGDSS Research Workshop10 December, 2006

The opinions and observations expressed in this presentation are mine alone and do not represent the opinions of the DOD, the USAF, or the USN.

10 December, 2006 2

Overview

Research Motivation & Relevance Decision Support Method GRASP Research Model & Hypotheses Methodology & Analysis Results Limitations & Future Research

10 December, 2006 3

Research Motivation & Relevance

Changes in the DOD workforce Personnel reductions Retirement of “Baby Boom” generation Shortage of analytical skills in FM

Financial Manager & Comptrollers School Decision Support concept introduced

What makes this study interesting? Decision Support training effectiveness Professional population Many IT, GSS, DSS changes underway in DOD

10 December, 2006 4

GRASP - BackgroundGroup Resource Allocation Simulation Program

Course includes “Strategic Economics” Macro view of political, military, and economic

relationships

GRASP Usage Capstone exercise, used in both curricula

Decision Task National level budget decisions Group oriented (president & cabinet) Face to face, synchronous, no “right” answer Multiple objectives to optimize

10 December, 2006 5

GRASP – As a GDSSGroup Resource Allocation Simulation Program

Contains a body of knowledge (DIKW) Data + Rules + Relationships + Context =

Knowledge Record keeping ability

Displays data history (17 variables / 4 turns) Provides standardized reports to the decision

maker Allows decision maker choice in KM activities

Enables decision support from “cabinet” Provides estimates of likely outcomes from

decisions => new knowledge

10 December, 2006 6

Relevant Literature Carroll & Johnson (1990)

Temporal stages of decision making

Nunamaker, Dennis, et al. (1991) GSS Research Framework

Zopounidis et al. (1997) Survey of DSS in financial management Proposed a “Knowledge Based DSS”

Hall & Paradice (2003) Adapted Simon’s IDC model, incorporating KM

10 December, 2006 7

GRASP Decision Support Method

Identify Issue Analyze Issue Develop Alternatives Evaluate Alternatives Make a recommendation

DS provides a framework for FM&C course Addresses need for DS training in DOD FM Congressional call for improved FM practice

10 December, 2006 8

Research Hypotheses

H1: Groups that only use FtF communication methods will achieve greater decision quality than groups that use hybrid proximity (FtF & paperless).

H2: Groups with higher complexity tasks (wars) will achieve lower decision quality than groups that have lower complexity tasks.

H3: Groups that display greater adaptation of GRASP capabilities (intelligence gathering) will achieve greater decision quality than groups with a lower rate of adoption.

H4: Groups that used DS methods will achieve greater decision quality than groups that did not.

10 December, 2006 9

Analysis Model

Communication

Complexity

Adaptation

DS Method Used

Outcome (DQ)

Goal Accomplishment

H1

H2

H3

H4

10 December, 2006 10

Methodology Subjects (~850) randomly assigned

Seminars balanced for military/civilian/job Each seminar was split into two teams:

President is self selected President picks cabinet (4 – 6 members) Resulted in n=172 teams

Competition A faculty facilitated exercise = lab experiment

Four years of exercise data Used for debriefing, student feedback

10 December, 2006 11

Analysis

Linear Regression Model: AR2 = .291 Decision Quality = Communication + Complexity +

Adaptation + DS Model Use

Operationalizing Decision Quality… Analysis of variance (ANOVA):

Unequal cell sizes Brown-Forsythe test (constancy of error

variance)

10 December, 2006 12

Results (at α= 0.05)

H1: Communication – Not supported Beta = -0.097, p = 0.228

H2: Complexity – Supported Beta = -0.532, p = 0.000

H3: Adaptation – Not supported Beta = 0.540, p = 0.410

H4: DS Model – Supported Beta = 0.170, p = 0.034

10 December, 2006 13

Limitations & Future Research Observational study

Hard to know cause & effect relationship Other possible explanatory factors

More detailed analysis required Content analysis of user reports will allow a

better understanding of the decisions made, priorities, & tradeoffs.

Gather more data & use PLS Future research

The decision making environment of DOD financial managers

10 December, 2006 14

Summary Simulations useful, but limited:

Allows isolation of decision behaviors (individual, group, organization)

Limited generalizability to other groups

Need more research on decision support Role of financial managers in decision making Effectively decision support training methods

Questions?

10 December, 2006 16

GRASP Exercise Model

Decision

Quality

Score

Decision

Maker

Environmental Factors:

- Competition Stress

- External Distractions

- Algorithm Anomalies

Personal Factors:

- Computer Literacy

- Education Level

- Military / Civilian

- Rank

- Gender

Decision

Support

Group

Class Factors:

- Class Timing

- Class Size

- Class Character

- Year Attended

Instruction Factors:

- Instructor

- Course type

- Decision Support Concept

- Decision Support Methodology

Decision

Support

4th year

Score

Bonus

Points

Feedback

- Internal- External

Group Factors:

- Group Size

- Group Character

Task Factors:

- Complexity

+Wars

+Communications

- Time Available

- Resources Available

10 December, 2006 17

GSS Research Framework(Nunamaker, Dennis et al., 1991) adapted

Group

Task

Context

DSS

Intervening &

Adaptation Process

Outcome

Decision Group Environment

Organizational Decision-Making

Environment

Feedback

n ~ 850 Randomly Assigned

Controlled:

•Instructor

•Course

•Phase

H2:Complexity

H1:Communication

H4:DS Methods Used

H3:Adaptation

GRASP

DV:

Decision Quality

Next Phase / PLS