CONSIDERATIONS IN ACQUISITION LESSONS-LEARNED SYSTEM DESIGN.pdf

Considerations in Acquisition Lessons-Learned System Design

67

LESSON LEARNED

CONSIDERATIONS INACQUISITION LESSONS-LEARNED

SYSTEM DESIGNKeith F. Snider, Frank J. Barrett, and Ram Tenkasi

This article describes issues affecting the design of lessons-learned systemsfor defense acquisition organizations. It draws both on studies of existing lessonslearned systems and the literature of organizational learning and knowledgemanagement. The exploration of these issues suggests that attention to socialprocesses within organizations is as important, if not more so, as thedevelopment of information technology processes in the success of lessons-learned systems. The articles conclusions can assist in the determination ofappropriate requirements and resources for an acquisition lessons-learnedsystem.

resources that give it a competitive edge(Davenport, 1997a; Zack, 1999a).

Amid this growing interest, a variety offorms of LLS have emerged, from staticdatabase collections of lessons (U.S.Army, 1997), to groupware tools (e.g.,Lotus Notes) for electronic collaboration(Davenport, 1997c), to heuristics-drivenexpert or intelligent systems (Aha,2000). Yet, as Zack (1999b) notes, verylittle research has been done regarding themost appropriate LLS form for a particu-lar organization. With an inadequate under-standing of design issues and alternatives,leaders may implement systems that fittheir organizations poorly.

Defense acquisition presents particularchallenges for design of LLSs. Particularly

T hough the idea of learning from ex-perience is timeless, formalized sys-tems for capturing and disseminat-ing lessons within organizations havereceived increased attention in recentyears. Interest in such lessons-learnedsystems (LLSs) has grown through thepopularization of concepts such as thelearning organization (Senge, 1990),through developments in knowledge man-agement (Davenport & Prusak, 1998;Nonaka, 1991), and through informationtechnology advances that hold out thepromise of wider, more efficient distribu-tion of lessons within an organization. Thisgrowth is especially evident in the privatesector, where a firms learning capabili-ties and knowledge are viewed as strategic

Report Documentation Page Form ApprovedOMB No. 0704-0188Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE

2002 2. REPORT TYPE 3. DATES COVERED

00-00-2002 to 00-00-2002

4. TITLE AND SUBTITLE

Considerations in Acquisition Lessons-Learned System Design 5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Naval Postgraduate School,Monterey,CA,93943

8. PERFORMING ORGANIZATIONREPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITORS ACRONYM(S)

11. SPONSOR/MONITORS REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution unlimited

13. SUPPLEMENTARY NOTES

Acquisition Review Quarterly, Winter 2002 14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT

Same asReport (SAR)

18. NUMBEROF PAGES

18

19a. NAME OFRESPONSIBLE PERSON

a. REPORT

unclassified b. ABSTRACT

unclassified c. THIS PAGE

unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

Acquisition Review Quarterly Winter 2002

68

The term lessons-learned systemrefers to the activi-ties, people, andproducts that sup-port the recording,collection, anddissemination oflessons learned inorganizations.

with major weapon systems, acquisitionis a highly complex enterprise that encom-passes multiple contexts those of poli-tics, business, technology, and the military,to name a few and multiple stakehold-ers with often competing interests (Mayer,1991; McNaugher 1989). Its processes areidiosyncratic (i.e., different parties impro-vise different solutions to unique chal-lenges) and contextual (i.e., differentprojects pose different kinds of chal-lenges). Acquisition also has a highly in-terdisciplinary character in that it entailsthe integration of a broad range of techni-cal and management skills, including con-tracting, system engineering, finance, and

many others(Fox, 1988).Thus, success-ful acquisitionprofessionalsmay be viewedas knowledgespecialists, inthat they haveunique experi-ences in solv-ing specialized

kinds of problems. They master a com-plex kind of knowledge in which theyimprovise solutions to challenges forwhich procedural knowledge offerslimited guidance. How might such knowl-edge be transferred to others in defenseacquisition?

This article focuses on two questionsto address this issue:

What design aspects and issues are in-dicated by analysis of existing LLSs?

What do organizational learning andknowledge management concepts

suggest are key aspects of LLS designfor acquisition organizations?

This article begins with an overview oforganizational learning concepts to estab-lish the intended benefits of LLSs. It thendescribes operation and characteristics ofLLSs to develop a set of factors and issuesto be considered in design. Next, it turnsto the literature of organizational learning,communications, and knowledge manage-ment to describe issues relating to howorganizations learn and how they transferknowledge among their members. Thearticle concludes with some specific im-plications for LLSs in acquisition organi-zations. Although the discussion focuseson defense acquisition, the conclusionsextend to lessons learned and knowledgemanagement systems in other areas aswell.

LLS

The term lessons-learned system in thisarticle refers to the activities, people, andproducts that support the recording, col-lection, and dissemination of lessonslearned in organizations. These systemsmay focus on negative lessons of fail-ures, deficiencies, and other problems tobe avoided, or on positive lessons ofinnovative techniques and best practicesto be emulated. Definitions of lessonslearned vary. Although the Army definesthem as validated knowledge and expe-rience derived from observations and his-torical study of military training, exercises,and combat operations (U.S. Army, 1997,p. 1), in the Marine Corps they are pro-cedures developed to work around short-falls in doctrine, organization, equipment,


69

Organizationallearning is usuallydefined in terms ofmembers learningfrom each other.

training and education, and facilities andsupport (U.S. Marine Corps, 1994). Mostsystems in current use are web-based.

INTENDED BENEFITS ORGANIZATIONAL LEARNING

Organizational learning (Argyris, 1999)is usually defined in terms of memberslearning from each other, that is, whenmembers share theories of action(Argyris & Schn, 1978) or mental mod-els (Senge, 1990). Lipshitz, Popper, andOz (1996) take a structural approach byfocusing on organizational mechanismsthat facilitate, make explicit, or routinizesuch sharing. These mechanisms are theinstitutionalized structural and proceduralarrangements that allow organizations tosystematically collect, analyze, store, dis-seminate, and use information that is rel-evant to the effectiveness of the organiza-tion (p. 293). Such mechanisms includeorganization histories, project reports,after-action reviews (Busby, 1999) andmore generally, LLSs. These mechanismsare intended to allow an individuals learn-ing to become recorded in an organiza-tions documents, processes, and othermemory media in such a way that othermembers may learn from it, thereby con-tributing to improved effectiveness orfacilitating an organizations adaptation toa changing environment (Argote & Mc-Grath, 1993; Myers, 1996). Many see theidea of a learning organization as inextri-cably tied to an organizational culture thatreflects such sharing through honest andopen communication (Cook & Yanow,1993; Schein, 1985).

CENTER FOR ARMY LESSONS LEARNEDOne of the earliest and best-known

LLSs is the Center for Army Lessons

Learned (CALL) at Fort Leavenworth, KS,established in 1985 for the purpose of col-lecting lessons learned during simulatedcombat training exercises (U.S. Army,1997). Over the years, CALLs mission hasexpanded to encompass lessons fromactual combat and other military opera-tions (e.g., JUST CAUSE in 1989). CALLalso employs dedicated expert observerteams to collect lessons from selectedhigh-priority operations. CALL is staffedwith resources necessary to accomplish avariety of lessons-learned functions, in-cluding collection; analysis; processing;dissemination;archiving; andresearch. It pub-lishes tailoredlessons-learnedproducts in awide variety ofmedia, includ-ing newsletters, handbooks, bulletins, andthe Internet (including both secure andpublic on-line databases).

LLS CHARACTERISTICSZack (1999a, pp. 4849) describes the

general sequence of operation of LLSs:generation, submission, processing, anddissemination. Specific aspects of designand operation vary widely. Table 1 adaptsAhas (2000) characterization, developedfrom an analysis of existing LLSs, togroup system characteristics under lesson,operational, and organizational factors.

Lesson factors describe the productof the system, that is, whether it produceslessons only (pure) or includes other prod-ucts such as best practices or informationupdates (hybrid). The other lesson factordescribes the type(s) of processes ad-dressed by the lesson or other product.


70

Technical processes usually deal withscientific, engineering, or other highlytechnical matters. Administrative pro-cesses usually involve fairly routine pro-cedures or decisions made by a singleindividual, for example, a purchasingspecialist. Planning processes entail morecomplex and strategic matters involvingmultiple stakeholders. CALL, for ex-ample, focuses mainly on tactics, tech-niques, and procedures for operationalforces rather than on macro-issues orstrategic operations; hence, its processeswould be classified as technical.

Operational factors describe how LLSsfunction. Access refers to the extent towhich those outside an organization mayuse its system. Open systems may beaccessible to the general public, whereasclosed systems have security features thatlimit their use to members of the organi-zation. Formal systems have establishedprocedures and processes of operation,such as those described for CALL. TheU.S. Geological Survey has taken a moreinformal approach in generating lessons

that are based on analysis and synthesisof the findings of more than 250 scientificstudies of the environment (U.S. Geologi-cal Survey, 1995). These assist local natu-ral resource managers in policy makingand decision making. Another example ofan informal system was the Navy Acqui-sition Reform Offices (ARO) recentChange Through Ex-Change Initiative.Every three months, ARO solicited acqui-sition organizations to provide two ap-proaches, ideas, process innovations, orlessons learned, which were distributed viadiskette and posted on the ARO web site(U.S. Navy, 1999).

CALL is an example of a centralizedLLS that serves the Army worldwide fromits offices at Fort Leavenworth, KS. TheDepartment of Energy (DoE) operates adistributed system with a networked in-frastructure of systems and lessons-learned coordinators at various sites andcontractor facilities. LLSs are embeddedif they operate in an integrated fashionduring other organizational activities, asin the case of Army units conducting after-

Table 1.Lessons Learned System Characteristics (adapted from Aha, 2000)

Lesson Content Pure HybridProcess type Technical Administrative Planning

Operational Access Open ClosedFormality Formal Ad hocLocus Centralized DistributedProcess relation Embedded StandaloneAcquisition Active PassiveHandling Rigorous OpenDissemination Active Passive

Organizational Interpretive context High Medium LowType

Adaptable Rigid


71

action reviews in the course of trainingexercises (Baird, Holland, & Deacon,1999). Embedded systems usually featureactive acquisition and dissemination(pull and push) of lessons, whereasstandalone systems wait for user inputand retrieval of lessons. The U.S. MarineCorps Lessons-Learned System (MCLLS)relies heavily on decentralized reporting(i.e., passive acquisition) from unit after-action reports of exercises and operations.

Handling refers to the level of treatmentgiven a lesson after it has been generated.Rigorous handling implies significant con-trol through some review and approvalprocess, whereas open handling implieslittle or no control of lessons. Fundamen-tally, handling involves decisions as towhether one individuals learning, as re-flected in the lesson, should be shared withothers.

Questions that arise here may include:Does the information in the lesson needto be verified, substantiated, or validated?Is additional information or discussionnecessary to make it understandable toothers? Does it sufficiently describe con-text and circumstances so that other mem-bers of the organization can judge thelessons relevance under differing condi-tions? Is it consistent with organizationalgoals and policies?

For example, CALL includes in its pro-cess of lesson development a coordinationstep to solicit comments from agencies andcommands that may be affected by or haveinterest in dissemination of a lesson. TheMCLLS features a process of lessons-learned reviews by various working groupsand committees, which assign responsibil-ities for analysis, action, and disposition.DoE lessons-learned coordinators, amongtheir other duties, perform a validation

function before a lesson is submitted forpublication.

Two organizational factors may be con-sidered when determining how handlingshould occur. Interpretive context (Zack,1999a, p. 50) refers to the extent to whichmembers of an organization share similarknowledge, backgrounds, and experi-ences. In an or-ganization witha high interpre-tive context,most membersare likely to un-derstand thecontent and sig-nificance of lessons generated by othermembers. Lessons generated in an orga-nization with a low interpretive contextmay need to include more detail in termsof description and explanation, and mayneed to be translated during handling forbroader understanding.

The other organizational factor to beconsidered is how rigid or adaptable anorganization is in terms of changing itshabits of action in response to lessonslearned by its members. An organizationmay have a culture that inhibits its abilityto change, or it may be constrained bylaws, professional standards, or by otherorganizations. Such constraints indicatethe potential need to review, validate, andperform coordination on lessons beforethey are disseminated to and shared withthe rest of the organization.

LLS ISSUESThough the benefits of knowledge man-

agement systems are widely accepted,their success is not guaranteed. One of themost significant pitfalls, according to Dav-enport (1997b), is the if you build it, they

Handling refersto the level oftreatment given alesson after it hasbeen generated.


72

will come fallacy. That is, merely imple-menting an LLS does not ensure that mem-bers of an organization will use it, eitherto generate lessons or to seek out thoselearned by others. Reasons for such lackof use are usually attributable to issues ofmotivation or organizational culture. In-dividuals may simply not have time togenerate lessons after a learning experi-ence, or perhaps they feel unwilling to

acknowledgethat problemshave occurred.Some who arefacing new sit-uations may beunwilling toseek out les-sons learnedby others if

they feel their problem is unique and notamenable to solution by past methods.Such participation issues may be ad-dressed through a championing of thesystem by the organizations leaders orthrough rewards and incentives designedto institutionalize use of the system(Fulmer, 1999).

The effectiveness of an LLS might alsobe affected by the substance of lessons,particularly if handling is not rigorous. In-dividuals may generate lessons contain-ing problematic information such as un-substantiated opinions, controversial find-ings, or self-serving claims, to name but afew. They may be poorly written, perhapswith little background or context thatwould allow others to judge the wider ap-plication, or with too much detail thatbores or confuses readers. Such problemspoint out the need for some degree of rigorin handling.

Clearly, too much rigor in handling maysquelch participation. Processes of review,editing, validation, and approval may be-come so burdensome that organizationalmembers lose interest in submitting les-sons. This indicates the need for LLSs toinclude some feedback mechanism so thatthose involved in handling can keepmembers apprised of the status of theirsubmissions.

Finally, LLSs require maintenance. Forexample, databases need to be reviewedfor outdated content, and periodic up-grades may be needed to incorporate newtechnology. Of course, such maintenancerequires resources, which means that LLSsmust compete with other organizationalprograms for scarce resources. The fail-ure of leaders to provide adequate re-sources may be perceived as a lack oforganizational commitment, leading tolow participation levels.

SUMMARYThe foregoing discussion has illustrated

a range of alternatives available to design-ers of LLSs. This range indicates the needto examine several different factors of anorganization and its knowledge needs toarrive at a system design that fits theorganization well.

KNOWLEDGE TRANSFER INDEFENSE ACQUISITION

Putnam, Phillips, and Chapman (1996)described several differing perspectives onorganizational communications held bymanagers and theorists. Probably the mostubiquitous of these is the conduit per-spective (Axley, 1984), in which com-munications are thought of as objects or

The effectivenessof an LLS mightalso be affected bythe substance oflessons, particularlyif handling is notrigorous.


73

transmissions that flow through organiza-tional channels from a source to a receiver.Such a view is often reflected in a mana-gerial emphasis on communications tools(e.g., information systems) and techniquesto enhance the speed, efficiency, andaccuracy of knowledge flow within anorganization (Eisenberg & Phillips, 1991).Against the conduit view is the idea ofknowledge as socially constructed (Berger& Luckman, 1967), or put another way,as the product of interpersonal relation-ships. This constructivist perspectiveemphasizes social processes that lead toknowledge creation and sharing (Putnamet al., 1996). Giddens (1979) theory ofstructuration the production and repro-duction of social systems through mem-bers interactions has been particularlyinfluential (Orlikowski, 1992; Poole &DeSanctis, 1990). For example, Orli-kowski and Yates (1994) describe how,over time, a distributed group of knowl-edge workers produced a rich, varied, andchanging structure (genres) of commu-nicative practices. From such a view,organizing is not a one-time event, butrather a continual process carried outthrough the social interactions of members(Weick, 1979).

Recognizing this distinction has pro-found implications for the design of LLSs.From the conduit perspective, one maydesign a system to emphasize the ease oflesson input and extraction by organiza-tion members, whereas from the construc-tivist view, one may focus instead on thetexture and development of interpersonalrelationships within an organization. Thefollowing sections draw on research fromseveral fields organizational learning,knowledge management, and organiza-tional communications to explore such

issues in greater detail. This is not intendedto be a comprehensive synthesis of the lit-erature, but rather a leveraging of selectedresearch to shed light on LLS designissues.

KNOWLEDGE COMPLEXITY ANDINTERDEPENDENCE

Theorists of knowledge managementhave distinguished between different kindsof knowledge. These determine the mosteffective method of transfer. For example,learning how to solve an algebra problemcan be understood via the transmission ofabstract concepts. Learning carpentry, onthe other hand, involves purposeful activ-ity and experimentation, a kind of learn-ing that is beyond verbalization. Many(e.g., Nonaka, 1991) have built on thework of Polanyi (1966) to create a distinc-tion and a continuum between explicit andtacit knowledge. Knowledge that is con-sidered complex or difficult to verbalizeor codify in writing, is tacit knowledge; itcan be acquired only through experience.Tacit knowl-edge is highlypersonal, deeplyrooted in action,and consists ofmental models,beliefs, and per-spectives so in-grained thatthey are often taken for granted and noteasily articulated (Argyris, 1999). Explicitknowledge is that which can be codified(Zack, 1999b). It is acontextual and cor-responds to banking concepts oflearning, which assume that knowledgecan be transferred from one party toanother regardless of context. Most stud-ies agree that explicit knowledge is easier

Theorists ofknowledgemanagementhave distinguishedbetween differentkinds ofknowledge.


74

to transfer than tacit, complex knowledge(Hansen, 1999).

Tyre & von Hippel (1997) emphasizethe situated nature of knowledge. Theyclaim that knowledge is not absolute, butrather is dependent on context and setting.Actors draw on codified, abstract theoryin their local, informal routines, but theyadapt them as they work on problemswithin particular circumstances. Lookingat the situated nature of learning raisesthe issue of knowledge representation.Organizations may fail to recognize the

complexity ofknowledge bydescribing jobsin simple, ca-nonical steps,which inhibithow managersinterpret andvalue the im-

portance of noncanonical practices. Actualon-the-job practice requires interpolationsbetween abstract knowledge and practical,situated demands.

Orr (1996) points out the dichotomybetween managers understanding of jobrequirements and actual practices: Al-though the documentation becomes moreprescriptive and ostensibly more simple,in actuality the task becomes more impro-visational and more complex (p. 42). Thispoint is illustrated in his study of Xeroxstraining of service representatives. Thetrainers attempted to document everyimaginable breakdown in copiers, so thatwhen technicians arrived to repair amachine, they simply referred to themanual and followed a predetermineddecision tree to perform a series of teststhat dictated a repair procedure. Trainersbelieved that a diagnostic sequence could

be devised to respond to predictable prob-lems. However, the study revealed that noamount of documentation could includeenough contextual information to makeevery problem understandable. Orr de-scribes a technical representative confront-ing a machine with error codes and mal-functions that were not congruent with thediagnostic blueprint. This machines mal-function did not fit the kind of errors thatwere documented nor had anything likethis problem been covered in his training.Both he and the technical specialist hecalled in to help were baffled. To simplygive up the repair effort and replace themachine would have been a solution, butwould have meant loss of face with thecustomer an unacceptable solution.After exhausting the approaches suggestedby the diagnostic, they attempted to makesense of this anomaly by connecting it toprevious experiences and stories they hadheard from others experience. After a 5-hour troubleshooting session of trials anderrors, they developed a solution.

Many jobs require this type of impro-visation a patching together of bits andpieces of experience to gain understand-ing of problems that do not provide de-finitive solutions. In Orrs account, thetechnicians go through constructing acoherent account of malfunction out of theincoherence of the data and documenta-tion. They go through a long storytellingprocedure, talking about the machineserratic behavior, their memories of othertechnicians stories, and information fromusers, which they try to put together in acomposite story. The process of formingthe story actually becomes an integral partof the diagnosis (Watzlawick, Beavin, &Jackson, 1967; Weick, 1979). This processbegins and ends with communal under-

Trainers believedthat a diagnosticsequence could bedevised to respondto predictableproblems.


75

standings that are not available fromcanonical documents; narration is an im-portant element in integrating the variousfacts of situation.

This suggests the necessity of informalinteractions between individuals for com-plex knowledge to be transferred. Mem-bers must engage in informal, unstructuredcommunications and processes of sense-making (Weick, 1979); discussion, nego-tiation, and argument are central to thelearning process. Daft and Lengel (1984)addressed the role of so-called rich media(e.g., face-to-face communications) in re-solving uncertain and equivocal situations.Such studies led Brown and Duguid(1991) to identify communities of prac-tice, small, informal, collaborative groupsthat generate a common understanding ofevents and a shared orientation for actingin the future. To foster learning, they con-tend, organizations must see beyond con-ventional, canonical job descriptions andrecognize how learning occurs in the richcontext of practice.

RELATIONSHIPS BETWEEN LEARNERSAccording to Lucas and Ogilvie (1999),

group heterogeneity is an important fac-tor in the dissemination of knowledge.Homogenous groups usually have accessto the same information and therefore offerlimited opportunities for learning. Poten-tial for knowledge transfer is increasedwhen members backgrounds and experi-ences are varied and complementary.However, the price for this richer learningpotential is the cost of maintaining rela-tionships among individuals who do nothave common experiences (e.g., greaterpotential for conflict).

According to social network theory,knowledge transfer works best in organiza-

tions with weak ties (i.e., distant and infre-quent relationships) among members. Gran-ovetter (1982) claimed that weak ties areefficient for knowledge sharing becausethey provide access to novel informationby bridging disconnected groups. Strongties, on the otherhand, lead to re-dundant infor-mation, becausemembers knoweach other well.Hansen (1999)argued, how-ever, that net-work theory isfocused on access to new information,which overlooks the issue of complexknowledge transfer. Weak ties, he claimed,indeed facilitate the processing of explicitknowledge, but they slow down the pro-cessing of complex knowledge, becausethere are fewer interactions for transferringcomplex knowledge. [T]ransferringnoncodified and dependent knowledge isless difficult to the extent that the partiesto the transfer understand each other (p.88).

In a similar vein, Fulk (1993) found apositive relationship between group mem-bers technology-related attitudes andbehaviors and their attraction to their workgroups. The implications are significantregarding productivity. In Hansens study,in which units were exchanging explicitknowledge, those with weak links (i.e.,connected only by e-mail) completed theirprojects 25 percent faster than those withstrong links. When tacit knowledge wasexchanged, however, units with weak linkswere at a disadvantage; they took 20percent longer to complete projects thandid units with strong links.

According to Lucasand Ogilvie (1999),group heterogeneityis an importantfactor in thedissemination ofknowledge.


76

TWO CASESAlthough many organizations in the

past decade have adopted informationsystems in attempts to enable knowledgetransfer, there are few empirical studiesthat actually document the enablers andconstrainers in these efforts. Most of theliterature is anecdotal and prescriptive.Two studies illustrate some issues thatarise when organizations attempt to im-plement lessons learned through theadoption of groupware systems.

Orlikowski (1993) studied an account-ing firm, Alpha Consulting, which adoptedLotus Notes, a documentary support sys-

tem with bulle-tin board, post-ing, discussiongroup, and elec-tronic mail ca-pabilities. Al-phas leaderswere concernedthat its consult-

ants, spread among different offices acrossNorth America, were working on similarproblems but not sharing their expertise.They hoped that Notes would provide asystem for storage and sharing of solutionsto a variety of problems.

Alphas junior consultants used Notesinfrequently. They seemed uninterested inlearning how to use the program and gaveup easily when faced with frustrations.Orlikowski found that the younger con-sultants had less incentive to learn to usethe program. Their promotions were basedon billable hours tied to client work, andthey could not justify billing clients forthe time it took them to learn this newsystem. Also, they did not understandpotential applications of Notes in theirwork.

Senior consultants, on the other hand,used Notes more often. Because they hadjob security they were more willing toinvest time to explore and experiment withNotes. Because of their experience, theywere able to see how the system wouldbenefit their work.

Another group of tax consultants inWashington, DC, adopted the Notes pro-gram. The study suggests that they hadsignificant incentive to show that theywere visible and valuable within the firm,and using Notes was an opportunity tobroadcast their visibility, to electronicallypublish their advice and make it availableto many of the consultants around the firm.Their motivation was to show that theWashington office was an important partof the firm. Orlikowskis conclusion is thatorganizational incentive systems need tobe taken into account when adoptinginformation systems.

Davenport (1997c) documents howanother large consulting firm (Ernst &Young) successfully adopted a Notesprogram. They created a miniorganization(the Center for Business Knowledge) thatorganized Ernst & Youngs consultantsinto specific areas. This organization wasstaffed with consultants from other offices,who were given 6-month assignments toplay a special role as knowledge net-workers. Within a short time, 22 networksof consultants with expertise in certainindustries and technology sectors were inplace. Each network was assigned a half-time person who codified the Notes data-bases, organized the insights from differ-ent projects, prompted line consultants toadd their own insights, and edited theprojects discussion and document data-bases. These knowledge networkers cameto understand consultants needs and

The study suggeststhat they had sig-nificant incentive toshow that they werevisible and valuablewithin the firm.


77

Some factorshave received littleor no attention byresearchers of orga-nizational learningand knowledgemanagement.

topics very well. Because the knowledgenetworkers were on short-term assign-ments, they were expected to use this newexpertise to advance their careers whenthey returned to their consulting positions.

Taken together, these studies suggestthat it is important to pay attention to thesocial context in which people use infor-mation technology, as well as the particu-lar incentive systems for knowledge trans-fer. The groups in Alpha Consulting andErnst & Young had different incentives toshare information, and their respectiveoutcomes were different. An aspect cen-tral to the second case is the presence of agroup that facilitates the learning process.It is notable that this group came from thevery consultant group for whom the pro-gram was designed. They were able tobring field expertise to bear, and were ableto build ongoing relationships in the fieldto prompt users to contribute. They be-came familiar with the dilemmas and chal-lenges in the field and were able to guidethe users to various resources in theprogram.

These studies reflect issues mentionedearlier, particularly issues of motivation,which inhibit use of LLSs. Other barrierssuch as low priority, resistance to change,lack of commitment, and turf protectionare also cited. However, another study ofknowledge transfer offers a somewhatdifferent perspective. Szulanski (1996)studied the phenomenon of knowledgestickiness, the extent to which problem-atic situations are experienced duringknowledge transfer. He found that the pri-mary obstacles to transfer of best practicesin organizations are (1) lack of absorptivecapacity, which refers to the ability of therecipient to identify, value, and apply newknowledge; (2) causal ambiguity, or the

uncertainty regarding cause-effect rela-tionships on the part of the knowledgerecipient; and (3) arduous relationshipbetween the source and recipient, whichis described by the degree to which therelationship serves as a conduit for knowl-edge and the degree of communication andintimacy in the relationship between thesource and the recipient. This suggests thebenefits of attention to aspects of organi-zational culture, such as developing thelearning capacities of organizational units,fostering closer relationships betweenunits, and understanding more systemati-cally the practices and learning readinessof organizational units.

Much studying remains to be done.Some factors have received little or no at-tention by researchers of organizationallearning and knowledge management. Forexample, much of the knowledge manage-ment literature relies on cases from privatesector firms, and so unique aspects of pub-lic organiza-tions may havebeen neglected.Private firmstypically haveclearer goals,such as marketshare and prof-itability, thanpublic organiza-tions. Indeed, the knowledge managementsystems for private firms are oftenclosed and unavailable to outsiders sothat the firms can protect their knowledgeand learning to achieve a competitive edge.Public organizations, particularly largeones, have less clear-cut goals and perhapseven conflicting and competing goalsamong their constituent elements.


78

Clearly, a level of competition existswithin the Department of Defense (DoD)among the services, programs, and depots,test centers, and smaller organizations(Kronenberg, 1990). It seems likely thenthat such competition could adversely af-fect the nature and extent of participationin an LLS that spanned several acquisi-tion organizations. Conversely, competi-tion might promote participation in aclosed LLS within an organization. Sucheffects could be manifested more widelyin DoDs current policy drive to becomemore business-like in its operations.

IMPLICATIONS

The literature reviewed in this articleindicates that organizational characteristicsand issues have at least as much, andperhaps more, importance than informa-tion technology issues in knowledge trans-fer. A key first step in LLS design, then, isto specify the organization for which the

system will be designed. This is especiallycritical in regard to defense acquisition or-ganizations, which vary in many respects.One could design an acquisition LLS forthe entire DoD; for any of the services;for a major command such as ArmyMaterial Command; for one of its subor-dinate commands such as Aviation andMissile Command; or for a local organi-zation, such as a lab, test range, or engi-neering center. One also could design anLLS within any of these organizations fora particular specialty or acquisition careerfield, such as contracting, cost estimation,or program management. Clearly, learn-ing will occur in different ways amongmembers of these various organizationsaccording to whether their relationshipsare strongly or weakly tied, the organiza-tions interpretive contexts, and manyother factors.

Table 2 shows how organizations mightvary according to some factors addressedin this article, which would affect designof acquisition LLSs. Design may be most

Table 2. Variations in Organizational Factors inAcquisition Lessons-Learned System Design

Organization Ties Interpretive OrganizationalContext Goals

DoDa Weak Low Ambiguous/conflicting

Career field across DoDa Weak High Ambiguous/conflicting(e.g., all systemsengineers)Service Weak Moderate Possibly mixed

Major command Weak Moderate Possibly mixedSubordinate comand Moderate Moderate Clear

Local organization Strong High Cleara DoD, Department of Defense


79

problematic for a DoD-wide system. Be-cause most DoD members have weak ties(i.e., distant, infrequent, or nonexistentrelationships) and low interpretive context(e.g., due to dissimilar backgrounds), asystem targeted towards sharing of explicitand codified versions of lessons learnedwould seem most appropriate. However,possibilities of competition among DoDorganizations may inhibit participation insuch a system. Members of an acquisitionspecialty or career field in DoD share ahigher interpretive context, which signalsgreater potential for lesson sharing. To theextent members of the same field share aprofessional loyalty to their field over loy-alty to their organizations, some of thedeleterious effects of competition may bemitigated. Members of the services and ofmajor commands have weak ties, thoughinterpretive context in these organizationsis certainly higher than that of DoD. Or-ganizational competition may yet exist atthese levels, particularly for scarce fiscalresources. Sharing of tacit and complexlessons would be, as expected, mosteffective in subordinate commands andlocal organizations.

Such variation points out the need forcontingency approaches in LLS design.Achieving effective knowledge transferbetween members of the acquisition com-munity will require substantial investmentof time and resources in determining whatkind of knowledge is appropriate for dif-ferent groups based on their relationalhistories and contexts.

An important question regarding LLSsis the possibility of creating a communityof practice when members connectionsare virtual rather than face-to-face. Virtualcommunities lack the synchronic feedbackand reciprocal exchange characteristic of

a discourse that creates and reflects ashared history. Transfer of tacit, complexknowledge requires a great deal of infor-mal, face-to-face contact. Thus, if an or-ganization relies on weak links such asintranets to exchange complex, tacitknowledge, it will likely meet with littlesuccess.

True communities of practice are prob-ably not possible in larger organizationswhere members are weakly tied and inter-pretive context is low. In such organiza-tions, members are more likely to searchfor novel information in explicit and codi-fied lessons.Thus, leaders oflarger organiza-tions should fo-cus their effortson eliminatingbarriers to ande n c o u r a g i n gmembers par-ticipation insystems that tar-get such lessons. Davenports findings(1997c) may be especially significant inthis regard. Leaders of larger acquisitionorganizations may consider assigning tem-porary duty to knowledge networkers,who would actively seek lessons, map outknowledge needs, encourage inquiry, andmake connections among members.

When members of an organizationshare strong ties, close proximity, andfrequent interaction, complex and tacitknowledge will be easier to transfer. How-ever, explicit knowledge may be redun-dant. If attempts are made to transfer ex-plicit knowledge between strongly tiedmembers, they might lose interest in theLLS. Thus, leaders of smaller organiza-tions should emphasize strong ties among

True communitiesof practice areprobably not pos-sible in larger orga-nizations wheremembers are weaklytied and interpretivecontext is low.


80

members, creation of a high interpretivecontext, and establishment of clear goalsto maintain an environment that facilitates

sharing of complex and tacit lessonsamong members of a true community ofpractice.

Keith Snider teaches in the Graduate School of Business and Public Policy atthe Naval Postgraduate School in Monterey, CA. He received his B.S. degreefrom the U.S. Military Academy, his M.S. from the Naval Postgraduate School,and his Ph.D. in public administration and public affairs from Virginia Tech. Hisrecent journal publications appear in American Review of Public Administration,Administration & Society, Acquisition Review Quarterly, Administrative Theory &Praxis, and Journal of Public Procurement.

(E-mail address: [email protected])

Frank J. Barrett is associate professor of systems management at the NavalPostgraduate School in Monterey, CA. He received his B.A. and M.A. degreesfrom the University of Notre Dame and his Ph.D. in organizational behavior fromCase Western Reserve University. He has published articles in the Journal ofApplied Behavioral Science; Human Relations; Organization Science; andOrganization Dynamics, as well as writing numerous book chapters. He is coeditorof Appreciative Inquiry and Organizational Transformation (Greenwood, 2001).


Ramkrishnan (Ram) V. Tenkasi is Associate Professor with the Ph.D. Programin Organization Change and Development, Benedictine University. His research,which has been funded by multiple divisions of the National Science Foundationand the Department of Defense, and appears in over 40 articles and bookchapters, examines Organizational Knowledge, Learning, Innovation and Change.He has a special interest in the role of Information Technology in facilitatingorganizational knowledge creation and learning processes.


ACKNOWLEDGMENT

The authors gratefully acknowledge the support of theOffice of the Assistant Scretary of the Army (Acquisition, Logistics and Technology).


81

REFERENCES

Brown, J. S. & Duguid, P. (1991). Orga-nizational learning and communitiesof practice. Organization Science,2(1), 4057.

Busby, J. S. (1999). An assessment of post-project reviews. Project ManagementJournal, 30(3), 2329.

Cook, S. N. & Yanow, D. (1993). Cultureand organizational learning. Journalof Management Inquiry, 2(4), 373390.

Daft, R., & Lengel, R. (1984). Informa-tion richness: A new approach tomanagerial information processingand organizational design. In B. Shaw& L. Cummings (Eds.), Research inorganizational behavior (pp. 191234). Greenwich, CT: JAI Press.

Davenport, T. (1997a). Secrets of success-ful knowledge management. Knowl-edge, Inc., February.

Davenport, T. (1997b, June 15). Knownevils: Common pitfalls of knowledgemanagement. CIO Magazine.

Davenport, T. (1997c). Knowledge man-agement case study: knowledge man-agement at Ernst & Young. Available:http://www.bus.utexas.edu/kman/E&Y.htm, December 15, 2000.

Aha, D.W. (2000, April). Briefing: Intel-ligent lessons learned systems. Paperpresented at the Department of EnergySociety for Effective Lessons LearnedSharing Spring 2000 Meeting. Avail-able: http://tis.eh.doe.gov/ll/proceed-ings/aha400.pdf, October 15, 2000.

Argote, L. & McGrath, J. (1993). Groupprocesses in organizations: Continu-ity and change. In C. Cooper & I.Roberts (Eds.). International reviewof industrial and organizational psy-chology. New York: Wiley, 333389.

Argyris, C. (1999). On organizationallearning (2nd ed.). Oxford, UK:Blackwell.

Argyris, C., & Schn, D. (1978). Organi-zational learning: A theory of actionperspective. Reading, MA: Addison-Wesley.

Axley, S. (1984). Managerial and organi-zational communication in terms ofthe conduit metaphor. Academy ofManagement Review, 9, 428437.

Baird, L., Holland, P., & Deacon, S.(1999). Learning from action. Orga-nizational Dynamics, 27(4), 1931.

Berger, P. & Luckman, T. (1967). The so-cial construction of reality. GardenCity, NY: Doubleday, Anchor.


82

Davenport, T., & Prusak, L. (1998). Work-ing knowledge: how organizationsmanage what they know. Boston:Harvard Business School.

Eisenberg, E. & Phillips, S. (1991). Mis-communication in organizations. In N.Coupland, H. Giles, & J. Wiemann(Eds.). Miscommunication andproblematic talk. Newbury Park, CA:Sage, 244258.

Fox, J. R. (1988). The defense manage-ment challenge: Weapons acquisition.Boston: Harvard Business School.

Fulk, J. (1993). Social construction ofcommunication technology. Academyof Management Journal, 36, 921950.

Fulmer, W. E. (1999). Buckman labs (CaseN9-899-175.). Boston: Harvard Busi-ness School.

Giddens, A. (1979). Central problems insocial theory: Action, structure, andcontradiction in social analysis. Ber-keley: University of California Press.

Granovetter, M. S. (1982). The strengthof weak ties: A network theory revis-ited. In P. V. Marsden & N. Lin (Eds.),Social structure and network analy-sis (pp. 105130). Beverly Hills, CA:Sage.

Hansen, M. T. (1999). The search-trans-fer problem: The role of weak ties insharing knowledge across organiza-tion subunits. Administrative ScienceQuarterly, 44(1), 82111.

Kronenberg, P. S. (1990). Public admin-istration and the Defense Department:Examination of a prototype. In G.Wamsley, R. Bacher, C. Goodsell, P.Kronenberg, J. Rohr, C. Stivers, O.White, & J. Wolf, Refounding publicadministration. Newbury Park, CA:Sage.

Lipshitz, R., Popper, M., & Oz, S. (1996).Building learning organizations: Thedesign and implementation of organi-zational learning mechanisms. Jour-nal of Applied Behavioral Sciences,32, 292305.

Lucas, L. & Ogilvie, D. T. (1999, June).Inter-unit knowledge transfer in mul-tinational corporations. Paper pre-sented at the Third International Con-ference on Organizational Learning,Lancaster, UK.

Mayer, K. (1991). The political economyof defense contracting. New Haven,CT: Yale.

McNaugher, T. L. (1989). New weapons,old politics: Americas procurementmuddle. Washington, DC: Brookings.

Myers, P. (1996). Introduction. In P. Myers(Ed.), Knowledge management andorganizational design. Boston:Butterworth-Heinemann.

Nonaka, I. (1991). The knowledge creat-ing company. Harvard Business Re-view, 69, 96104.


83

Orlikowski, W. J. (1992). The duality oftechnology: Rethinking the concept oftechnology in organizations. Organi-zation Science, 3, 398426.

Orlikowski, W. J. (1993). Learning fromnotes: Organizational issues in group-ware implementation. InformationSociety, 9, 237250.

Orlikowski, W. J., & Yates, J. (1994).Genre repertoire: The structuring ofcommunicative practices in organiza-tions. Administrative Sciences Quar-terly, 39, 541574.

Orr, J. (1996). Talking about machines: Anethnography of a modern job. Ithaca,NY: IRL Press.

Polanyi, M. (1966). The tacit dimension.London: Routledge and Kegan Paul.

Poole, M., & DeSanctis, G. (1990). Un-derstanding the use of group decisionsupport systems: The theory of adap-tive structuration. In J. Fulk & C.Steinfeld (Eds.) Organizations andcommunication technology (pp. 173193). Newbury Park, CA: Sage.

Putnam, L., Phillips, N., & Chapman, P.(1996). Metaphors of communicationand organization. In S. Clegg, C.Hardy, & W. Nord (Eds.), Handbookof organization studies (pp. 375408).London: Sage.

Schein, E. (1985). Organizational cultureand leadership. San Francisco:Jossey-Bass.

Senge, P. (1990). The fifth discipline: Theart and practice of the learningorganization. New York: DoubledayCurrency.

Szulanksi, G. (1996). Exploring internalstickiness: Impediments to the trans-fer of best practice within the firm.Strategic Management Journal, 17,2743.

Tyre, M., & Von Hippel, E. (1997). Thesituated nature of adaptive learning inorganizations. Organization Science,8, 7182.

U.S. Army. (1997). Center for Army les-sons learned handbook, 97-13. FortLeavenworth, KS: Author.

U.S. Geological Survey. (1995). San Fran-cisco Bay program: Lessons learnedfor managing coastal water resources(Fact sheet FS-053-95). Available:http://water.usgs.gov/wid/html/sfb.html, October 15, 2000.

U.S. Marine Corps. (1994, April). Order5000.17A, Marine Corps LessonsLearned System.

U.S. Navy. (1999, February 11). Changethrough ex-change process and re-quest for new ideas. Navy acquisitionreform executive memorandum. Avail-able: http://www.acq-ref.navy.mil/,October 15, 2000.

Watzlawick, P., Beavin, J., & Jackson, D.(1967). Pragmatics of human commu-nication: a study of interactional pat-terns, pathologies, and paradoxes.New York: W.W. Norton.


84

Weick, K. (1979). The social psychologyof organizing (2nd ed.). New York:McGraw-Hill.

Zack, M. H. (1999a). Developing knowl-edge. California Management Review,41(3), 127145.

Zack, M. H. (1999b). Managing codifiedknowledge. Sloan Management Re-view, 40(4), 4558.

CONSIDERATIONS IN ACQUISITION LESSONS-LEARNED SYSTEM DESIGN.pdf

Documents

Transcript of CONSIDERATIONS IN ACQUISITION LESSONS-LEARNED SYSTEM DESIGN.pdf