The Experience Factory May 2004 Leonardo Vaccaro.

The The ExperienceExperience Factory Factory

May 2004 May 2004 Leonardo VaccaroLeonardo Vaccaro

Leonardo VaccaroLeonardo Vaccaro The Experience FactoryThe Experience Factory

Introduction Introduction

Reuse of products, processes and experience Reuse of products, processes and experience

originating from the system life cycle is seen today originating from the system life cycle is seen today

as a feasible solution to the problem of developing as a feasible solution to the problem of developing

higher quality system at lower cost.higher quality system at lower cost.


OutlineOutline

Importance of Experience Factory in the Software BusinessImportance of Experience Factory in the Software Business

The Experience FactoryThe Experience Factory

Software Engineering LaboratorySoftware Engineering Laboratory

Q-Labs Experience Management SystemQ-Labs Experience Management System

Experience Factory ImplicationsExperience Factory Implications


Experience FactoryExperience Factory(Definition)(Definition)

It is a logical and/or physical organization that It is a logical and/or physical organization that supports project developments by analyzing and supports project developments by analyzing and synthesizing all kinds of experience, acting as a synthesizing all kinds of experience, acting as a repository for such experience, and supplying repository for such experience, and supplying that experience to various project on demandthat experience to various project on demand


Importance of Experience Factory Importance of Experience Factory in the software businessin the software business

The software business requires understanding, continuous The software business requires understanding, continuous improvement, and the packaging of experience for reuse.improvement, and the packaging of experience for reuse.

Reusing experience in the form of processes, products and other Reusing experience in the form of processes, products and other forms of knowledge is essential for improvementforms of knowledge is essential for improvement

Experience needs to be packagedExperience needs to be packaged

Experience must be evaluatedExperience must be evaluated

Software development must support reuse of experienceSoftware development must support reuse of experience

Packaged experiences need to be integratedPackaged experiences need to be integrated


Experiences Factory SchemaExperiences Factory Schema(Victor Basili)(Victor Basili)


Project OrganizationProject Organization

Produce and maintain softwareProduce and maintain software

Provides the analysis organization withProvides the analysis organization with– project and environment characteristicsproject and environment characteristics– development data development data – resource usage information resource usage information – process informationprocess information


Quality Improvement ParadigmQuality Improvement Paradigm

The basic methodologicalThe basic methodological device for thedevice for the Experience FactoryExperience Factory

Developed by Victor BasiliDeveloped by Victor Basili

Is the result of the application of the scientific method to the problem Is the result of the application of the scientific method to the problem of software quality improvementof software quality improvement

Based upon the notion that improving the software process requires Based upon the notion that improving the software process requires the continual accumulation of evaluated experiences (learning) in a the continual accumulation of evaluated experiences (learning) in a form that can be effectively understood and modified (experience form that can be effectively understood and modified (experience models) into a repository of integrated experience models models) into a repository of integrated experience models (experience base) that can be accessed and modified to meet the (experience base) that can be accessed and modified to meet the needs of the current project (reuse) needs of the current project (reuse)


Quality Improvement Paradigm Quality Improvement Paradigm (cont.)(cont.)

It is articulated in six steps:It is articulated in six steps:


1. Characterize1. Characterize

Understand the environment based uponUnderstand the environment based upon

– Available models Available models – Data Data – IntuitionIntuition

Establish baselines with the existing Establish baselines with the existing business processes in the organization business processes in the organization and characterize their criticalityand characterize their criticality


2. Set Goals2. Set Goals

Set quantifiable goals for successful Set quantifiable goals for successful project and organization performance and project and organization performance and improvement.improvement.

– Use of Goal \ Question \ Metric ParadigmUse of Goal \ Question \ Metric Paradigm

The reasonable expectation are defined The reasonable expectation are defined upon the baseline provided by the upon the baseline provided by the characterization stepcharacterization step


3. Choose Process3. Choose Process

Choose the appropriate processes for Choose the appropriate processes for – Improvement Improvement – Support methods and tools Support methods and tools

The processes must be consistent with the The processes must be consistent with the goals that have been setgoals that have been set


4. Execute4. Execute

Perform the processesPerform the processes

– Constructing the products Constructing the products – Providing project feedback based upon the data on Providing project feedback based upon the data on

goal achievement that are being collected goal achievement that are being collected


5. Analyze5. Analyze

At the end of each specific project, analyze At the end of each specific project, analyze the data and the information gathered to the data and the information gathered to

– Evaluate the current practices Evaluate the current practices – Determine the problems Determine the problems – Make recommendation for future project Make recommendation for future project

improvements. improvements.


Analysis OrganizationAnalysis Organization

Processes the information received from Processes the information received from the development organization the development organization

Returns direct feedback to each project, Returns direct feedback to each project, together with goals and models tailored together with goals and models tailored from similar project.from similar project.


6. Package6. Package

Consolidate the experience gained in the Consolidate the experience gained in the form of new, or updated models form of new, or updated models

Store it in a experience base so it is Store it in a experience base so it is available for future projects. available for future projects.


Support OrganizationSupport Organization

Sustains and facilitates the interaction Sustains and facilitates the interaction between developers and analysts between developers and analysts

Saving and maintaining the information Saving and maintaining the information

Making it efficiently retrievable Making it efficiently retrievable

Controlling and monitoring the access to it.Controlling and monitoring the access to it.


Experience BaseExperience Base

Contains an accessible and integrated set of Contains an accessible and integrated set of analyzed, synthesized and packaged experience analyzed, synthesized and packaged experience models that capture past experiencemodels that capture past experience


Packaged ExperiencePackaged Experience

The experience factory can package all The experience factory can package all kinds of experience kinds of experience

Resource modelsResource modelsChange and defect models Change and defect models Process definition and modelsProcess definition and modelsProduct modelsProduct modelsA variety of quality modelsA variety of quality models


Experience PackageExperience Package

Is the main product of the experience Is the main product of the experience factoryfactory

The content of this product vary upon the The content of this product vary upon the kind of experience clustered in the kind of experience clustered in the package package

There is a central element that determines There is a central element that determines what the package is.what the package is.


Examples of Experience PackageExamples of Experience Package

Product PackagesProduct Packages

Processes PackageProcesses Package

Relationship PackagesRelationship Packages

Tool PackagesTool Packages

Management PackagesManagement Packages

Data PackagesData Packages


Product PackagesProduct Packages

Have as their central element a product, Have as their central element a product, clustered with the information needed to reuse it clustered with the information needed to reuse it and the lessons learned in reusing itand the lessons learned in reusing it

Examples: Examples:

Programs Programs

ArchitecturesArchitectures

Designs Designs


Processes PackagesProcesses Packages

Have as their central element a process, Have as their central element a process, clustered with the information needed to execute clustered with the information needed to execute it and lessons learned in executing it.it and lessons learned in executing it.

Examples:Examples:

Processes modelProcesses model

Methods Methods


Relationship PackagesRelationship Packages

Have as their central element a Have as their central element a relationship or a system of relationship relationship or a system of relationship among observable characteristics of a among observable characteristics of a software project. These packages are software project. These packages are used for analysis and/or forecast of used for analysis and/or forecast of relevant phenomenarelevant phenomena

Examples:Examples:Cost and defect models, Cost and defect models,

Resource modelsResource models


Tool PackagesTool Packages

Have as their central element a specific tool Have as their central element a specific tool either constructive or analytic.either constructive or analytic.

Examples:Examples:

Code generator (constructive)Code generator (constructive)

Static analyzer (analytic)Static analyzer (analytic)


Management PackagesManagement Packages

Have as their central element any Have as their central element any container of reference information for container of reference information for project management.project management.

Examples:Examples: Management handbooks,Management handbooks,

Decision support modelsDecision support models


Data PackagesData Packages

Have as their central element a collection Have as their central element a collection of defined and validated data relevant for a of defined and validated data relevant for a software project or for activities within it.software project or for activities within it.

Examples:Examples: Project databasesProject databases

Quality recordsQuality records


Goal \ Question \ Metric ParadigmGoal \ Question \ Metric Paradigm

The mechanism used by the Quality The mechanism used by the Quality Improvement Paradigm for defining and Improvement Paradigm for defining and evaluating a set of operational goals using evaluating a set of operational goals using measurement. measurement.

Represents a systematic approach for Represents a systematic approach for tailoring and integrating goals with models tailoring and integrating goals with models of the software processes.of the software processes.


Goal \ Question \ Metric ParadigmGoal \ Question \ Metric Paradigm

1.1. Development of goalsDevelopment of goals

2.2. Generation of questions Generation of questions that define the goals that define the goals

3.3. Indication of metrics that Indication of metrics that answer the questionanswer the question

Objective: Objective:

Minimize the number of Minimize the number of defectsdefects


SofSoftware Engineering tware Engineering LaboratoryLaboratory

A software organization that has for a long A software organization that has for a long time recognized the value of accumulation time recognized the value of accumulation and reuse of experience is the NASA and reuse of experience is the NASA Goddard Space Flight Centre which has Goddard Space Flight Centre which has developed since 1977 the Software developed since 1977 the Software Engineering Laboratory Engineering Laboratory


Software Engineering LaboratorySoftware Engineering Laboratory

Is a very advanced examples of the concept of Is a very advanced examples of the concept of Experience Factory.Experience Factory.

The experience packages developed by the The experience packages developed by the Software Engineering Laboratory have mainly Software Engineering Laboratory have mainly focused on project management and control, focused on project management and control, acquisition and tailoring of new technologies for acquisition and tailoring of new technologies for software development and maintenancesoftware development and maintenance


The Software Engineering The Software Engineering Laboratory OrganizationLaboratory Organization


Software Management Software Management EnvironmentEnvironment

The most interesting experience packages The most interesting experience packages developed by Software Engineering Laboratorydeveloped by Software Engineering Laboratory

Is a set of data, tools manuals and analysis Is a set of data, tools manuals and analysis techniques supplied to the project management techniques supplied to the project management in order toin order to

– control the execution of project,control the execution of project,– compare it with similar ones compare it with similar ones – detect and analyze problems, detect and analyze problems, – identify solutionsidentify solutions


Q-LabsQ-Labs

An internationally distributed software An internationally distributed software engineering consulting company with office engineering consulting company with office in Europe as well as in the United States in Europe as well as in the United States

Needed an infrastructure to share Needed an infrastructure to share experience between employees no matter experience between employees no matter where they are located where they are located

Q-Labs Experience Management SystemQ-Labs Experience Management System


Experience Management SystemExperience Management System

A physical implementation of the Experience A physical implementation of the Experience FactoryFactory

Composed ofComposed of– ContentContent

Data, Information, ExperienceData, Information, Experience

– StructureStructureThe way the content is organizedThe way the content is organized

– ProceduresProceduresInstructions on how to use, package, delete and update the Instructions on how to use, package, delete and update the experienceexperience

– ToolsToolsVisual Query InterfaceVisual Query Interface


Q-Labs Experience Management Q-Labs Experience Management SystemSystem

Stores documents and their description in the Experience Stores documents and their description in the Experience BaseBase

The search is performed using a tool called Visual Query The search is performed using a tool called Visual Query InterfaceInterface

Experience packages in the context of Q-Labs are single Experience packages in the context of Q-Labs are single computer documentcomputer document

– TextsTexts– Slides Slides – GraphsGraphs– Scanned imagesScanned images– Numeric dataNumeric data


Experience Factory ImplicationsExperience Factory Implications

Offers an organizational structure that Offers an organizational structure that – Separates the product development focus from the learning and Separates the product development focus from the learning and

reuse focus. reuse focus. – Supports learning and reuse Supports learning and reuse – Generates a tangible corporate asset in the form of packaged Generates a tangible corporate asset in the form of packaged

experiences.experiences.

There are costs involved in instituting such a program. There are costs involved in instituting such a program. – Based upon the SEL experience where a full measurement Based upon the SEL experience where a full measurement

program has been in progress for over 14 years, project data program has been in progress for over 14 years, project data collection overhead is estimated to be about 5% of the total collection overhead is estimated to be about 5% of the total project cost.project cost.


ReferencesReferences““The The ExperienceExperience Factory” – Victor Basili, Gianluigi Factory” – Victor Basili, Gianluigi Caldiera, Dieter Rombach Caldiera, Dieter Rombach

http://wwwagse.informatik.uni-kl.de/pubs/repository/basili94c/encyclo.ef.pdf http://wwwagse.informatik.uni-kl.de/pubs/repository/basili94c/encyclo.ef.pdf

““Implementing the Experience Factory concepts” – Victor Implementing the Experience Factory concepts” – Victor Basili, Mikael Lindvall, Patricia Costa Basili, Mikael Lindvall, Patricia Costa http://www.cebase.org/www/researchActivities/eBase/websepp.pdfhttp://www.cebase.org/www/researchActivities/eBase/websepp.pdf

Software Engineering Laboratory Software Engineering Laboratory http://sel.gsfc.nasa.gov/website/welcome.htmhttp://sel.gsfc.nasa.gov/website/welcome.htm

““Lessons Learned about Structuring and Describing Lessons Learned about Structuring and Describing Experience for Three Experience Base” – Mikael Lindvall, Experience for Three Experience Base” – Mikael Lindvall, Michael Frey, Patricia Costa, Roseanne TesorieroMichael Frey, Patricia Costa, Roseanne Tesoriero


Visual Query InterfaceVisual Query InterfaceVisualize the content of Visualize the content of the Experience Basethe Experience BaseThe attributes are used to The attributes are used to describe the experience describe the experience packages packages – Package IdPackage Id– NameName– SubmittedSubmitted

X axis represents the X axis represents the “Open Date”“Open Date”Y axis represents “The Y axis represents “The Person Responsible”Person Responsible”The color represents the The color represents the “Costumer” “Costumer”

The Experience Factory May 2004 Leonardo Vaccaro.

Documents

Transcript of The Experience Factory May 2004 Leonardo Vaccaro.