Post on 26-May-2020
1995 SOFTWARE BEST PRACTICEQUESTIONNAIRE -
ANALYSIS OF RESULTS
by
S. DUTTA*S. KULANDAISWAMY**
andL. N. VAN WASSENHOVEt
96/53/TM
Associate Professor of Information Systems at INSEAD, Boulevard de Constance, 77305 FontainebleauCedex, France.
** MBA Research Graduate at INSEAD, Boulevard de Constance, 77305 Fontainebleau Cedex, France.
t Professor of Operations Management and Operations Research at INSEAD, Boulevard de Constance,77305 Fontainebleau Cedex, France.
A working paper in the INSEAD Working Paper Series is intended as a means whereby a faculty researcher'sthoughts and findings may be communicated to interested readers. The paper should be consideredpreliminary in nature and may require revision.
Printed at INSEAD, Fontainebleau, France.
1995 SOFTWARE BEST PRACTICEQUESTIONNAIRE
Analysis of Results
Soumitra DuttaSelvan KulandaiswamyLuk Van Wassenhove*
Research Initiative in Software ExcellenceTechnology Management Area
INSEAD77305 Fontainebleau, France
*Names of authors are mentioned in alphabetical order
Software Best Practice Questionnaire - Analysis of Results
TABLE OF CONTENTS
INTRODUCTION 4
LEGEND 5
DEFINITIONS AND METHODOLOGY 6
SECTION 1- ORGANISATIONAL ISSUES 7
SECTION 2 - STANDARDS AND PROCEDURES 9
SECTION 3 - METRICS 11
SECTION 4 - CONTROL OF THE DEVELOPMENT PROCESS 13
SECTION 5 - TOOLS AND TECHNOLOGY 15
SECTOR ANALYSIS 17
SIZE ANALYSIS 18
Software Best Practice Questionnaire - Analysis of Results
EXECUTIVE SUMMARY
This report presents an analysis of the results of the EC Software Best Practice Questionnaire enclosed withthe project proposals submitted during the ESSI Call ending in March 1995. There was a strong response tothe questionnaire and 463 valid responses were obtained from 17 different countries and 33 different sectors.
The aim of the EC Best Practice Questionnaire was to collect data on software engineering and managementbest practices within European organisations. A "best practice" is defined as a management practice that iswidely recognised as excellent and is recommended by most practitioners and experts in the field. The EC BestPractices questionnaire was designed such that each question addressed a particular best practice. Thequestionnaire had 42 questions divided into five sections: organisational issues, standards and procedures,metrics, control of the development process and tools and technology. Filling out the questionnaire that wasenclosed as part of the Information Package was mandatory for project proposals to be accepted for theevaluation process.
Based on the collected data, an analysis of best practices has been performed along various dimensions. Inparticular, the frequency of adoption of different best practices across countries has been studied. Thevariations in best practice adoption across sectors and company size (in terms of number of software relatedemployees) have also been analysed.
The analysis of the data has yielded many interesting results. At an aggregate level, about half of allresponding companies have adopted most of the best practices defined in the questionnaire. For a largepercentage of the best practices surveyed, the adoption rate was between 40% and 60%. Though theuniformity of adoption within individual organisations cannot be observed from the questionnaire, we caninfer an average level of awareness of leading edge software management practices within Europeanorganisations. It should be noted that the structure of the questionnaire (only yes-no answers) and the fact thatit accompanied Call for Proposals could have led respondents to portray a scenario more optimistic thanreality. There are distinct areas where European companies exhibit either very high or very low adoptionrates. For example, a large percentage of the responding companies manage their project staffing, planningand tracking proficiently. In contrast, only very few companies use formal estimation and quality assurancemethods.
The levels of best practice adoption across countries shows significant variation. For instance, in the contextof the usage of tools and technology, the levels range from a high of 55% for the UK to a low of 25% forSweden. The UK implements more best practices in the categories of organisational issues, standards andchange control than any other European country while France follows the most number of best practices in thecategories of metrics and use of tools. Some countries such as Greece have very low levels of softwaremanagement best practices adoption overall. While these variations across countries are to be expected, itpoints to the need for a greater emphasis on the sharing of knowledge and the transfer of best practices acrosscountries.
Best practices also vary greatly across sectors. The Aircraft and Spacecraft sector comes out ahead of allother sectors. The Telecom Products and Electronic Engineering and related Technical Consultancy sectorscome out in second and third positions respectively (ahead of Software Consultancy and Supply, DataProcessing and Related Services).
A comparative study of software management best practice adoption by companies with differing numbers ofemployees involved in software yielded interesting results. Large organisations adopt more best practices thansmaller ones. We observed this phenomenon of higher adoption rates within larger companies uniformlyacross all best practices included in the questionnaire. This is probably indicative of the fact that largercompanies are more likely to have implemented the best practice somewhere within their organisations (asmentioned earlier, the uniformity of the implementation cannot be deduced from the questionnaire).
3.
0.
p4
g 60
4s
40.0E 30
20
10
0
90
80
70
8278
3631 29
13 1579
09
16
8
MI
I I I)., . =..0 re 73' •-4S 09
0, 00 a74 ' C r / )i 4 i
"A'
1713
89
Software Best Practice Questionnaire - Analysis of Results
Introduction
This report presents an analysis of the data received in response to the Software Best Practice Questionnaireenclosed with the project proposals submitted during the ESSI Call ending in March 1995. The questionnaire(see Appendix) is divided into two parts:
(A) General Information(B) Section 1 - Organisational Issues (8 questions)
Section 2 - Standards and Procedures (13 questions)Section 3 - Metrics (8 questions)Section 4 - Control of the Development Process (6 questions)Section 5 - Tools and Technology (7 questions)
The questions in part (B) require a response of the form "Yes" or "No". For example, one of the questions inSection 1 is "Does each software project have a nominated software project manager?". Some questions alsohave a "Not Applicable" response category.
Responses were received from companies in 33 sectors and 17 European countries. Of the responses received,463 were found to be valid.
The maximum number of responses were received from Italy, Spain and Germany. The chart below gives adistribution of responses by country.
Number of valid responses received from each country
4.
Software Best Practice Questionnaire - Analysis of Results
The maximum number of responses were received from the Software Consultancy and Supply, Telecom Products andIndustrial Process Control Systems sectors. No responses were received from companies in 13 sectors. Seven sectorshad only one responding company each.The chart below outlines the distribution of responses by sector.
Number of valid responses received from each sector
LEGENDFl - Finance and Insurance EP Industrial Process Control IT - Telecom Products
SystemsEE - Electrical Engineering & ME - Mechanical Engineering AS - Aircraft & Spacecraft
Technical Consultancy & Technical ConsultancyIS - Software Consultancy and IE Electronic Engineering & EQ - Machinery, Electrical &
Supply Technical Consultancy Optical instruments
5.
Software Best Practice Questionnaire - Analysis of Results
Definitions and Methodology
We define a "best practice" as a management practice that is widely recognised as good and that isrecommended by most practitioners and experts in the field. The structure of the questionnaire is such thateach question addresses a specific best practice.
We define the "best practice level" for a company as the percentage of best practices that the companyfollows. The analysis is conducted along several lines. Each section of the questionnaire is analysedindividually and the following analyses are presented: an analysis that outlines the overall extent of adoptionof the practices in the section, a country-wise analysis focusing on which countries have the highest practicelevels and a presentation of the average practice levels for each country. The responses are then analysed onsector lines and on the basis of the size of their software organisation. Hungary, with a single response, hasbeen omitted from country-wise analyses. Sectors in which only one response or no responses were receivedhave been omitted from the sectorwise analysis.
6.
Software Best Practice Questionnaire - Analysis of Results
Section 1- Organisational Issues
The first section of the questionnaire deals with software management issues pertaining to the organisation.The results reveal that about half of the companies have well established management practices relating toorganisational issues.
,Most companies About half of the companies• Appoint a project manager for each project • Have a separate quality assurance function
• Have a change control function for eachproject
• Train all newly appointed managers• incorporate customer input throughout the
project• Ensure availability of non-software resources• Maintain awareness of the latest software
engineering technology
While 91% of the companies surveyed nominated project managers for each project, only 41% had a separatequality assurance function. Only 42% of the companies train their project managers to familiarise them within-house practices. Customer input is ensured by 60% of the companies. This is surprising when we see it inthe context of one of the results of the ESI Survey on Software Engineering Practices in Europe [ESI] whichindicates that improving customer satisfaction is the most critical aspect for the organisation according tomore than 85% of the software managers.
Distribution by country and sector
The table below outlines the countries and sectors with high and low adoption levels of various practices.
Management practice Countries with Countries with Sectors with Sectors withhigh adoption low adoption ' high adoption low adoption
Appointing project managersfor each project
All countries ' All sectors
Software quality assurancefunction with independentreporting line
France, UK,Finland
Belgium, Spain,Greece
Aerospace,, Telecom
MechanicalEngineering,Electr. & Optical
Establishing a change controlfunction
UK, Austria,Ireland
France, Sweden,Greece
Aerospace,' Telecom
Finance, Electrical& Optical, ProcessControl
Training new projectmanagers
Ireland, Finland Norway, Austria,Denmark
Aerospace,Telecom, Finance
All other sectors
Maintaining awareness of newdevelopment technologies
Ireland, Finland,Netherlands
Sweden, Belgium,Portugal
Telecom All other sectors
Ensuring user input at allstages of the project
Switzerland, UK,Finland
Austria, Belgium,Spain
All sectors,especiallyMechanical Engg.
The UK, while having the best practices overall, has a surprisingly low 37% when it comes to training projectmanagers and maintaining awareness of the state-of-the-art tools and techniques. Belgium and Greece, with18% and 20% respectively, exhibit low adoption in establishing a separate quality assurance function. TheAerospace sector maintains high levels of best practices except in attempting to keep abreast of newtechnologies, where they show a low 27% adoption rate. Mechanical Engineering companies are especiallyefficient at ensuring user input at all stages with an 89% adoption.
7.
Organisational issues - Best practice levels for various countries
Software Best Practice Questionnaire - Anal ysis of Results
Organisational issues - Best Practice levels for various countries and sectors
Section 1 contains 8 questions, each of which addresses a best practice for organisational issues. We find thebest practice level' by calculating the percentage of the best practices a company follows. To calculate thebest practice level for a country, we find the average of the best practice levels for all the companies in thecountry. For example, if we wish to find the best practice for France, we first find the percentage of bestpractices each French company follows in Section 1 of the questionnaire. Then we average across all theFrench companies to find a best practice level for France. There is a wide variation in best practice levelsbetween countries. To find the best practice level for a sector, we follow a similar approach whereby weaverage across a particular sector instead of a country. The following bar graphs give a ranking of thecountries and sectors for best practices pertaining to organisational issues.
A - Austria E - Spain I - Italy P - PortugalB - Belgium F - France IRL - Ireland S - SwedenD - Germany FIN - Finland N - Norway SF - SwitzerlandDK - Denmark GR - Greece NL - Netherlands UK - United Kingdom
Organisational issues - Best practice levels for various sectors
Fl - Finance and Insurance IP - Industrial Process Control IT - Telecom ProductsSystems
ME - Mechanical Engineering AS - Aircraft & Spacecraft& Technical Consultancy
IE - Electronic Engineering & EQ - Machinery, Electrical &Technical Consultancy Optical instruments
EE - Electrical Engineering &Technical Consultancy
IS - Software Consultancy andSupply
I See Introduction for a definition of best practice level
8.
Software Best Practice Questionnaire - Analysis of Results
Section 2 - Standards and ProceduresThe second section of the questionnaire deals with software management issues pertaining to the existence ofstandardised processes for estimation, coding, quality assurance etc. The results reveal that most companieshave established project assessment and tracking methods. However, quality assurance procedures are
Most companies About half of the companies Few companies• Assess viability, risks and
benefits before committing• Ensure that subcontractors
follow formal processes• Have formal methods of
estimating software sizeto a project • Conduct inspection and
• Conduct periodic reviews walkthroughs at each stageof the status of projects • Use formal methods to
• Have common coding estimate schedule and coststandards for projects • Plan testing before coding
• Perform independent testing
A high 76% of companies conduct risk assessment and 86% conduct periodic reviews of project status. In thearena of estimation, only 26% of the companies have formal methods for software size estimation. Thenumber of companies using formal procedures for effort and schedule estimation is higher at 51%. About 48%of the companies perform independent testing by quality assurance and 51% of the companies plan testingbefore coding. This shows that, though 47% of European companies use no formal standards and only 5% ofthe software is developed according to formal quality specifications [ESI], at least half of the Europeancompanies do have some kind of quality process.
Distribution by country and sectorThe table below outlines the countries and sectors with hi gh and low adoption levels of various practices.Management practice Countries with Countries with Sectors with Sectors with
high adoption low adoption i high adoption low adoptionAssessment of risk and viabilityof projects
Finland, UK,Belgium
Greece, AustriaSwitzerland,
All sectors exceptElectr. & Optical
Periodic review of status UK, Netherlands,Spain
Sweden, Portugal All sectors exceptElectr. & Optical
Walkthroughs and inspectionsof design and code
UK, Switzerland,Finland
Portugal, France,Spain, Belgium
Aerospc, Telecom,Electronic Engg
All other sectors
Application of common codingstandards
UK, Switzerland,Austria
Belgium, Denmark All other sectors Electrical Engg
Formal procedure forestimation of size
All, especiallySweden, Portugal
All sectors
Formal procedure forestimation of effort, scheduleand cost
Finland, Austria,Switzerland
Sweden, Belgium,Germany
Finance, Telecom Electronic Engg,Electrical Engg.
Formal procedures for hand-offfrom one group to another
UK, France Belgium,Denmark, Finland
Aerospace Electr & OpticalProcess Control
Planning testing prior to coding UK, Switzerland,Ireland
Portugal, Belgium,Greece, Austria
Aerospace Electr. & Optical,Electrical Engg,Mechanical Engg.
Testing by an independentquality assurance group
UK, Ireland,France
Spain, Belgium,Italy
Electronic Engg,Telecom,Aerospace
Electrical &Optical, ElectricalEngg,
The UK exhibits a strong adoption of most practices relating to the usage of standards and procedures. 96%of UK companies assess the risk and viability of projects. 81% of UK companies apply common codingstandards. The usage of formal procedures for the estimation of size is very low in European companies,
9.
Standards and Procedures - Best practice levels for various countries
Software Best Practice Questionnaire - Analysis of Results
especially in Swedish and Portuguese ones (0% of respondents from each of these two countries). TheElectrical Engineering sector exhibits particular weaknesses in the implementing procedures and standards. Itexhibits only a 20% adoption in the practice of common coding standards and walkthroughs and a 10%adoption in using formal procedures to estimate program size. All Aerospace companies surveyed followedformal methods of hand-off between groups and commenced test planning prior to coding giving them a 100%adoption in these areas.
Standards and Procedures - Best Practice levels for various countries and sectors
Using the definition of the best practice level found in the introduction, we calculate the best practice level fora company as the percentage of best practices that a company follows. We include only the questions inSection 2 for this analysis. To find the best practice level for a country, we average the best practice levels forall companies in that country. To find the best practice level for a sector, we average the best practice levelsfor all companies in that sector. The following bar graphs give rankings of the countries and sectors for bestpractices pertaining to standards and procedures.
A - Austria E - Spain I - Italy P - PortugalB - Belgium F - France IRL - Ireland S - SwedenD - Germany FIN - Finland N - Norway SF - SwitzerlandDK - Denmark GR - Greece NL - Netherlands UK - United Kingdom
Standards and Procedures - Best practice levels for various sectors
FI - Finance and Insurance
EE - Electrical Engineering &Technical Consultancy
IS - Software Consultancy andSupply
IP - Industrial Process ControlSystems
ME - Mechanical Engineering& Technical Consultancy
IE - Electronic Engineering &Technical Consultancy
IT - Telecom Products
AS - Aircraft & Spacecraft
EQ - Machinery, Electrical &Optical instruments
10.
Software Best Practice Questionnaire - Analysis of Results
Section 3 - Metrics
This section of the questionnaire addresses best practices pertaining to software metrics. We observe that thecompanies surveyed collect data about errors found, but do not analyse them for their causes. In general,tracking and performance metrics are used more widely than quality and estimation metrics.
Most companies About half of the companies Few companies• Log post-implementation • Use version control methods • Use size of software
problems and track theirresolution
• Use actual versus estimatedresults of previous projects
measures in their estimationprocess
while estimating new ones • Perform a root cause• Use "earned-value" project analysis of errors detected
tracking • Gather statistics of errors• Compare actual against
expected performancefound in each stage of theproject
Weaknesses in the area of estimation using previous data are again observed. While 55% of the organisationsuse previous records for scheduling, only 21% of them use data from previous projects for size estimation.Only 30% of the companies perform a root cause analysis of the errors discovered.
Distribution by country and sector
The table below outlines the countries and sectors with high and low adoption levels of various practices.
Management practice Countries with Countries with Sectors with Sectors withhigh adoption low adoption high adoption low adoption
Record and feedback ofestimated versus actual effortsinto estimation process
Sweden,Netherlands,Belgium
Greece, Ireland,Italy
All sectors exceptElectronic Engg.
Record and feedback of sizeinto estimation process
All countries Aerospace All other sectors
Analysis of sources of errors Finland All other countries All sectorsGathering statistics of testingefficiency
All countries All sectors
Comparison of estimated andactual performance levels
France, Austria,Sweden
Ireland,Netherlands, Spain
Aerospace,Electronic Engg.
Electrical &Optical
Logging of problem reports All, especially UK All sectors exceptAustria, Portugal Finance
Sweden and Portugal have a null adoption of size estimation and testing efficiency measurement. However, itcomes out among the top three countries in performance estimation and comparison with actual levels andPortugal comes out among the top three countries in the logging and tracking of post-implementation softwareproblems. The ESI survey [ESI] indicates that the penetration of quality programs like ISO 9001 is more than50% in Germany, the UK and Italy but less than 10% in Spain and France. However, we observe that inquality procedures such as the analysis of errors and measuring testing efficiency, Germany exhibits very lowlevels (24% and 7% respectively). Besides, these levels are lower than those in France (34% and 14%,respectively). None of the Electrical and Optical and Electronic Engineering companies that we surveyedgathered statistics of testing efficiency giving them a 0% adoption rate. All Aerospace and ElectronicEngineering logged their problem reports, giving them a 100% adoption rate. The Electrical & Optical sectorshowed weaknesses in other areas too, with only 8% of companies recording and feeding back size into theestimation process and 25% comparing estimated and actual performance levels.
11.
Software Best Practice Questionnaire - Analysis of Results
Metrics - Best Practice levels for various countries and sectors
The calculation of best practice levels is much the same as in Sections 1 and 2. To find the best practice levelfor a country, we simply find the best practice levels for the companies in that country and average across allthe companies. To find the best practice levels for a sector, we find the best practice levels for companies inthe sector and then average across all companies. The following bar graphs give a ranking of the countries andsectors for best practices pertaining to Metrics.
Metrics - Best practice levels for various countries
A - Austria E - Spain I - Italy P - PortugalB - Belgium F - France IRL - Ireland S - SwedenD - Germany FIN - Finland N - Norway SF - SwitzerlandDK - Denmark GR - Greece NL - Netherlands UK - United Kingdom
Metrics - Best practice levels for various sectors
FI - Finance and Insurance
EE - Electrical Engineering &Technical Consultancy
IS - Software Consultancy andSupply
11:' - Industrial Process Control IT - Telecom ProductsSystems
ME - Mechanical Engineering AS - Aircraft & Spacecraft& Technical Consultancy
IE - Electronic Engineering & EQ - Machinery, Electrical &Technical Consultancy Optical instruments
12.
Software Best Practice Questionnaire - Anal ysis of Results
Section 4 - Control of the Development ProcessThis section of the questionnaire addresses best practices pertaining to procedures and mechanisms for changemanagement.
Most companies About half of the companies Few companies• Assign the task of
estimating schedules forchanges to managers who
• Gain sign-off from all partiesinvolved before revisingproject plans
• Perform regressiontesting following eachchange
directly control the project • Have procedures forcontrolling changes to code,requirements, design etc.
• Ensure that every function istested
About 55% of European companies gain sign-off from all parties involved before revising project plans andan equal number have established processes for controlling changes to code, requirements and designs. Only29% perform regression testing routinely. However, other quality assurance processes are followed: 51% ofthe companies ensure that every function is tested.
Distribution by country and sector
The table below outlines the countries and sectors with high and low adoption levels of various practices.
Management practice Countries with Countries with Sectors with Sectors withhigh adoption low adoption high adoption low adoption
Obtaining signoff from allparties before changing plans
Switzerland, UK Sweden, Portugal All sectors exceptElectr. & Opticaland Finance
Procedures for controllingchanges to requirements,designs and documentation
UK, Denmark,Austria, Portugal
Belgium, Sweden Aerospace,Electronic Engg,Mechanical Engg
Finance
Procedures for controllingchanges to code
Netherlands,Austria, UK
Sweden Aerospace Finance
Regression testing followingeach change
All countries Aerospace All other sectors
Ensuring testing of everyprocedure
Ireland, UK,Greece
Netherlands,Finland, Belgium
Aerospace Electrical &Optical, Finance
Sweden exhibits particular weaknesses in the area of change management with only 25% of the companiesgetting signoff on changes to plans, a low 38% of companies with processes for controlling changes to code,requirements and specifications. Except in the area of regression testing, the UK exhibits very clear strengths:85% of the companies have established methods for controlling changes to code, specification andrequirements. Unlike in the other sections, Greece ranks third in the control of the software process. Allaerospace companies surveyed have procedures for controlling changes to requirements, designs,documentation and to code. Their adoption rates in these areas are therefore 100%. The Electrical & Opticaland Finance sectors show particularly low adoption in the domain of testing. Only 12% of Finance sectorcompanies perform regression testing following every change, while no company (i.e. 0% adoption) in theElectrical & Optical sector performs regression testing. With respect to ensuring testing of every proceduresthe adoption rates are 17% for the Electrical & Optical sector and 29% for the Finance sector.
13.
Control of the development process - Best practice levels for various countries
Software Best Practice Questionnaire - Analysis of Results
Control of the development process - Best practice levels for various countries and sectors
Section 4 has 6 questions which address the control of the development process. To determine the bestpractice level for a company in this section, we first determine the best practice levels for each company inthis country. Then, we average across all companies in the country to obtain a best practice level for thecountry. Similarly, to find the best practice level for a particular sector, we average across all companies inthe particular sector. The following bar graph gives a ranking of the countries for best practices pertaining tothe control of the development process.
A - Austria E - Spain I - Italy P - PortugalB - Belgium F - France IRL - Ireland S - SwedenD - Germany FIN - Finland N - Norway SF - SwitzerlandDK - Denmark GR - Greece NL - Netherlands UK - United Kingdom
Control of the development process - Best practice levels for various sectors
Fl - Finance and Insurance
EE - Electrical Engineering &Technical Consultancy
IS - Software Consultancy andSupply
IP - Industrial Process ControlSystems
ME - Mechanical Engineering& Technical Consultancy
IE Electronic Engineering &Technical Consultancy
IT - Telecom Products
AS - Aircraft & Spacecraft
EQ - Machinery, Electrical &Optical instruments
14.
Software Best Practice Ouestionnaire - Analysis of Results
Section 5 - Tools and TechnologyThis section of the questionnaire addresses best practices pertaining to the utilisation of tools and technologyfor management and development activities.
Most companies About half of the companies Few companies• Use software tools for
project planning,estimating and scheduling
•
•
Use design notations likeSADT in product designUse prototyping methods for
• Use software tools intracing code to design torequirements
development • Use automated testing• Use a data dictionary for tools
systems design anddevelopment
• Use software tools fortracking status ofsubroutines in thedevelopment library
The usage of software tools is fairly high for planning and scheduling, but very low (26%) in forward andbackward tracing of requirements through to designs and code. As in other sections, the use of automatedtools in quality assurance related activities is low at 30%. The overall levels of usage of tools is fairly lowwhen we view this in light of the fact that the ESI survey [EST] revealed that tools were considered by 92% ofthe managers as one of the most important requirements for improving productivity.
Distribution by country and sector
The table below outlines the countries and sectors with high and low adoption levels of various practices.
Management practice Countries with Countries with Sectors with Sectors withhigh adoption low adoption high adoption low adoption
Usage of tools for tracingforwards and backwardsthrough requirements, designand code
All countries,especially Finland,Ireland andPortugal
All sectors
Usage of design notation likeSADT
Finland, France,Denmark, UK,Austria
All othercountries,especially Sweden
Aerospace Electrical &Optical, Finance
Usage of automated testingtools
All countries,especiallySwitzerland
Aerospace Electr. & Optical,Mechanical Engg,Softwareconsulting
Usage of prototyping methods Belgium, Ireland,Netherlands
Sweden, Italy,Spain
Electrical Engg Electrical &Optical
Usage of software tools for All, especially All, especiallyproject planning, estimation Belgium, Denmark Aerospace andand scheduling and Switzerland _Telecom
Swiss companies exhibit a very low 7% usage of automated testing tools. Finnish companies, while showinghigher usage of design notations like SADT, have a surprising 0% usage of tools in tracing requirements todesign and code. While Aerospace companies have a high 91% usage of design notation and automated testingtools, usage of these tools and techniques is only 8% in the Electrical & Optical sector. The highest usage(80% adoption) of prototyping is found in the Electrical Engineering sector.
15.
Software Best Practice Ouestionnaire - Analysis of Results
Tools and Technology - Best practice levels for various countries and sectors
As in the other sections, we calculate the best practice level for a company using the definition given in theintroduction. Specifically, for this section, we calculate the percentage of the 7 best practices that a companyfollows. To find the best practice level for a country, we simply average across all the companies that belongto the country. To find the best practice levels for a sector, we similarly average across all the companies thatbelong to the sector. The following bar graphs give a ranking of the countries and sectors for best practicespertaining to the usage of tools and technology.
Tools and Technology - Best practice levels for various countries
A - Austria E - Spain I - Italy P - PortugalB - Belgium F - France IRL - Ireland S - SwedenD - Germany FIN - Finland N - Norway SF - SwitzerlandDK - Denmark GR - Greece NL - Netherlands UK - United Kingdom
Tools and Technology - Best practice levels for various sectors
Fl - Finance and Insurance
EE - Electrical Engineering &Technical Consultancy
IS - Software Consultancy andSupply
IP - Industrial Process Control IT - Telecom ProductsSystems
ME - Mechanical Engineering AS - Aircraft & Spacecraft& Technical Consultancy
1E - Electronic Engineering & EQ - Machinery, Electrical &Technical Consultancy Optical instruments
16.
Software Best Practice Ouestionnaire - Analysis of Results
SECTOR ANALYSIS
Until this point, we have performed a sector analysis for each section of the questionnaire. Here, we calculatethe overall or aggregate best practice levels for various sectors. We first calculate the best practice level foreach company considering all the questions in the five sections of the questionnaire. To calculate the bestpractice level for a particular sector, we find the average of the best practice levels for all the companies in thesector. For example, if we seek to find the best practice level for the Telecom Products sector, we first find thepercentage of best practices each Telecom company follows in all sections of the questionnaire. Then weaverage across all the Telecom companies to find a best practice level for the Telecom Products sector. Thereis a wide variation in best practice levels between sectors. The following bar graph gives a ranking of thesectors for best practices.
Quite predictably, the Aircraft and Spacecraft sector came out ahead of all the others with an adoption of75% of the best practices. The Telecom Products and Electronic Engineering and related TechnicalConsultancy sectors come out in second and third positions respectively (ahead of Software Consultancy andSupply, Data Processing and Related Services). The Machinery, Electrical and Optical Instrument sectorcomes out last with a 36% adoption of best practices.
Best practice levels for various sectors
Legend
FI - Finance and Insurance
EE - Electrical Engineering &Technical Consultancy
IS - Software Consultancy andSupply
IP - Industrial Process Control IT - Telecom ProductsSystems
ME - Mechanical Engineering AS - Aircraft & Spacecraft& Technical Consultancy
IE - Electronic Engineering & EQ - Machinery, Electrical &Technical Consultancy Optical instruments
17.
58% 59%50%47%45%42%
.0 80% -a?g 70%
60%
271 so%41 40%
a' 30%
• 20%ts.▪
o 10% -0
0%<10 10- 25- 50- 100- >500
25 50 100 500
Software Best Practice Questionnaire - Analysis of Results
SIZE ANALYSIS
Distribution by size of software organisation (no. of software employees)
To calculate the best practice level for companies of a particular size, we follow a method similar to the onedone for the sector analysis. We find the average of the best practice levels for all the companies in the sizerange. For example, if we wish to find the best practice level for companies with less than 10 softwareemployees, we first find the percentage of best practices each company with less than 10 software employeesfollows in all sections of the questionnaire. Then we average across all these companies to find a best practicelevel for companies with less than 10 employees.
We found that the size of the software organisation influences best practice levels. Software organisationswith less than 10 employees, on average, followed about 42% of the best practices, while a softwareorganisation with more than 500 employees, on average, followed about 59% of the best practices. Theinteresting phenomenon is that we observe a monotonic increase in the average number of best practices withincreasing size. Thus, it appears that software organisations with a larger number of employees tend toemploy more best practices than software organisations with smaller numbers of employees.
Best practice levels for companies with various numbers of software employees
REFERENCES1. [ESIJ "Software Engineering Practices in Europe", European Software Institute, Bilbao, Spain, December 1994
18.