Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta...

24
Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan and Karo Mazidzhyan

Transcript of Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta...

Page 1: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Improving Software Testing by

Observing Process-Ossi Taipale

-Kari SmolanderLappeenranta University of

Technology, Finland

Presented by Albert Saryan and Karo Mazidzhyan

Page 2: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Breakdown Introduction Related Research Research Process Analysis Results

Process Improvement Propositions Conclusion

Page 3: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Introduction The objective of this study was to

understand how software testing is conducted by observation.

From observations propose improvement to the testing process.

Improvements by reducing development and testing costs, and improving quality.

Page 4: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Software Costs and Quality Software Engineering strives to reduce

development costs and improving quality. Software Process Improvements (SPI) are

the means to reaching these goals. Commitment to SPI’s by all from all

organizational levels is key to success Quality can be tested into products or

developed and built into products

Page 5: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Software Costs and Quality Cont. External events such as deadlines affect

software quality. The cost of software testing is high,

therefore SPI’s are necessary to reduce cost.

Page 6: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Related Research Involvement of testing during

development occurs when testers develop test for developers to analyze

The complexity of testing increases as a function of the complexity of the systems under testing.

Testing strategy defines the contents of testing.

Page 7: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Related Research Cont. Communication and interaction between

development and testing processes requires cooperation and coordination.

The use of software components are increasing rapidly

Design outsourcing and distributed development increase the use of components.

Cost of Quality is “Free”, but being late with products may be more costly than fixing faults.

Page 8: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Research Process This study consisted of Organizational

Units (OU) which develop and test technical software for automation or telecommunication in Finland.

Initial Sample included 26 OU’s, from which 5 were used as case studies.

Cases were chosen to show polar types

Page 9: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Research Process Cont. Data for the research was collected by a

series interviews. Each interview had a different theme in

mind and possibly a different interviewee in mind.

The interviews took place during five rounds, based on the theme.

Page 10: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Research Process Cont. Interview Rounds1. Development and Testing Managers were

asked to define their testing process .2. Managers of Testing were asked to

define their testing process in depth.3. Testers were interviewed.4. Systems Analysts interviewed.

Page 11: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Research Process Cont.Case Breakdowns

Page 12: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Research Process Cont. Data Analysis

Information gathered from these interviews were then categorized.

Categories were then analyzed to see how they were connected.

The categories were then used to identify factors which affected testing.

Page 13: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Results Description of Cases:

Case A - Developed and tested Manufacturing Execution Systems :

Turnover 50% product, 50% service Services included systems integration and customization Testing against requirements was a challenge because

customers had special in-house requirements standards Developers and testers worked physically close to each

other Time allocated to testing was consistent, although over

time it has been reduced Use of components low, hinders testing

Page 14: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Results Cont. Case B - Tested in house products and

provided testing services for external customers:

Turnover 75% service, 25% product Majority of requirements specifications were based

on standards. Delays in development allowed for fewer time

allocated for testing Communication flexible, developers talked face-to-

face with testers Use of components high, testability of components

must be considered

Page 15: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Results Cont. Case C – Customized Software Development:

Turnover 2/3 service, 1/3 product Testers were involved early, involved in development

process Testers often had issues due to lack of advisement

from developers Delays in development does not often result in

reduction of testing time Developers and testers communicate face-to-face Use of components low Testing of software components seen as difficult b/c

of different implementation.

Page 16: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Results Cont. Case D - Electronics:

Turnover 100% product High product orientation required high quality

because recalls are very expensive Avoided testing of unfinished product Testing tasks clear and well documented Testing involved in development late but planning of

testing automation provided information on upcoming tests

Communication between developers and tester planned, formal and transparent

Use of components high Components tested initially by suppliers then again at

system testing.

Page 17: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Results Cont. Case E – Software Testing Services:

Turnover 100% service Working as an external testing organization required the

adaptation of the process of the customer Early involvement of testing was necessary for the testing

company to increase the testability of the software Sometimes testing was involved late Budgets for testing affected testing time Communication was handle through a contact person, but

was active and clear Use of components depends on customer As an external testing organization it was hard to receive

information about customers purchased components.

Page 18: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Results Cont.

Page 19: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Results Cont. Cause and effect

Page 20: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Result Cont. Cause and Effect

Business Orientation Directly associated with use of components and testing

schedules Business Model – value adding process

Purely Service Oriented System integration Customizing Consulting Customers directly affect development and testing process

Purely Product Oriented Product Development Marketing Customers do not directly affect development and testing

process

Page 21: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Results Cont. Process Improvement Propositions

Testing ought to be adjusted to business orientation

Product oriented should adopt formal planned testing process

Service oriented should adopt a flexible testing process

Enhanced Testability of software Consider testability when selecting components Review testing process of suppliers

Page 22: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Analysis Results Cont. Effective Communication and interaction

between development and testing Early involvement of testing and planning of

testing Use of risk based testing

Page 23: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.

Conclusion Proposals by observing best practices using

grounded theory Better documentation improved testability of

software and components Efficient communication between development

and testing improved quality When time is a major issue, risk based testing is

the best solution Business orientation affects:

Use of components Amount and quality of communication Allocated testing time and the planning of testing

Page 24: Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan.