© Copyright Software Engineering Competence Center 2012 1€¦ · · 2017-07-18In Software...

© Copyright Software Engineering Competence Center 2012 1

Amr Noaman


Amr Noaman

These are mapped categories to the waste categories of manufacturing.

An excellent overview of these waste categories in: http://agile.dzone.com/articles/seven-wastes-software

This topic is very important in Configuration Management, because many traditional CM processes inject many wastes in all these categories.


Amr Noaman

http://agile.dzone.com/articles/seven-wastes-software





In simple terms, the four common concepts in these definitions are:1. Given many copies of the same artifact, you will be able to identify which version

each copy is. 2. Ability to induce information about the work products, like their release date,

related issues, etc.3. Make sure that a given set of files constitute a correct version. Also, ability to

make sure that a given set of work products fulfill a set of functionality. 4. Verify that released products are packaged correctly, and fulfills requirements.

These definitions may not be that understood to practitioners. Why, for example, shall I bother about configuration items’ specific versions? I would rather make that I’m working on a correct branch and that’s it !


Amr Noaman

These two are the and most practical definitions of all. It mentions the single a most important objective about configuration management, which is tracking and controlling changes.


Amr Noaman

From another angle, the diagram identifies a property of a strong configuration management environment. This is the ability to trace:• Trace releases to its work products. • Trace releases to workitems worked in this release.• Trace workitems to work products which resulted from them.• Trace workitems to their releases. • Trace work products to releases.• Trace work products to workitems which produced or changed these work

products.

If I can do this, then I have a strong configuration management environment


Amr Noaman


Amr Noaman

Following is an explanation of every process increment:

Version ControlProject configuration items are under version control, and team is trained on basic copy-update-merge and lock-modify-unlock procedures

BaseliningBaselining procedure is defined and enforced at points where work product(s) are delivered to an external party

Workitem TrackingIssues types are identified, and issues are managed and tracked

TraceabilityBi-directional traceability of requirementsand work products is defined and enforced

Release ManagementRelease and release scope is identified; Changes are received, prioritized, and planned; packaging, releasing, and post-release verification procedures are enforced

CM EnvironmentProject structure is defined, access rights are enforced, and backup/restore procedure is employed

Continuous IntegrationBuilds are automated and integration between team members, and between teams is automated and frequent


Amr Noaman

This is the first and most basic technical practice in software development. However, you may still find so many teams doing version control in an unproductive way.


Amr Noaman

Identifying configuration items is an important task in software projects. It is not enough to put your code on svn. There may be other work products that need to be versioned.

For example, consider a vast product developed by several teams. At first, an architecture team agrees on the architecture, develop component diagram, and identify components interfaces. Then, teams start working, and after a while, the product get. In this case, the architecture document is a configuration item, and need to be baselined. If not, there is a great risk that teams fail to integrate their work successfully.


Amr Noaman


Amr Noaman

This is the first and most basic technical practice in software development. However, you may still find so many teams doing version control in an unproductive way.


Amr Noaman

The issue here is what would be the nature of a change management process? In Agile, change management process is the usual development process, nearly no overhead to mention.

Many companies employ a sophisticated change management procedures which incur a lot of cost to introduce a single change request. This effectively keeps customers afraid from changing anything, and fires back at time of deployment.


Amr Noaman

Again, take care of the baselining procedure. It is tempting to put all kinds of overhead in this procedure.


Amr Noaman


Amr Noaman

Workitem tracking is a central topic in Configuration Management. However, agile team usually overlooks it. Most of the teams I have worked with (even teams thinking they are Agile), do not have a clear definition of what workitem types they should track. Even if they have a basic setup of Scrum workitems (user story, task, bug, test case), they do not have an understanding of what state information they should collect, what kind of relations to be maintained between workitems, and what are the optimal workflows for every workitem type.

One sign of a mature agile team is that they start creating new workitem types which suites their own environment, and store specific state information about every type. They also know what to do with these workitems. They not only use it to track their everyday development activity, but they also use it to generate release reports, document customer correspondence, and advanced reporting.


Amr Noaman

Workitems are things that are carried out throughout the project. In Project Management terms, these are the Work Breakdown Structure (WBS) of the project.

Many teams limit themselves to predefined types in management frameworks like Scrum, RUP, etc. Or even to predefined types in tracking tools. Workitems need to be well thought off with special attention to two things: Existing management system, and Hidden Factory.

Existing Management System/Framework:Organizations give different meanings to terms like: Requirement, Enhancement, Change Request, Defect, Support Request, Trouble Incident, etc. It is very important to identify what is meant by the name, and when it should be used. In general, it is advisable to re-use the same terms already used in the organization, but it is necessary to write down what is meant by every term, and when it should be used.

Hidden factory:Hidden Factory can be defined as the set of tasks that are done to amount for the process inefficiency. For example, rework due to integration bugs, technical debt accumulated due to very little time spent on unit testing, etc. These tasks are usually not tracked as workitems, or are never tracked. Usually, a good practice that Agile teams do is that these workitems are done as part of several intermediate ‘stabilization’ iterations, or at the end of every release as ‘Release Iteration’. However, if such work get accumulated, tracking is usually overlooked in order not to let others discover the process/team inefficiency.

The golden rule is: “Hidden factory is bad. Try to eliminate it. If you can’t, track it as part of your process”

Another factor that may help identify workitems is value-stream mapping. This is a technique used in Lean manufacturing to track the progress and flow of materials and information. In Software development, value stream mapping help identify activities and intermediate work-products which result from each activity. Kanban boards are an example of a simple value stream, in which materials (requirements) and products (software) are mapped and tracked.


Amr Noaman

Workitem relationships are rules defined as part of the definition of the workitem system. Usually, they are optional, but in other times they are mandatory.

In some teams, workitem relationship rules are not defined, although all kinds of relationships is open to create (using a tool, may be). This is an example of a process that may never be improved, because there is no process to start with. It is important to note that having a process definition is half way towards process improvement.

There are many examples of critical situations where we needed to trace some workitem to another and we couldn’t, due to the loose rules defined in workitem relationships.


Amr Noaman

Automating workitem tracking systems is a controversial issue. Many teams do workitem tracking on a Scrum or Kanban Boards, and that’s it. Many others use simple excel sheets, one sheet per iteration or release.

This level of automation is very little to achieve lean implementation of configuration management. For example, we need to report the status of what has been accomplished in this release. If we do not systematically track workitems on a tracking tool, it will be an overhead activity to generate this report from row paper or simple excel sheets. Another point is how backup will be done, who is responsible for it, and how restore will take place?

Are Workitems configuration items? Usually yes. Because we need to track the history of changes done to this workitem.

Can they be part of a baseline? How? Sometimes we need to include a use-case survey report as part of the baseline, or attach a list of user stories delivered for every release. In this case, they are baselined.


Amr Noaman

Traceability is one of the major and key process increments of configuration management. As will be explained later, good traceability policy and implementation empowers the team with techniques to do impact analysis, root cause analysis, and status accounting. These tools are key tools in improvement.

Nevertheless, still a lot of voices are talking about traceability as a real challenge, and a major problem area in the industry. It may induce costs and overheads which “grows exponentially with the size and complexity of the software system”1. Also, “despite many advances, RT remains a widely reported problem area by industry”2

This is why most of the teams (regardless team or project size) I have worked with do not pay attention to implementing traceability. And, the few teams which employs the so called “Traceability Matrix” are complaining that it is pure overhead, and they are only doing it for compliance purposes.

1 http://www.crosstalkonline.org/storage/issue-archives/2009/200907/200907-Kannenberg.pdf2 http://eprints.ucl.ac.uk/643F156E-1448-4974-9200-17ECF56449AA/FinalDownload/DownloadId-F8A63C958BA3EF3176E8AFD0B48687DA/643F156E-1448-4974-9200-17ECF56449AA/749/1/2.2_rtprob.pdf


Amr Noaman

http://www.crosstalkonline.org/storage/issue-archives/2009/200907/200907-Kannenberg.pdf










http://eprints.ucl.ac.uk/643F156E-1448-4974-9200-17ECF56449AA/FinalDownload/DownloadId-F8A63C958BA3EF3176E8AFD0B48687DA/643F156E-1448-4974-9200-17ECF56449AA/749/1/2.2_rtprob.pdf

























































I found teams who implement traceability for compliance purposes mix traceabilitywith traceability matrix. Traceability matrix is one way of implementing traceability.

Ok, what is traceability?

Rather than introducing a formal definition, it is easier to look at the network and deduce what traceability is. Requirements entities can trace to each other (white), and trace to other information (green), and finally trace to physical artifacts (gray).


Amr Noaman

Forward traceability enables the team to study the impact of a change and assess its costs and risks. In this scenario above, the customer has requested a change on an already implemented user story. To assess the change, the team revisits the written code, the impacted design and products, etc.


Amr Noaman

If there is a defect reported by a customer, and we want to know its root cause. As in the scenario above, the defected code is traced back to the reasons of change, which is found to be a change request by a customer requested sometime ago.

This is also called Backward traceability.


Amr Noaman

Any kind of reporting in software projects makes use of traceability one way or another.


Amr Noaman

These are the set of objectives which need to be fulfilled:1. How release and release scope are identified2. What is the procedure for receiving changes, then prioritizing and planning them?3. Why and how baselines take place, and what is included in baselines?4. How does the team package their work, and what is the procedure for releasing

and verifying releases at customer environment?

Some Agile teams overlook many details of the above, and this may cause problems throughout the project.

The good news is that Agile methods have done excellent job defining release management procedures. In the next slides, I will try to identify some Agile practices that does the above requirements very well.


Amr Noaman

These are some rules defined in Scrum, and governs how release management is done.

First, the customer can propose changes anytime, during review meetings, while estimating, during informal reviews inside iterations, etc. What is important is that these changes are not carried out on spot, they are kept in a Backlog of workitems.

The Customer and the Product owner work together continuously to refine the details of the backlog items, and activity called ‘Backlog Grooming’, in which:• The backlog items are detailed• Big requirements are broken down• Business priorities are revised

Backlog Grooming is a continuous activity that continues to take place throughout the project.

Requirements that reached a good stability level are estimated. Then, next iteration’s scope is identified based on business priority, technical risk, and cost. This effectively fulfills the necessary impact analysis for the flow of changes requested by the customer.

Still, there is a very important rule in the next slide


Amr Noaman

This rule is a golden rule which governs the change management process in Scrum, and makes change control possible.

Some agile teams do not abide by this rule, and it results in the following symptoms:- Team receives inconsistent requirements or even contradictory requirements,

which makes the team override several old requirements- Velocity cannot be calculated, and iteration length is not kept constant- Team start to be less productive, and customers start to complain

The next slide outlines an end-to-end change management procedure in a typical Scrum Agile team.


Amr Noaman

This is a typical change management process in Agile. This is a description of the process:

1. Requirements and other request types are gathered into a backlog as backlog items

2. Team (including product owner and/or customer representatives) plans the next release.

3. Development team plans the next iteration, and start development. 4. After iteration ends, the team set for a demo and iteration review. 5. Typically, changes are requested, which are gathered in the backlog.6. Impact is assessed and may result in changing the scope of the release, which in

turn may result in changing the contract. 7. Re-iterate

In the meanwhile, changes may occur due to other business circumstances other than usual review in the iteration review meetings. These are referred to as Business Changes (drawn as an accept event action).

What is interesting about this process is that changes do not need a special procedure to be developed. In other words, the development process is the same as the change management process.


Amr Noaman

This is one of the interesting Agile practices:

Development Process is the Change Management Process

What is interesting about it is that it removes so many wastes due to a separate change management process. In traditional processes, change management is done in a separate path (or even separate process) rather than the development path, and this introduces the following wastes:1. Task switching between the development and change management tasks. 2. If two team are involved, late integration between the teams may introduce

defects.3. If teams who apply changes are different from the development team, learning is

required, which is another type of waste. If they do apply changes without proper learning, they may introduce defects.

4. Extra management effort to plan, estimate, track two separate paths of development.

In large teams, these problems are magnified, and may cause the development process to halt altogether, and all the team only work on high priority changes only!


Amr Noaman

For teams larger than 10, it is advisable to split the team, and let every sub-team take a specific responsibility. ‘splitting may be done on business feature/function base, or on architectural module base.

If you note, the roles above is slightly changed. There is a Chief PO who takes care of the business along with the team PO’s. Also, there are multiple Scum masters who are representing their teams in product level planning and analysis.

Several notes about this approach:1. There are always integration risks resulting from the fact that multiple teams

working separately for a time period, then come together and integrate. In general, it is advisable to “integrate frequently, but not that much”. This means that every team is responsible to integrate his work on a mainline and keep all other work unchanged. This will come later in Release Branching Patterns.

2. Product level impact analysis activities are taken up one level, to the group of Scrum Masters working on the product level.

3. Change management is typically the same. However, on the product level, it takes place several iterations (every release). In this case, the product should be planned to have several internal releases before any public release is available.


Amr Noaman

This is a typical packaging and releasing procedure. These procedures are very risky and if not managed well may result in project failure and a lot of overhead costs.

Several important waste:- Environments are not defined, which may inject defects and incur relearning costs

every time a deployment is done. - Improper baselining take place, or very basic code baseline.- Integration defects due to very late or non-frequent integration- System integration defects due to un-defined system interfaces- Most important one: Defects injected due to human factor. Any manual step in the

process may inject defect.

The key solutions in lean packaging and releasing are:1. Automate as much as you can of this procedure to prevent defects due to manual

operation2. Frequent integration (every iteration at least). This is related to the ‘Done-Done’

definition, an interesting practice in Agile. In large and risky projects, a ‘Done-Done’ definition which stops at the development (or even testing) environment is a poor definition, because it concentrates integration risk at the end of the release. Rather, a typical mitigation for integration risks is to integrate frequently every iteration.


Amr Noaman

For small teams, CM environment can be set up in one or two hours. However, for larger projects with multi-teams, CM environment have other dimensions of complexity, and need to be well thought off, and unified across all teams.

These are some complexities related to large teams:- Team members may move across teams and will need to re-learn the CM

environment upon every move. - Code level integration depends on a unified view of the project structure on the

CM tool. - Release management may not be possible except if proper release level branching

& merging techniques are employed. Otherwise, incorrect releases may take place and costly post-deployment defects may incur so much extra costs.


Amr Noaman


Amr Noaman

A unified project structure is very important. In a multi-team project, all team should work in harmony.

This is just an example of a project structure. These are some notes:- The project has three mainstream called ‘Trunc’- There are other active streams called ‘Branches’ every branch is an active code

stream that may be updated anytime a new change request is approved. In this case, the responsible team checks out this branch and start working on it

- The last folder is the ‘Tags’, which keeps baselined streams- For work products which may not be part of any baseline (like meeting minutes,

and periodic reports), it may be kept in a special ‘Archive’ folder


Amr Noaman


Amr Noaman

Backup & restore procedure is an indicator of team discipline. A team with no or inactive backup plan is a team which indicates little discipline and is introducing a great risk in the project.


Amr Noaman

The intent of this section is to highlight the topic of ‘Release branching & merging’. The patterns presented are just a sample of so many others.

Release level branching & merging is a different from individual development branching & merging practices. The former deals with the whole release, which is a responsibility of one whole team. The latter deals with an everyday development activity.

Factors that may determine which pattern to follow are:- Number of development teams working in parallel- Number of customers and amount of customization for every customer- Cost or risk of Integration- Required release frequency


Amr Noaman


Amr Noaman

Due to the major risks related to integration. Continuous integration is added as a standalone process increment for Configuration Management.


Amr Noaman

This is the de-facto model for integration between teams. This model has so many problems and cause so many delays. The reasons why this model doesn’t work is mentioned in the next slide.


Amr Noaman


Amr Noaman

© Copyright Software Engineering Competence Center 2012 1€¦ · · 2017-07-18In Software...

Documents

Transcript of © Copyright Software Engineering Competence Center 2012 1€¦ · · 2017-07-18In Software...