Regular use of static code analysis in

team development

Authors: Evgeniy Ryzhkov, Andrey Karpov

Date: 09.08.2010

Abstract Static code analysis technologies are used in companies with mature software development processes.

However, there might be different levels of using and introducing code analysis tools into a

development process: from manual launch of an analyzer "from time to time" or when searching for

hard-to-find errors to everyday automatic launch or launch of a tool when adding new source code into

the version control system.

The article discusses different levels of using static code analysis technologies in team development and

shows how to "move" the process from one level to another. The article refers to the PVS-Studio code

analyzer developed by the authors as an example.

Introduction Static code analyzer is a tool to search for software errors throughout source code. This tool helps avoid

errors at the early stage of developing a software solution instead of later stages of testing or using it.

However, companies do not often manage to get benefit from such tools. There are different reasons

for that. Some projects are merely not suitable for introducing a code analyzer from the economic point

of view, some others are not large enough for the effect to be visible. That is why you should

understand when integrating static code analysis into the development process can be beneficial and

when not before doing this.

Relying on the authors' experience (developing, promoting and selling their own static code analyzer),

the article formulates main considerations that you should take into account when introducing such

tools into the development process.

About static code analysis Static code analysis is a technology of searching for errors in programs by parsing source code and

seeking for patterns of known errors in it. This technology is implemented in special tools called static

code analyzers.

The word "static" means that code is parsed without executing the program. Tools that analyze a

program while it is running are called dynamic code analyzers.

The most popular static analyzers are produced by companies Coverity, Klocwork, Gimpel Software.

Popular dynamic analyzers are produced by Intel (Intel Parallel Inspector) and Micro Focus (DevPartner

Bounds Checker) companies. We should also mention a specialized static code analyzer PVS-Studio

developed and promoted by the authors of this article.

The result of a static code analyzer's operation is a list of potential errors found in code with reference

to a particular file and line. In other words, this is a list of errors that very much resembles the list

generated by the compiler. The term "potential errors (issues)" is used here not accidentally:

unfortunately, a static analyzer cannot be absolutely precise and tell you if some potential error is a real

issue in your code. Only programmer knows this. That is why code analyzers sometimes generate false

alarms (alas, and there is no way to avoid them).

Static code analyzers are distinguished according to the type of programming languages they support

(Java, C#, C, C++) and issues they can diagnose (general-purpose analyzers or specialized analyzers, for

instance, intended for 64-bit or parallel software development).

Kinds of projects static code analysis is relevant for It is reasonable to employ static code analysis not in all the projects but only in medium and large ones.

The discussion of what to consider small/medium/large projects is obviously outside the scope of this

article, but we recommend, relying on our experience, to take static analysis into account when dealing

with projects taking more than 30 person-months. If a software project takes less, it is better just to

have several skilled developers in this project instead of using static analysis. A team of two-four skilled

workers will well manage such a project and provide its good quality from the viewpoint of software.

But if there are either more people working on a project or it takes more than half a year, it is very naïve

to hope that "all we need to do is just write without errors".

Ways (scenarios) of using static code analyzers Let's consider cases when a developer team may find it necessary to use static code analysis. We

intentionally take a case when static analysis is only being integrated into the development process, for

if it has been employed for a long time, there is no sense in discussing the issues of introducing it.

So, suppose there is a team of 5 persons whose job is to port the code of a software project to 64 bits.

Suppose also that the project is written in C/C++. We note beforehand that these assumptions are made

deliberately so that we can employ our code analyzer PVS-Studio in the example. So, the developers fix

main compilation errors, build the application and make the distribution package. They begin to test it

and find out that there are very mysterious errors in the program which appear only in the 64-bit

version. Developers go to Google, type "64-bit platform C++ issues" and among 8.5 million results, on

the first page they see a link to our article "20 issues of porting C++ code on the 64-bit platform" from

which they learn that, as it turns out, there are various issues in C/C++ applications that were not seen

before but appear in their 64-bit versions. In the same article they read that there is a PVS-Studio tool

that will let them find and fix these issues. Then the developers download the tool, study its demo

version, purchase it if it suits, find a number of errors in their code with the help of our analyzer, correct

them and the program gets rid of issues. After that the developers consider the task of creating the 64-

bit version of their program solved and refuse to use the analyzer further since they think they do not

need it anymore.

Here is another scenario close to the previous one. A team of 5 developers encounters an error in one of

third-party modules when developing a Java-application. Unfortunately, they do not manage to find the

error "with their eyes", so they download a demo version of some code analyzer, find the error in this

third-party module, fix it but do not purchase the license for this tool due to limitations of the project's

budget. The results are: the error is fixed, the application is released, and the license agreement for the

tool is not violated. Everything looks well but this way of using static analyzers cannot be called right as

well.

The third way of using static analysis. Developers start to use Visual Studio Team Foundation Server that

provides a capability of launching code analysis for files being added into the version control system.

Some weeks later, the developers turn off the code analysis since adding new code turned into a game

"make the analyzer let us add the file".

All these three scenarios discussed above are not good ways of using static analysis, even despite the

fact that the analyzer helped to find real errors in code in the first two cases and in the third case the

code seems to be absolutely poor. What are the reasons for these failures?

What prevents developers from using static code analyzers to the full

extent Let's study the reasons why all the ways of using static analysis described above are not good.

If a team employs a specialized code analyzer (as in the described case when searching for 64-bit issues),

the temptation to stop using the tool after all the issues seem to be found and fixed is very great.

Certainly, if the 64-bit version of a software product is released, it may seem that there is no much sense

to use a special tool further. However, it is not so. If you refuse to use such an analyzer, some time

(actually some months) later you will encounter those errors that could be detected by the code

analyzer in your new code. That is, although you have the 64-bit version of your application and it was

debugged (some time ago), still the new code might have errors typical of 64-bit applications. The

conclusion from the first scenario: refusing to employ a specialized code analyzer after its work is done

leads to appearance of new similar software errors rather soon.

In the second case, the team decided to use a specialized tool only when they understood that their

project contained hard-to-find errors. After fixing those errors the team refused the tool. The problem

about this approach is that hard-to-find errors will appear in the project again, sooner or later. But the

next time it might be users who will notice them first and not developers or testers. The conclusion from

the second scenario coincides with that in the first case: refusing to use a tool will inevitably lead to new

hard-to-find errors.

In the third scenario when developers refused to use static analysis when adding new code because of

troubles they encountered when adding new code into the version control system, the problem is not in

the static analyzer at all. The problem is in the insufficient skill level of the team. First, the team did not

manage to set the tool so that its warnings helped them. Second, maybe their code was really rather

bad since the analyzer produced so many diagnostic messages.

So, let's formulate the main problems that prevent developers from permanently using static code

analyzers in their work:

1. High price of static code analyzers does not allow developers to use them in small (especially

from the budget's viewpoint) projects. You should just understand that there are projects static

analysis is not suitable for due to economic reasons and not technological ones.

2. A static code analyzer produces many false alarms. Alas, any code analyzer generates false

alarms and in some cases their number might be very large. The reason for this lies in the

philosophy of such tools: it is better to generate ten or hundred false messages than miss one

real error. You should not hope that some analyzers produce fewer false alarms than some

others. It is better to choose a tool that can somehow handle them. For instance, our PVS-Studio

analyzer has the "Mark as False Alarm" function. With its help you can mark false alarms of the

analyzer right in your code, i.e. specify that the analyzer must not generate some type of

messages in some particular line.

3. Bad integration into the development environment. If a static analyzer does not provide for

smooth "seamless" integration into the development environment, hardly will anyone use it.

4. Absent capability of automated launch of the tool from the command line. This limitation does

not let developers analyze the code of the whole project regularly, for instance, during everyday

builds.

5. Absent capability of integrating the tool with the version control system. Although analysis of

new code when adding it into the version control system as described in the example above

caused the developers to refuse using such tools, still the possibility of such integration is useful

in itself.

6. Too complicated or, vice versa, too simple settings of the code analyzer.

A good solution is cooperation of the company that wants to use static code analysis technologies with

the company that provides these technologies. That is, their relations must turn from "buy the tool and

use it" into "buy the solution, integrate it and only then use it". Whether you like it or not, in most cases

you will not manage to get benefit from simply buying and using an "analyzer-applet". You must pull up

the development process in your company and together with the vendor of static analysis solutions

integrate the tool it offers into the permanent regular team development process.

This scheme is employed by the leaders of static analysis market like Coverity or Klocwork. By the way

its signs might be not very visible - it is not so easy to get any demo version from the sites of these

companies. And to get an answer to the question "how much does it cost?" is impossible at all until their

sales-managers get maximum information about the customer.

Summary

If your company plans to employ static code analysis, you must consider the following things:

1. Integration of static code analysis influences the whole development process.

2. A static analyzer is not an "applet" and not another copy of Windows that you may buy and use

without any relations with the vendor. You must always keep in mind that you will have to

establish close communication with the analyzer's developers and that the integration

procedure demands much effort and time.

3. A static analyzer increases the general culture of software development in a team - but only if

the team is ready for this. So, this is a mutual process.

4. Increase of the development's culture through using static code analyzers is an expensive

process. You must be ready for it and understand that it will require large investments.

References 1. Alexey Kolosov. Using Static Analysis in Program Development. http://www.viva64.com/art-2-2-

3613493674.html

2. Andrey Karpov. Changes in programmer tools' infrastructure. http://www.viva64.com/art-2-2-

417409089.html