Co-author: @el_kudro

Tools to improve and control code quality can be a key success factor in a complex software project implementation. Static analyzers belong to such tools. Nowadays, you can find various static analyzers: from free open-source to cross-functional commercial solutions. On the one hand, it's great – you can choose from many options. On the other hand – you have to perform advanced research to find the right tool for your team.

What is static code analysis and what's its purpose?

Before we talk about selection criteria, let's briefly discuss the methodology. Static analysis is an automated method to find errors in source code by means of specialized programs. The purpose of such analysis is to identify unsafe and problem parts of the project that can lead to more serious consequences – vulnerabilities.

Keep in mind that users are not QA engineers. Bugs must not get into the release. That is why it's better to use a static analyzer regularly. Run the analyzer once – get lots of messages. Believe us, you don't want to sort them out. Moreover, to find and fix some errors you have to use more expensive methods at the following project stages.

The best way to use a static code analyzer is to run the tool regularly and fix the detected issues immediately. For example, you can run the analysis every day along with night builds. Thus, you can find and fix lots of errors at an early stage of the project development. You do not have to wait when they get to QA engineers or users.

If you want to learn more about static analysis, how it works, its pros and cons – read this article on our website. It will dispel popular myths about static analysis.

Selection criteria

Don't rush to browse how to integrate a tool into your development process. Don't start testing every tool you find. At first, estimate the effectiveness and simplicity of integration. Then, evaluate your workflow, define basic needs of the business and the team. Following these steps, you'll get the best solution seamlessly without wasting time and resources.

You definitely don't want your employees to burn out searching for the right static analyzer. Now, let's discuss the main features of the best analyzer.

Programming language

You may find this weird, but this stage is important. You need to know the size of the project's code base in a language that your company uses.

For example, you have a project with multiple languages – C++, PHP, Java, and others. And the C++ code is not the major one. Therefore, you should not choose the analyzer for C++ only that has a large set of rules. Of course, such an analyzer will find errors and issue warnings, but it won't be as effective as it could be. You can also use other methods (such as code review) to maintain the code quality of a project up to 10,000 lines.

Therefore, if your company uses multiple languages, decide whether you want to check them all or some of them. Sometimes, the more languages the analyzer supports, the more superficially it checks each of the languages and vice versa.

Integration

As a rule, a team already has configured development tools as an integral part of the SDLC (Software Development Life Cycle) workflow. To simplify your and other departments' workflow, the integration of static analysis should be smooth and easy.

For example, popular IDEs have ready-made plugins. They allow you to spend less time and effort on further configuration. You don't have to use multiple apps and sync your data.

Otherwise, you have to deal with several tedious tasks:

firstly, you need to find out how and how easy the analyzer integrates into your development process;
secondly, you have to create, test, and continuously support your import, export, and reporting mechanisms with the help of API platforms;
thirdly, some difficulties arise in the work process anyway.

If your system is unusual, you have only one option – contact support. They must help you.

False positives

Many developers don't like static analysis tools because of false positives – especially if they have to implement a tool into a large project. The code runs, but the report contains dozens of false positives. Such fuss makes code analysis inefficient and complicated. It undermines all efforts and raises doubts about the tool's efficiency. Moreover, this discourages the development team.

In fact, it's not a big deal since analyzers provide various mechanisms to configure and suppress unnecessary warnings. You can mark false positives and customize the product so that further you can analyze your code without excess fuss. Alas, this is an inevitable process. You have to interact with it at the initial stages to simplify further checks.

Frequently, programmers give up a tool because it doesn't provide false positives suppression. Even if the tool will benefit in the future. That is why the analyzer should provide the baselining analysis results feature for such error messages. The mechanism allows you to return to messages if required. Otherwise, you may have no resources to sort out all the warnings at once and find bugs.

Documentation and support

Developers must provide a clear documentation for a tool. The documentation must explain all the details of the configuration and use. It helps developers to integrate the tool and simplify its use. Great when documentation has examples of how to fix errors in addition to the general configuration tips. As a result, users get similar cases of code errors to understand even the most confusing rule.

Of course, tips and documentation can't solve all issues. Therefore, you must be sure that the tool provides high-quality expert support that will help you to solve issues and achieve your business goals.

Here are the peculiarities of good support:

the level of competence. How good the experts cope with the problems;
the reply speed. How quickly the support team replies to your request.

Have you encountered a situation when support throw your question from one manager to another? At best, they reply to you a month later. Obviously, this is poor support. Besides solving clients' issues and supporting users, a good company should be able to implement clients' "wishes" on request. Thus, the users can configure the analyzer as they need.

Coverage of safety and security standards (OWASP, CWE, SEI CERT, MISRA, AUTOSAR)

It's crucial to check the code according to safety and security standards to reduce risk when developing software. Standards save your projects from security vulnerabilities. Besides, standards play a very important role when you evaluate and choose static analysis tools.

If you know that your project requires a certain standard, you have to study in detail the features of each tool. How well it covers various types of errors, coding standards, and depth of analysis. For example, the number of MISRA rules a tool supports, whether it covers the OWASP TOP 10, whether the CWE TOP 25 includes its rules, etc.

Types of diagnostic rules to meet your goals

The analyzer rules should meet your goals. For some projects, only the code quality and clarity are important: array index out of bounds, typos elimination, undefined behavior, buffer overflow, etc. For the automotive industry, MISRA, AUTOSAR and other standards of reliable development are important. For financial software, you should consider known application security issues (OWASP, SEI CERT, CWE, etc.) in the first place. Some analyzers have combined rules that support several standards at once. There's no good or bad option. There are the goals you want to achieve. The market offers many solutions according to your needs.

Analysis speed

Static analyzers can check a project in several minutes or in dozens of hours. During analysis they build a semantic model and study it from different angles.

Various factors affect the speed of code analysis:

the project size;
the server performance;
the depth of analysis.

Usually, programmers check large projects at night. Therefore, the analysis should not exceed 10 hours. By the time the developers and their higher-ups get back to work, they can view the results. You can try the following methods to speed up this process:

use IncrediBuild to speed up the project analysis;
try a more powerful server;
check only the most error-prone components.

If the above or your methods didn't help, you can contact support. The support team should help you to configure the tool or fix this "weird" bug in the next release.

Reports

Sometimes, to find the best analyzer, developers use the following approach. They run several analyzers on the same code and compare the results. Then, they choose the one that issues the greatest number of warnings. As a result, such quick and superficial review may produce long warning reports with many false positives. Usually, it takes lots of efforts and time to work with such reports. Moreover, it discourages the team from implementing the analyzer.

Therefore, in addition to the number of errors, reports should contain information about the quality of errors. This helps a developer to understand whether this error is fatal and prevent further failures. The analyzer that sorts out errors by the level of certainty simplifies the work. Thus, you can focus only on high-level errors and disable those with a high percentage of false positives.

Obviously, the reports of the static analyzer are useful for developers. However, the results are also important for managers. Every manager sets their performance indicators. Project and product managers are concerned about the overall risk level in dynamics. They need the number of vulnerabilities to reduce from day to day, or at least not increase. They also want to see the name of the person who made an error.

Reputation and reviews

The tool's reputation is crucial when it comes to the security of your project. Before you choose the analyzer, try to answer the following questions.

What companies use this tool?
How do the developers rate the tool?
Does the company contribute to the community?
Are there any verified reviews by experts?

If you can answer all these questions – you're likely to choose the right tool.

License cost

Professional static code analyzers are not cheap. They may have different payment models. Static analyzers rarely have solutions for an individual programmer. Usually, such tools have a B2B sales model. The number of programmers in the development team, the additional support or audit, and other factors affect the price. As a rule, subscription and one-time payments are available. In any case, you are unlikely to find the exact price on the official website. Usually, companies provide a time-limited license or tools with limited functions.

Some professional commercial solutions give a free license for different users: Open-Source projects, educational institutions, Microsoft MVPs, etc.

Doubt that the tool pays off? Not sure whether it saves programmers time? Then, consider ROI (return on investment). For example, read an article in our blog where we discuss whether the PVS-Studio implementation is worthwhile.

Besides commercial professional solutions, you can use free Open-Source tools. In this article, we do not compare them and their pros and cons. Many of the above criteria – the support team, seamless integration, and clear documentation – are poorly developed in Open-Source solutions. This is a complex topic for a separate article. If you don't want to miss this topic, subscribe to our blog on Habr and follow us on social networks :)

Let's sum it up

For effective performance, you definitely should use all these tips together. The tool must be user-friendly. Otherwise, no one will use it properly. Choose the product that easily integrates into your workflow without affecting the SDLC process. The right tool will help you to prevent unnecessary costs.

Choosing the tool, take into account rule sets and the coverage of CWE, MISRA, OWASP, AUTOSAR, SEI CERT, etc. These criteria ensure the code quality – the code will be fine-tuned and clean by the release.

Don't use the analyzer to punish developers. Static analysis tools' aim is searching for bugs, not looking for programmers who made a mistake. To make the most of this methodology, you should explain to your team why it's important to use static analysis regularly.

Static code analyzers can't replace programmers. These tools simplify the work and provide error reports. Only developers can verify and fix all vulnerabilities.

Don't hesitate to contact support. If your team has questions about the tool, that's totally fine!

Determine the budget and estimate return on investment. If you have some doubts, you can always request a trial version.

Promo

By the way, we develop the static analysis tool. You can try it on your projects and test its capabilities. You can get a free one-month trial by clicking the link : )

How to choose a static analysis tool