This is a translation of my article in Russian
In this article, I want to describe my experience of paying off technical debt on our project in the form of a guide. In this guide, I will highlight some of the most common cases of technical debt and suggest methods for solving them. Since this is a rather extensive topic, I will recommend several books for study, because I do not see it possible to talk about everything within the framework of this article. Everything described applies to the BackEnd part, but it may be suitable for other developers. I would be glad if you share your experience on this topic in the comments.
How does technical debt appear?
In simple words, technical debt arises when the priority is to quickly achieve business goals, which leads to good short-term business results, but further complicates the support and scalability of the project.
There are already many definitions and types of technical debt in the world, but I will describe only 5 problems that we encountered on our project, and outline how to prevent them, and if they did, how to fix them.
Poor code and architecture
Definition
The largest section, where the most technical debt can form.
Code written on a tight schedule, without proper planning and adherence to conventions, can result in it being difficult to understand, maintain, and extend. This means that every time you add a bug fix or feature to such code, your technical debt will grow exponentially (pic. 1). And this, in turn, will lead to the fact that in such a code there will be more and more bugs and critical errors, which in turn already affects the business, because due to frequent breakdowns, user confidence will decrease . This is a good argument to convince business/customers that more development time is needed or that a certain amount of time needs to be set aside to pay off technical debt.
Poorly built architecture at the initial stage can further lead to a limitation (development slowdown) in the scalability of the project and the complexity of adding / changing components: programming languages, frameworks, databases, caches, message brokers, integrations with external systems, etc.
Fix
To avoid bad code and architecture, you need to read a lot of books and redo a bunch of different projects. But here are some basic tips:
Follow the unified style of writing code that is accepted in yours, if there is none, then create it. Attach a linter to your project and run the code through it, make it impossible to deploy the application without the approval of the linter.
Refactor your code regularly, use the best design patterns, and keep it readable and maintainable.
Follow SOLID principles, design patterns, and architecture styles to keep your code well structured and scalable.
Regularly review previously written code to ensure that the code is of high quality and meets the requirements. If necessary, make improvements to reduce technical debt.
At the end of the article, I will suggest several books in which these points are described in more detail, following the advice from these books, you can ensure that your code is of high quality, well structured and maintainable, and free from technical debt.
Tests
Definition
In my opinion, insufficient code coverage with tests or the use of bad test cases can also create technical debt.
Incorrect testing processes can lead to missed bugs, vulnerabilities, and other issues that can lead to technical debt. If these issues are not resolved prior to deployment, they can lead to increased maintenance costs, reduced user satisfaction, and decreased trust in the application.
Fix
The main thing in tests is that they should at least be :) Code coverage with tests should ideally be close to 100%, but some developers consider 80% to be an acceptable result. Is it possible not to test at all or not to test enough? It can. But this will lead to technical debt, which will slow down the speed of further development.
Here are some of the main types of tests you need to write:
Unit tests are the main tests that a developer writes. This kind of test is written for a small piece of code to make sure it correctly handles the behavior you put into it. Once written, you later run them when adding a new codebase to make sure that the new code does not break previous developments.
Integration testing is needed in order to catch bugs that were not caught in unit tests. Accordingly, we move the level of the code under test higher, to the level of interaction between different pieces of code. For example: integration between different components of the online store system - shopping cart and payment system.
There are also different types of testing: functional tests, end-to-end tests, performance tests, security tests. But if the team is large, it is done by other people, if not, it can also become your responsibility.
Overuse of resources
Definition
Inefficient use of resources can lead to technical debt as it can increase the cost of maintaining and operating the application. For example, a system that uses memory, processing power, and storage resources inefficiently may be slow, unresponsive, or crash-prone. This can negatively affect the user experience and lead to a decrease in satisfaction and trust in the application.
In addition, inefficient use of resources can cause an application to require more resources to run and maintain, resulting in higher hardware costs. This can lead to increased development time for new features and updates, reduced competitiveness, and increased technical debt.
Fix
Here are some optimization tips:
Optimize algorithms and data structures: Make sure that the algorithms and data structures used in your code are the best for the given situation.
Use libraries and tools that are known to be efficient and well-optimized for your use case.
Use caching and other performance optimizations: Implement caching and other performance optimizations to minimize resource usage and improve overall efficiency.
Test and profile code regularly to identify performance bottlenecks and areas where resource usage can be improved.
Legacy
Definition
There was no legacy problem on our project, because the project is new, and, accordingly, all the latest technologies that were at that time were used. But there were some problems in the related microservices that I had to fix. Over time, outdated technologies, conventions, anti-patterns can become technical debt that will prevent the application from growing.
Fix
Before changing old technologies, you need to make sure that it is necessary. As they say: if it works, don't touch it. Here is a plan on how to decide and what to do next:
Assess impact: Evaluate the impact of legacy technologies on your code, including any compatibility, performance, security, or support issues.
Identify alternatives: Examine and evaluate alternative technologies that can replace legacy technologies and identify the advantages and disadvantages of each alternative.
Plan your transition: Develop a detailed transition plan from legacy technologies, including timelines, resource requirements, and any potential risks or issues.
Take a phased approach. Consider a phased migration approach, starting with non-critical systems and gradually moving up to more critical systems.
Implementation of the plan. Implementing a plan to phase out obsolete technologies, including any necessary code changes, developer training, and support system updates.
Monitoring and evaluation: tracking the impact of the transition and evaluating the results, making any necessary adjustments to the plan.
Continuous Update: Regularly evaluate the technologies used in the code and make updates as needed to keep the code up-to-date and efficient.
Documentation and logging
Definition
Ignoring writing documentation for your project can also lead to technical debt. It is worth paying close attention to this part of those. debt, because the documentation will directly determine how quickly new developers will delve into the project and how long they will write their code. The longer there is no documentation or no detailed documentation, the more difficult it will be to enter the project, and it will become more and more difficult for current developers to make changes.
I also include logging as a potential technical debt, because incorrect or insufficient logging can lead to slower debugging and analysis of application performance.
Fix
A few tips for documenting your code:
Comments. Add comments to your code if you understand that another developer, focusing on the names of classes, methods, variables, will not understand what he is doing at first glance, in other words, write comments if there is some tricky business logic. However, some influential developers consider comments an anti-pattern, so they should be used rarely and with caution.
API documentation. Services like Swagger allow you to very quickly and conveniently describe each specific endpoint of your API, and are also used to send requests and pre-test your code.
README files. Create such files to describe the top-level, for example, microservice architecture, what it does and what functionality it has. You can also add important points, without which work with these microservices is impossible, everything else is better to write in the Wiki.
Wiki pages. Here you can already describe in more detail about each complex component that you have developed. What microservices does the project have, a section with best practices, some important problems that were solved in the past and now it affects the current development. In general, all the answers to questions for a new developer on your project.
Logging can be of different levels, each of them needs to be applied in certain situations. The most important of them:
INFO level - information logs that are needed to track the behavior of the system, for example, to track the addition / removal of elements in the database.
WARNING level - logs that indicate certain potential problems that must subsequently be analyzed and resolved.
Level DEBUG, TRACE - logs that are used for debugging and which provide more detailed information about the behavior of the application for diagnosing, troubleshooting or testing the application. TRACE provides even more advanced information
ERROR level - information that there is an error that needs to be fixed urgently, possibly with a hotfix.
As you may have noticed, I did not describe in detail every detail of how and what to do, because it is impossible to fit this whole big topic into one article. Therefore, this is called a mini-guide, in which I gave directions, what to look for and briefly described the basics. In order to understand it in more detail, you need to read more than one book and write many, many lines of code. Here are some good books I recommend to become a better programmer and never meet tech debt again ;)
Clean Code - Robert Martin
Refactoring - Martin Fowler
Object-Oriented Design Patterns - Erich Gamma, John Vlissides
Grokking Algorithms - Aditya Y. Bhargava
Designing Data-Intensive Applications - Martin Kleppmann