We all strive to build better software, and to do it faster. I believe Test-driven Development offers us a path towards this goal.
Introduction
Test-driven development
At its core, TDD is a simple, never-ending Red-Green-Refactor cycle:
Red: Write a failing test for a new piece of functionality
Green: Write the minimum amount of code necessary to make the test pass
Refactor: Once you know it works, make the code appealing to the eye
Do you use it on a daily basis? Have you tried it but didn't find it appealing? If you answered "no" to either of these questions, I invite you to explore TDD together in this article.
Why are people struggling with TDD adoption?
Despite its benefits, why do many developers struggle to consistently adopt TDD? Many developers find the idea appealing but face challenges in adopting it consistently. In my opinion, these are some common reasons why:
Steep learning curve: Developing the skill to write effective tests and integrate them into the development workflow takes time and practice. Determining the right tests to write upfront can be challenging. However, this initial investment leads to more robust and maintainable code in the long run.
Existing codebase: Introducing TDD into projects with a large amount of existing code – with tests written after the fact or, even worse, no tests at all – can be difficult and time-consuming. For this reason, people often give up on using TDD in real-life projects. However, I agree that while challenging, there are ways to overcome this and make your code more testable, preparing it for the test-first approach moving forward.
Difficulties in setting up the test environment: Complex applications might require significant effort to create an isolated and reliable test environment. This is usually the case when starting to test already written code, where mocking dependencies and configuring the system for even simple tests becomes increasingly difficult. You might end up with hundreds of lines of setup code, while the test itself takes only a few. However, good architectural design and isolation can help mitigate these challenges.
Lack of immediate benefits: The advantages of TDD might not be immediately obvious. The question arises: why spend extra time initially? While many people are excited by the concept on paper, this lack of immediate payoff can lead to TDD being deemed simply impractical. However, the long-term benefits of TDD compound over time, resulting in fewer bugs, easier refactoring, and greater confidence in the codebase.
Despite these challenges, the rewards of TDD can be significant. Let's now look at the potential benefits that TDD provides and find out if it's worth the effort.
What benefits TDD gives me?
I want to share the reasons why I believe it's a worthwhile practice. These are the benefits which I personally see from using TDD, benefits that have significantly improved my own development workflow:
Confidence in what the system should do: In my opinion, the main difference between test-first and test-last approaches lies in what they represent. Imagine a ceramic vase. You've created it yourself from scratch, and then you shape the mold around it. Next time, you can use the mold to verify that another vase has the same shape. But this doesn't guarantee the mold itself reflects the intended shape. However, if you start with the mold and express your intentions through it, the vase produced by it will undoubtedly meet the expectations, and so will subsequent ones. The same applies to tests. Tests written after implementation often represent only what the code does, not its intended behavior. TDD allows us to start from the other end and declare our expectations in an executable format. This test-first approach ensures that our code is built to meet specific, predefined expectations, leading to greater confidence in its intended functionality.
Confidence when project is evolving: Requirements are constantly changing. In reality, clear requirements are often absent. People usually do not know what they want. So-called requirements are often just educated guesses. Therefore, in this dynamic environment, the ability to adapt quickly without breaking existing functionality becomes crucial to stay competitive. Well-written tests are your safety net, catching you when you fall so you can get up and try again. While badly written tests can strangle and suffocate you to death. Your test suite can become an obstacle, preventing you from moving forward or refactoring. Making a small change can then lead to rewriting hundreds of tests that suddenly stop compiling or fail in unexpected ways.
Ensures the software is easy to use: TDD shifts your perspective on the software you build. It puts you on the other side, making you a user, even if only for testing purposes. You will inevitably start seeing different usage patterns, perhaps asking yourself why it can't be easier or more streamlined. On the other hand, you don't lose the internal view, as you still need to configure the system for effective testing. This temporary shift in perspective allows you to anticipate user needs and identify potential usability issues early in the development process, ultimately leading to an architecture that balances ease of use with the necessary configuration flexibility, avoiding excessively exposing internals only for testing.
Ability to quickly verify use case theories: Sometimes people ask "What if?" questions. A product owner might wonder, "What if the feature is used in an unintended way?". The QA engineer asks, "What if malformed input is provided to the system?". You might want to ask the business analyst, "What should happen if I do this or that?". In such scenarios, TDD's focus on defining expected outputs for given inputs provides a straightforward way to translate these theoretical questions into executable scenarios.
Prevents previous bugs from reappearing: Bugs are inevitable. Passing tests doesn't say anything about the number of bugs your software has; they only indicate that none have been found yet. So, when they finally appear, you need to be prepared. You need to prevent them from reappearing by exposing the problematic behavior and setting the correct expectation via the test and assert. Following TDD not only gives you that power but also produces tests strong enough to survive future refactorings, which you will also inevitably undergo.
In the following sections, I will illustrate how embracing TDD can provide significant advantages in the long run, addressing initial hurdles and ultimately leading to more efficient, reliable, and faster development. Overcoming the initial struggle is crucial; TDD needs to 'click' and fit into your broader development philosophy for you to continue following this approach. Personally, I didn't grasp its value immediately; it was only later that I finally began to see how it improved me as a developer. It took me quite a few attempts to get started and many more to find a way that truly boosts my productivity. In the next chapter, I hope to share insights that can help you on your journey.
Tips and tricks
Tests must be fast
If you are catching yourself scrolling through social media on your phone or looking at your second monitor while tests are running, your tests are probably not fast enough. Although it really depends on the project, we should strive for all module tests taking no more than 5 seconds, ideally. Otherwise, developers start losing focus when running them, often deciding to skip them and run later. The feedback loop becomes too long. TDD implies you to execute the test suite repeatedly, get fast confirmation about the stability. While striving for fast module tests, it's crucial to remember that integration tests also have their place, and TDD principles can certainly be applied to them. Your main focus is on the domain, but do not leave integration points without attention; you want to test them separately, but test nonetheless. It is just that TDD works best with a fast Red-Green-Refactor cycle, and doing any I/O, HTTP, or using containers will slow it down significantly.
Aim for the smallest change possible
Moving in small steps is essential to be able to confidently stand on your feet. The worst time to make any decision is right now, because you have as little knowledge about a problem as possible. By postponing the decision, you make room for new knowledge coming as time passes. By taking a huge leap when changing a hundred files all at once, you gamble on the solution fitting the problem. And even when in the end you fail, you might have found valuable pieces you want to keep. So next time, do not jump straight in; exhale, relax, think about the next smallest test you can write and make green in a couple of minutes. If you feel the need to branch off the current execution path, first do the refactoring to support branching (e.g., create an interface for multiple implementations) while preserving the initial behavior (e.g., initial implementation), and as the next step, create a new execution branch (e.g., a new implementation). Sometimes you see a new opportunity for a new test. Hold on to it, write it down in your to-do list, just focus on your current goal for now. Moving forward this way not only gives you the ability to trace back if needed and try a different approach, but also as a bonus, you get a small dose of dopamine each time making a commit with an although small but an improvement.
Move quickly to the green stage
When you have the failing test, you want to make it pass. Spend as little time as possible in the red stage; you can write the ugliest, the slowest code possible — all that matters is that it works. Because during the transition to the green stage, we want to discover the solution, to prove that it solves the problem at hand. Has it happened to you that you've spent a considerable amount of effort ensuring the code looks good only to find out in the end that it does not actually do what you need? You might as well discover other approaches with other benefits to consider. I like the mantra attributed to Kent Beck, the pioneer of TDD: "First, make it work, then make it pretty, and if necessary make it fast". This order is deliberate. Prioritizing working code first allows us to validate the solution before investing in aesthetics or performance. Premature optimization can lead to wasted effort and potentially more complex and less maintainable code. Also, the third part is optional; I've seen very few examples where speed was the primary need. Assuming you made it to the green stage, you are now open to opportunities to either improve the solution you've picked or maybe try out another one. Remember, previous git commits and your test suite are there to guide you and, if needed, help you make one step back and then two steps forward.
Abstract away all external dependencies
Your domain, your business logic, your main module, the exact reason why you are writing the software, the very heart, must be protected from the outside world. The framework, third-party libraries, the database, the cache, the message broker, everything else should be abstracted away. In SOLID principles, the Dependency Inversion one tells us that high-level modules should not import anything from low-level modules; both should depend on abstractions (e.g., interfaces). As an example, let's look at the classic layered architecture.
There is one violation: the service layer depends on the repository layer. The business logic depends on a storage mechanism and will need to be changed if the storage changes. To decouple, we can utilize one simple trick: we'll create an interface in the service layer representing an external dependency without mentioning the technology used for the implementation. For example, we need a way to add a new book into the system and later get it back by its ID. What will be the actual implementation — an SQL database, a NoSQL database, an in-memory database, a cache, an HTTP call to another service — it does not matter.
What matters is that the main module defines what it needs, becomes the owner of the interface, and implementations obey the contract. They do not dictate how to use them; the business logic dictates expected behavior instead. In the end, your architecture will resemble more of a Hexagonal one (or Robert Martin's Clean architecture). You expose the entry point into the domain for the framework or test suite to call and ports for external dependencies to be integrated through adapters.
Mocks are not your friends
Most of the time, we do not want to use our real dependencies during testing; we want to replace them. The problem with using mocks may not be obvious at first glance. You start using them; it takes only a couple of lines to set up. But modern systems are usually built on top of many other dependencies. So you mock more and more; the mock configuration part becomes so long that by the end of it, you forget what you wanted to test at the start. Through the pain, you make it pass, you move on on your adventure. But eventually, you stumble upon it again, trying to refactor the functionality being tested. And each little move results in mocks breaking immediately. That's what they are supposed to do; they are supposed to be very fragile. While it has its own merits, ultimately it prevents the software from being refactored; it does not allow it to be evolvable. This is one of the most crucial parts of success, this is the ability to adapt and change the system under new market conditions. And we should not let that slip away so easily. As an alternative to the potential pitfalls of mocks, consider using so-called fakes instead. They are simplified implementations of dependencies, offering a working substitute for the real component (e.g., an in-memory database). An obvious concern could be that it takes some effort to develop such fakes. But from my experience, writing a database substitute based on a HashMap and performing complex queries using Streams takes 20 minutes at most. Sometimes you also need the dependency to fail, but that could be another fake (e.g., a database which always throws an exception).
Testing through public APIs
Admit it, the end user does not care if the code goes to an external service or calls a particular method in a particular class. What's really important is the observable behavior, inputs and outputs. To write effective tests, you need to look at your system, your module, from the outside. Ask yourself: "What is the most effective way for an external entity to interact with and benefit from this system?" Focus on the domain (e.g., create an order in an e-commerce system) rather than on a particular technology (e.g., save an order in the SQL database). Build the API, entry points, and route test code through it. This allows implementation details to remain hidden inside the module under test, because they are ultimately unimportant and always subject to change. As an added benefit, public APIs change less often, which allows you to perform refactorings without fear of fixing hundreds of test cases. Have you ever found yourself in a situation when extracting part of a class into a separate file, or splitting one method into multiple, caused many tests to fail and you spent significantly more time fixing those rather than doing the refactoring? This exact situation we are trying to avoid; it will hold you back from evolving the system.
Always run the test to see it fail
Always run a new test and confirm its failure before proceeding. Do not underestimate the importance of this step. You need to see the test fail exactly the way it is supposed to fail. I've been in a situation when the test finishes with an error I didn't think of. It would mean I need to look for other things to fix to be able to proceed with the initial idea. I've also been in situations when the test passes right away. Your mind often is not able (and presumably does not want) to fit the entire complicated system you develop with all its nuances. And it is perfectly fine; congratulate yourself with a job well done and move on to the next task. Remember, without seeing the red test, you cannot say for sure that the code you've introduced contributed to the test becoming green. Our goal is to avoid unnecessary code, only write the bare minimum to make the test pass.
Avoid Primitive obsession
I believe at some point any developer creates a method or a class having multiple parameters of the same type and then later accidentally passes arguments in the wrong order. A good example can be an addBookToCart method with userId and bookId parameters of the same integer type and passing bookId first and userId last instead. Even in tests, such a simple human mistake is actually quite hard to catch. Do yourself a favor: treat a compiler as your first line of defense. Avoiding primitive obsession by using value classes (e.g., Java records). In the example above, if you had used types like UserId and BookId, the compiler would have been actively helping you and pointing at misplaced arguments. It is a very simple technique, although making the codebase a bit verbose, pushes the domain language into the heart of the system. In the end, it helps in staying on the same page with domain experts and evolving the product iteratively together. I would strongly suggest trying it out if you do not use it already.
Control the time
Occasionally, when asserting the output, time based values can get in you way, usually, timestamps. You can ignore them for a while, but I've found it more convenient to control the time instead. The existing Java class Clock can help you create an instance always providing fixed values, which helps to keep the tests repeatable. Unfortunately, there is no built-in implementation to manually advance the clock, however it will take you no more than 5 minutes to write your own. No need to even use another library! Another common situation is scheduled tasks. How to test the code, which is meant to run only in midnight, you might ask. To find an answer, you need to change the perspective. The key here is to separate the trigger from the action. Define what you want to happen at some point in your domain, call it in a test case, assert the output. Then it is just the matter of changing the trigger for the production environment.
Module testing
I want to emphasize your attention on the term Unit testing. People usually think about testing one class or one method when they hear about unit testing. While this can be the case, it is not necessarily true. Martin Fowler in his article defines two types of unit tests: sociable and solitary. Contrary to sociable, solitary tests are mocking all class dependencies to isolate the failure if it happens. However, in my practice, I've found the most effective way is to combine both these approaches into one. I prefer to use the term Module testing for it instead of simply Unit testing. The idea here is to treat the module/the system as a single piece and test it in isolation from external dependencies. It is still possible to trace down the source of the issue if it occurs, even when multiple classes are involved in a single test. When it comes to test cases separation, I like to put them in different classes representing different features. For example, in a domain about selling books, there can be BookCatalogTests, ShoppingCartTests, CheckoutTests.
Acceptance testing
Usually, people try to test the system itself: the business logic, the database, the classes calling other classes, etc. However, TDD, from my point of view, works best with so-called acceptance tests. They are more focused on what we want to assert, not how (e.g., asserting the output, not methods called). They verify that the software satisfies business requirements from the perspective of an end user, treating it as a black box. It is very easy to come up with one; just ask your business analyst or project manager. For example, given an existing user and an existing book, when the user adds the book to the shopping cart, then the cart contains the book. Such a requirement is very easily translatable to the test code and leaves out implementation details, allowing you to change the way it is fulfilled later. Core business capabilities described by acceptance tests very rarely change. You can also opt in for a more traditional unit testing for some parts of the system. However, I would personally suggest sticking to this style. If you feel like testing needs more emphasis on a particular subset of business requirements, maybe it makes sense to separate this part of the system into a new component instead and then import it as any other dependency. Focusing on these high-level functional objectives, acceptance tests provide a clear direction for development and ensure the software delivers value to the end user.
Similar code, different rules
Many colleagues of mine treat the tests the same way as the production code. However, the goals differ: production code should be robust, configurable, and reusable, while tests should be easy to read and to understand, isolated and fast. While we strive for good coding practices in our production code, the priorities for test code are slightly different. For example, usually we try to avoid magic numbers and strings for obvious reasons, but in tests I find them to be more expressive. Contrary to jumping through files, trying to find the exact value hiding in a constant, a field or a variable to understand a test failure, simple numbers and strings allows you to see the whole picture reading the test case. The same can be applied to duplications. Normally we follow the Don't Repeat Yourself principle, but in tests copying some parts achieves better isolation. When you get back to them, you will be happy not wasting time reading the whole suite, only one use case you are interested in should be enough. To add more context, more meaning to help future you consider using long descriptive method names. Again, it is a good idea to adopt standard java code style, but specifically for tests you can set free your imagination! There are many templates for test method names, I like this in particular: givenBookExists_whenUserAddsBookToCart_thenCartContainsThatBook. There is also an option to define a display name, but I've found 'physical' name often is enough.
Where to start?
So you're ready, you're sitting there staring at the monitor. You are doing TDD, you know what you want to achieve. But you stuck trying to come up with a first test to get you started. When I find myself in such situation, I like to use ZOMBIES to start moving forward. ZOMBIES is an acronym which stands for:
Zero: A case for a zero number, empty string or no elements. For example, given there are no books, when the user lists available for purchase books, then the user sees no books for purchase.
One: A case for a single value. For example, given a single existing book, when the user lists available for purchase books, then the user sees only that book.
Many: Usually many in testing means two values. For example, given two books, when the user lists available for purchase books, then the user sees only these books.
Boundary: Focus on testing the edges of your inputs. What happens at the maximum or minimum allowed values? Are there any off-by-one errors? For example, given the book has a title of length more than maximum allowed, when the user adds book to a catalog, then the exception gets thrown.
Interactions: Think about other collaborators and dependencies that your system interacts with. Come up with scenarios where they are involved. For example, given a book is rated 5 stars in an external rating system, when the user views the book, then the user sees 5-star rating.
Errors: Test how your code handles errors and exceptions. What happens when invalid input is provided, or a dependency fails? Are you providing enough context for a user to figure out what went wrong? Given a book with the title already exists, when the user add another book with the same title, then the user sees an error with the description "The book with the same title already exists" and with the book's title.
Simple: Start with simple, focused tests that directly address the requirements of the unit you're testing. Ensure each test has a clear purpose and verifies a specific behavior. Avoid overly complex tests that try to do too much at once.
Legacy code
It is rather easy to adopt TDD from the start of a greenfield project. However, introducing TDD into a long-running project with poorly designed abstractions and brittle or nonexistent tests can be daunting. In such cases, my best bet would be to grab and put all your things in a box, hide the current solution behind a single domain interface. Then think about the simplest use cases, test them first to see you new setup allows for TDD. You can still keep the old suite for a while, with a goal in mind to gradually replace it with a better one supporting refactorings and you going forward. Ensure your primary test scenarios are of sufficient speed. Identify what's slowing tests down, should it be spinning up containers or waiting for a message to arrive, replace them with something more speedy to keep you inside a feedback loop. Do not forget to actively refactor and look for opportunities to achieve better separation of concerns. From my experience, it is definitely possible to retrospectively adopt TDD, and at the same time not to cripple new features development too much. I was feeling more freedom and confidence in a way how I can put more stuff into the product without it falling apart.
A tool, not a restriction
Ultimately, remember that TDD is a tool to serve you. Experiment with these tips, and do not forget that everything that's been said is meant to provide you with benefits. Try it out, follow it to the letter for some time, see if it works out for you. But do not hesitate to step a little bit away from the beaten path or drop something out if it restrains or harms you too much. Do not omit unit testing classes and methods completely, use mocks sometimes, move at a speed suitable for you through the Red-Green-Refactor cycle, skip some steps if you are confident. Do not be dogmatic; evaluate all options and adopt the most useful ones.
Summary
I hope you've gained some fresh perspectives and practical insights today. If you've been hesitant, I urge you to reconsider TDD. Remember, it's not an innate talent, but a skill that grows with practice. Thank you for your time and attention!
Useful links
P.S. I encourage you to share your opinion on the article itself (would you change anything, do you see images fitting, etc.) or the things it describes (tips on TDD) in the comments
