🧑🏻‍⚖️ Putting Test-Driven Development under Trial: Could the Critics Be Right? 😳 | Learn Agile Practices

Introduction

Few practices in software development have sparked as much debate and controversy as test-driven development (TDD). Advocates think it's a panacea for writing clean, manageable code, while critics argue it's a time-consuming and unneeded burden. Today, we're putting TDD to the trial. Imagine a courtroom in which the prosecution throws charges and the defense rallies to demonstrate that those charges are wrong.

I collected all the critics I found online about TDD, and we will give all those critics the benefit of the doubt, trying to go very deep into understanding the cons that they typically suggest and to support their ideas - then, we will also try to demystify it. In the end, I will share with you what thoughts I had after all this work.

So grab your gavels and robes, for we're about to embark on the TDD trial of the century! 🏛️👩‍💻👨‍💻

Test-Driven Development on Trial 👆

Setting up the stage for the trial, as a (very bad) prosecution attorney, I was able to collect the 8 most common criticisms that people throw against TDD:

Time-Consuming
Too much overhead
Focus on Testing, Not Design
Inflexibility
Over-testing
Difficult to Learn
False Sense of Security
Not Suitable for All Projects

We will deep dive into each of those, in a ping pong challenge between prosecution and defense of TDD. Let’s see which criticism is concrete and which is not, and how concrete and real those problems are raised by skeptics.

TDD is time-consuming

Prosecution

Ladies and gentlemen of the jury, it's essential to acknowledge that Test-Driven Development consumes an exorbitant amount of time. The reason is that TDD necessitates developers to compose tests before crafting the actual code, and this supplementary step serves only to extend the already arduous development timeline. TDD's insistence on creating tests prior to developing the actual code is an irksome and time-consuming ritual.

To elucidate further, allow me to emphasize the problems with TDD's initial phase, where the developer is burdened with the task of devising test cases. This detracts significantly from the rapid inception of the code, leading to longer project timelines. In fact, the documentation-intensive nature of TDD often exacerbates the already onerous demands of software development, resulting in an unwarranted delay in reaching the final product.

What we want to point out and make very clear here, your honor, is that TDD increase time for development, making the life a lot harder - and while we cannot prove that there might be a gain of time on the long run, we can definitely see we are slower in the immediate - and that is not good for business.

Defense

Honorable people of the jury, we vehemently refute this accusation: claiming that Test-Driven Development is time-consuming is something inaccurate that starts from some bad habits about learning and applying a new practice.

What we want to say is that for any new practices we want to learn and apply, we cannot start applying them on the job - we should invest time in learning via exercises and training, starting from trivia software exercises and then moving to more complex ones. If you start doing TDD and try to apply it while still learning and not being confident enough, of course, you will be a lot slower than usual. But this is an unsafe way of approaching new methodologies or technologies on the job, and it’s also disrespectful of the money the company pays us.

The reference here is professional sport: if you want to be able to apply good techniques during the match, where points are at stake and there is pressure, you must work hard first to excel in those practices during training, where you have no pressure and all the time needed for learning. Developing a feature for software that is (supposed to) make money, it’s our performance moment, it’s like we are in the match - there is pressure due to money and time, and that is normal, so you cannot use a practice/methodology/technology you are not confident enough, and you should build that confidence before via training.

And remember: training is a developer's responsibility, not the company’s: “company doesn’t give me time for training” is just an excuse, the responsibility for the quality of our work and our learning is ourselves.

TDD causes too much overhead

Prosecution

The next problematic point we want to highlight about TDD is that the testing code creates an overhead that complicates projects. One of the more troublesome aspects of TDD, as our esteemed critics contend, is the unwarranted overhead it introduces to software projects. This weighty accusation concerns the task of maintaining the testing code alongside the production code, which, according to the prosecution's perspective, considerably exacerbates the complexities of project management.

Now, please indulge me as we delve into the nitty-gritty of this allegation. Our critics astutely point out that TDD's unwavering emphasis on creating tests in tandem with the production code results in an inherent conundrum. As developers, we find ourselves in the unenviable position of maintaining two parallel sets of code - the production code itself and the code responsible for testing it.

This approach adds layers of intricacy to the project. This situation isn't merely an issue of maintaining an additional set of files; it involves ensuring that the testing code is in harmony with the production code. In our experience, such meticulous synchronization introduces confusion and can escalate into an administrative nightmare.

The stark reality is that the act of maintaining these test codes often leads to a decline in overall project manageability. We contend that this complication is just one of many signs that TDD is a practice far too cumbersome for today's fast-paced software development landscape.

Defense

First of all, your honor, I want to point out that is not merely about TDD, it’s a general attack on having automated tests in our codebase - which is even worse, probably, but also easier to defend.

The moment you start wondering how to make software development better and to have a better life at work daily, you start realizing that there is typically a lot of time wasted in unplanned work: bugs, defects, issues in production, and any other kind of activities that you cannot plan because it’s unexpected. Agile, XP, and Lean all agree that we should strive to reduce the time we spend in unplanned work to the minimum possible (around 20% of the time for high-performance teams, according to the research in the “Accelerate” book, against 27% invested by low-performance teams).

Writing and maintaining an automated test suite is a minimal investment that pays off by enhancing code quality, reducing defects, and increasing the maintainability of the code itself - meaning it reduces unplanned work in the future. TDD can reduce that even more, to be honest, because it ensures a couple of additional points:

we are sure all the code is covered by tests without any additional effort
you will write only the minimum code required for your current needs, meaning the YAGNI principle will be respected and you will have to maintain the less code possible
big complex problems/features will not keep you stuck because you don’t need to overthink the entire design beforehand - you can just understand how to start and have a generic understanding, and tests will drive you to the solution

So we can say with absolute certainty that having an automated tests suite is very very helpful, and an investment that is worth doing - and TDD raises even more of that value.

TDD focuses on testing, not design

Prosecution

I'd like to draw your attention to yet another critical viewpoint, one that questions whether Test-Driven Development (TDD) leads us down a path of good design, as its proponents claim. Our esteemed critics argue that TDD's singular emphasis on testing overshadows the importance of promoting well-crafted software, and they propose that it often steers developers toward writing code primarily to fulfill test cases, rather than to foster sound design principles.

This critique, ladies and gentlemen, underscores a fundamental issue with TDD. The practice may encourage a myopic focus on satisfying test conditions rather than nurturing the kind of software that stands the test of time. Instead of nurturing the growth of elegant, maintainable, and efficient code, TDD, according to our critics, may drive developers to prioritize passing tests at the expense of the software's overall quality.

These criticisms, as unpalatable as they may be to some, should give us pause. We are left to ponder whether TDD is truly a beacon of good design or, as our critics assert, a practice that might inadvertently pave the way for poorly designed software. The prosecution contends that this is a matter we cannot afford to take lightly and warrants a closer examination of TDD's value in contemporary software development.

Defense

There are two questions we want to start our response from, to demonstrate that this accusation is completely without basis and concreteness.

The first question is: are you sure that it is bad to focus primarily on passing the tests? As the Agile Manifesto states, the first measure of progress is “working software” - and passing those tests means our software is working, so it is for sure something to celebrate and be happy about.

But the second question will add light also on the first one: are you sure that TDD emphasizes passing tests instead of good design? I think that the name here is probably causing some misunderstanding.

Let me explain: TDD means Test-Driven Development. Development is the process where we add functionalities to a software, and while we do that we also decide the design of the software to achieve that purpose. TDD means that you develop software by letting the tests drive this process, and TDD has a specific target that is stated in the very first page of Kent Beck’s book: “clean code that works”.

“Clean code” means code with a good design, intended as a design that makes it easy to read and maintain the code. “That works” means code that does what we expect from it. Tests we use to drive the process allow us to achieve both: if they are green, our code is working - and thanks to those tests, we can easily refactor our code to improve its design being sure that the behavior is still the same.

The focus of TDD is not only on testing, is “clean code that works” and the test-driven process is the best way known today to achieve that result - it also allows us to split the problem: we write the tests and make it green, focusing on “code that works” - then, we refactor focusing on “clean code”.

TDD doesn’t take care itself of it: it’s up to you to be sure you know what the software is supposed to do to write useful tests, and it’s also up to you to know the best design and refactor practices to apply - once you know those, TDD is the best tool to be sure you achieve them both.

TDD is not flexible

Prosecution

Now, your honor, please let us now cast our gaze upon the notion of inflexibility. As opponents of TDD, we want to present an argument that has struck at the very heart of its methodology, contending that the rigid nature of this approach, which insists on writing tests before any code, may stifle creativity and innovation.

The prosecution asserts that this critique is a striking indictment of TDD's potential drawbacks. We argue that the demand for tests to precede coding creates an environment where creativity and exploration take a back seat, as it's not conducive to imaginative problem-solving or the pursuit of alternative, innovative solutions.

In our ongoing scrutiny of TDD, the notion of inflexibility is indeed a substantial concern. It prompts us to consider whether, in its zeal to ensure testing, TDD might inadvertently suppress the very creativity and out-of-the-box thinking that often leads to groundbreaking software solutions. The prosecution submits that these allegations, while disputed by proponents, are nonetheless serious and warrant careful consideration as we weigh the merits of TDD in the realm of software development.

Defense

Letting aside the attack about TDD being focused on testing, that we already responded to in the previous point, we want to totally refuse the prosecution accusing TDD of being inflexible.

In this case, we can start our responding taking inspiration directly from Kent Beck’s TDD book itself: in the “Mastering TDD” chapter, the author respond to some hypothetical questions that may arise when trying TDD. Among all the incredible insights of this chapter, one of the questions is: “How much feedback do you need?”.

The question is about feedback because tests in TDD are described as a way to have a short feedback loop about our decision, so this question can actually be rephrased to “How many tests should I write?”. And the answer is: it depends!

With a very clear example, Beck shows us how the same problem caused him to write 6 tests, while another developer wrote 65. Quoting the book:

You will have to decide, from experience and reflection, about how many tests you want to write.

TDD's view of testing is pragmatic: tests are a means to an end, the end being “code in which we have great confidence”. Taking this idea to the limit, it means that if our knowledge of the implementation gives us confidence even without a test, then we might even decide to not write a test at all.

In the end, this means that the more you know the problem you are solving with the code, the less number of tests you will feel the need to feel safe - and as a consequence, you will write fewer tests. On the reverse, when you are facing a problem for the first time, you will probably prefer to have a lot more tests to feel safe.

And that sounds like a very flexible approach to me!

TDD cause over-testing

Prosecution

Ladies and gentlemen of the jury, let's restart this trial by delving into another pressing issue surrounding Test-Driven Development (TDD). Detractors claim that TDD, in its zealous pursuit of robust code, can inadvertently lead to over-testing. The concern is that developers, driven by the TDD approach, might find themselves entangled in an exhaustive quest to cover every conceivable edge case.

This meticulous testing, as critics argue, can be a time-consuming and unnecessary endeavor. Picture this: developers navigating a labyrinth of test scenarios, trying to account for every possible nuance. We must ponder whether this exhaustive approach to testing is truly warranted or if it's an undue burden on the development process. As we proceed, we'll unveil the intricacies of this claim and explore whether TDD's emphasis on comprehensive testing is a necessary virtue or an onerous vice.

Defense

Your honor, and ladies and gentlemen of the jury, please excuse me but I’ll have to be repetitive on this one - because once again, the prosecution against TDD has no concrete pieces of evidence.

As I mentioned last time, Kent Beck shows us how the same problem caused him to write 6 tests, while another developer wrote 65. Quoting the book:

You will have to decide, from experience and reflection, about how many tests you want to write.

TDD's view of testing is pragmatic: tests are a means to an end, the end being “code in which we have great confidence”. The number of tests we decide to write will depend on the confidence we have in the problem and a possible solution.

There is another thing that most often is not considered: tests in TDD lead you to a solution, but then you only want to keep with you the tests you need to feel safe when that code changes. It’s not uncommon that once your work is done, you might consider throwing away some of the tests that you used - because they already fulfilled their purpose in driving you through design decisions but they don’t add any confidence when changing the code.

TDD is hard to learn

Prosecution

There is another contention against Test-Driven Development (TDD) that we want to highlight today. Critics vehemently assert that TDD, with all its purported benefits, presents an insurmountable challenge for developers new to the practice. The crux of the matter is the formidable learning curve associated with TDD, which, according to the prosecution's argument, may serve as a deterrent to its widespread adoption.

The prosecution contends that TDD is not just about mastering the art of testing; it entails delving into the intricate principles that underpin effective test-driven development. This dual challenge, critics argue, places an undue burden on developers, especially those navigating the early stages of their TDD journey.

Defense

There are two sides to the medal when discussing the learning curve of TDD: first of all, it requires a shift in the way we think about problem-solving. Most of us learned to develop software in a specific way: code a solution, then test it. TDD requires a shift where you think about your tests before writing the actual code - and not only at a high level but also for every step we make in coding the solution.

The second point is that TDD also brings the approach of small steps and continuous design feedback thanks to the TDD cycle and the fact that we execute the test suite every minute; we are all typically so used to working with long-living branches and getting feedback only after a lot of time - and even if that late feedback is often uncomfortable, a lot of people think it is inevitable.

To learn TDD, you need an Agile mindset: you need to embrace change to accept that there might be another way to do things, maybe even better ways.

TDD is a way to ensure that you produce “clean code that works”, having a simple way to avoid bugs and defects or recognize them very early when they happen; it’s a way of doing one thing at a time, to avoid being overwhelmed by a lot of ideas when coding; it’s a way to improve the design one small step at a time, without getting stuck in implementing a certain design patterns because the step was too big.

I’ve never seen someone having a great level of confidence in those without TDD, and even if you can do it, ask yourself how much energy it requires you - with TDD, it’s all included.

TDD gives a false sense of security

Prosecution

We are not done yet, and now let's delve into another critical assertion against Test-Driven Development (TDD). Detractors argue that the pursuit of extensive test coverage, a hallmark of TDD, might inadvertently foster a false sense of security among developers. The crux of this contention is that developers, ensconced in a cocoon of extensive testing, might be led to believe that their code is impervious to bugs.

The prosecution posits that an overemphasis on test coverage could be a double-edged sword, providing a deceptive assurance that the codebase is devoid of defects. As we scrutinize this claim, we'll explore whether the purported sense of security induced by comprehensive testing is a commendable outcome or a perilous illusion that detracts from the rigor of software development.

Defense

No one advocating TDD will ever tell you that if you do TDD you will never have defects, bugs, or any problem. It would be just false, anyway.

TDD is a way to minimize unplanned work, which includes bugs and defects for sure. Minimize doesn’t mean avoiding completely - you can always make mistakes, for sure, even if TDD minimizes the fact that a mistake is integrated into the main branch and reaches production or even staging. But it can happen because we are humans.

Doing TDD doesn’t change the fact that you should have a good monitoring and alerting system - or more in general, a way to notice problems immediately when they happen, a good rollback strategy if required, and also working in Continuous Integration releasing a small batch of work every day so that even if an issue shows up in production the change will be very little and fixing it will be very fast.

TDD is not a replacement for that, for sure.

TDD is not suitable for all projects

Prosecution

And finally, your honor, let us now address the last critical viewpoint we want to level against Test-Driven Development (TDD). We want to contend that TDD may not be the optimal strategy for projects characterized by swiftly evolving requirements or those in the throes of highly experimental development.

The prosecution submits that TDD's insistence on defining tests before actual code might pose a challenge in the context of projects where requirements are in constant flux. We shall examine this claim closely, scrutinizing whether the rigidity of TDD aligns with the demands of dynamic and experimental development environments or if it becomes a hindrance in such scenarios.

Defense

Well, your honor, here we have something that can be partially true, but… the reasons why this can be true are completely different from what the prosecution suggested.

Is TDD suitable for all projects? Yes, it is. It’s also suitable for startups and small businesses because it will cost less money spent on developing software, most often because of less time invested in giving the feature to customers and reduced unplanned work. “Accelerate” (the book) proved this in the research it is based on.

Is TDD suitable for any coding situation? Maybe not, as always there are special cases.

For example, when you are doing a spike or some thinking around code, you might want to avoid tests, at least at the beginning, because you are just looking around and you don’t have any clear questions to respond to. Once you have it, Test-First is a good way to formalize it, for example, but you might decide to follow TDD strictly or not depending on the situation.

Well, this fake trial finally ended - and it was a good, fun exercise to try to collect all the concrete ideas against TDD and also try to debunk them afterward.

As you all know, I love Agile Practices, but I also want to be sure that it doesn’t become a dogma where I’m not able to recognize the reasons why it is good anymore - so I feel that sometimes trying to discuss against it, even internally, is a good idea.

What do you think? Reply to the email to let me know what you think about TDD and if you practice or are interested in learning it!

Until next time, happy coding! 🤓👩‍💻👨‍💻

Dan’s take 🙋🏻‍♂️

I will not add much today, because the target of the trial is to try to be non-opinionated most of the time, but hey - this is my personal take zone, so it’s opinionated by definition.

First of all, I want to say that it was really fun to build this fake trial: I used Chat GPT to help me write the prosecution part in an attorney language, and after a couple of prompts tests, I was able to get a good enough result, hopefully!

As always, collecting points against something I love is a great exercise: it makes not only reflect on some elements that are actual criticism, but also reminds me of why such practices are great, which is always important to avoid them becoming a blind dogma.

Here are some reflections I made in the process:

A lot of people consider TDD time consuming, while today I’m a lot slower when I don’t write tests before; this makes me reflect a lot on the fact that a lot of people don’t learn enough about something before using it in the real world, and this is bad - whatever it is that “something”. We should consider writing software for production more like a match for a professional sport: it’s our performance, and at that moment we have to win in any way (in our case, getting the result with the lowest cost possible) - it is not the moment to “try something new I never tried before”, for that you should the training to build the confidence you need to then use it during the real performance.
Resistance to change is a real thing for human beings, and it’s hard to fight. But since I started to talk about TDD around, and hosting workshop, I realized that a lot of people just need a bit of support - they start learning TDD but they are not pushed by the environment in growing into that direction, and then it requires a huge willing power to keep going on; if you want to learn TDD and you are having hard times, before giving up, find a mentor!
The “there is no silver bullet” and “there is always a trade-off” are very useful quotes, but sometimes we use them to hide ourselves from the truth. I want to be completely honest: I still haven’t found any actual cons to TDD. I keep looking for it because I want to be a critic to grow even more, but I can’t find it. I’ve never seen someone doing TDD and regretting it or highlighting limits - I’ve always seen people who find it hard and then start arguing against it. TDD produces better software, it’s a fact.

Kent Beck says: “I’m not a great developer - I’m just a good developer with great habits”.

I follow this quote daily, building better habits to avoid facing big issues. If I prevent the fire, I don’t need to learn how to face it.

Go Deeper 🔎

📚 Books

Test-Driven Development: An Empirical Evaluation of Agile Practice - this book includes the results of three different experiments, including concrete examples of how to conduct statistical analysis, using as evaluation indicators the number of acceptance tests passed (overall and per hour) and design complexity metrics.
Test Driven Development: By Example - the original book from Kent Beck that explains TDD, with examples, and also explain some pitfalls for experts practitioners
Learning Test-Driven Development - Author Saleem Siddiqui shows you how to tackle domain complexity using a unit test-driven approach. TDD partitions requirements into small, implementable features, enabling you to solve problems irrespective of the languages and frameworks you use.
Growing Object-Oriented Software, Guided by Tests - Implementing TDD effectively: creating cleaner, more expressive, more sustainable code using tests.

📩 Newsletter issues

Everything I know about introducing TDD in an org [The Engineering Organisation by svs]
Introduction to Predictive TDD [Make Your Code More Testable by Ted M. Young]
Use Test-driven Development to Retain Modularity [Crafting Tech Teams by Denis Čahuk]

📄 Blog posts

🎙️ Podcast episodes

TDD for today [Thoughtworks Technology podcast]
Unit Testing With Test Driven Development [Complete Developer podcast]
TDD with Ian Cooper [The Unhandled Exception podcast]

👨🏻‍🏫 Online courses

Test-driven Development: CodeMash [Pluralsight]
Introduction to Test Driven Development (TDD) [Coursera]
TestDriven.io [a platform dedicated to TDD courses]
TDD micro-learning - Our micro-learning courses catalog offers, among the others, a path dedicated to TDD: learn the core concepts at your own pace in an interactive learning experience

🕵️‍♀️ Others

➡️ How effective is Test-Driven Development - a research written by the University of Oulu, Carnegie Mellon University and Fraunhofer Center for Experimental Software Engineering

➡️ Quora post from J.B.Rainsberger about TDD effectiveness