Test-Driven Development

Traditional software development versus TDD. Source: Koskela 2007, fig. 1.3.
Traditional software development versus TDD. Source: Koskela 2007, fig. 1.3.

In traditional software development (such as the Waterfall Model), the order of work is design, code and test. Test-Driven Development (TDD) reverses this order to test, code and design/refactor. In other words, writing tests becomes the starting point. For this reason, TDD is sometimes called Test-First Programming.

It's common in traditional workflows to have lengthy design and implementation phases. Once code is "frozen", testing begins. TDD instead recommends small incremental cycles. This leads to frequent feedback and immediate course correction. Continuous Integration (CI) and Shift Left are practices or techniques that are aligned with TDD.

TDD has its roots in Agile methodology and eXtreme Programming (XP). While TDD brings many benefits, it may not suit all projects. Project managers must access the context and apply it accordingly.

Discussion

  • Which are the main steps in TDD?
    The test-code-refactor cycle of TDD. Source: Kralj 2021.
    The test-code-refactor cycle of TDD. Source: Kralj 2021.

    The TDD cycle or process has three distinct steps:

    • Test: Developer writes a unit test first. Since the corresponding feature is not yet implemented in the application code, the test should fail.
    • Code: Developer writes the code with the goal of quickly passing the test. Other existing tests should also pass to confirm that nothing is broken.
    • Refactor: Design is implicit in the preceding step. Since developer might have written the code quickly, this step is an opportunity to improve the design. Since tests are in place, developer can confidently refactor and improve the design.

    The three-step process is also called Red-Green-Refactor, where red implies a failing test and green implies a passing test.

    Sometimes the TDD cycle is described in five steps: understand the requirements first, execute the three steps of test-code-refactor, and finally repeat the process.

  • What's the essence of TDD?
    TDD is essentially test-first in small increments. Source: Erdogmus et al. 2010, fig. 1.
    TDD is essentially test-first in small increments. Source: Erdogmus et al. 2010, fig. 1.

    While testing is an essential aspect of TDD, TDD is not about testing. Tests are used as a means towards clean code that's less buggy. TDD is a way of developing software. It's not about how to write or execute tests. It's tests driving implementation. For this reason, it's been said,

    Only ever write code to fix a failing test.

    While testing first is certainly important, studies have shown that the real benefits come from incremental development. Developers must work on small features and in short cycles of test, code and refactor. To work on large and complex features that take many days or weeks is the wrong way of doing TDD.

    TDD gives developers quick feedback on whether the code works as desired. Developers can refactor code more confidently since failing tests can catch problems early on. A related aspect is that tests are written by developers, not by a separate QA team.

  • What are the benefits of TDD?
    Causal network of TDD benefits and contributing factors. Source: Buchan et al. 2011, fig. 2.
    Causal network of TDD benefits and contributing factors. Source: Buchan et al. 2011, fig. 2.

    Tests in TDD are derived from requirements. Tests therefore specify precisely what needs to be built and nothing more. This helps us avoid shipping a wrong product or an overengineered product. Tests are "living documentation", helping us understand both the requirements and the code.

    TDD can help developers avoid "paralysis by analysis". By breaking down the requirements into smaller parts, incremental and consistent progress can be achieved. These parts becomes less coupled and system design becomes less of a monolith. TDD's incremental nature helps the creative process since the developer focuses on one small part at a time.

    Tests become a safety net. Developers can fearlessly experiment or refactor. They can continuously improve the design or implementation.

    TDD creates unit tests that become useful within a CI/CD pipeline. Tests can be made to execute at every build or code commit. When tests fail, developers notice it immediately. This avoids costly integration effort later on.

    TDD encourage developers to write more tests. With more tests, debugging time reduces. There are fewer defects. Code becomes more cohesive and less coupled.

  • What techniques are there for writing tests in TDD?

    Start with small tests. For example, a feature or bug fix may involve many building blocks. Test each of those building blocks before attempting an end-to-end feature test. Each small test may run in isolation or within an end-to-end workflow containing stubs for the other blocks.

    Unit tests should be automated and they should run fast. They shouldn't depend on one another. Tests that are hard to initialize, have many dependencies, cover many scenarios or show little reuse are code smells. Too many tests per class or too many mocks are also code smells. Test code should follow SOLID design principles.

    Units tests shouldn't create side effects such as calling an external API. Instead, mock external dependencies. This makes tests deterministic and repeatable.

    Have a good naming convention for test names. When such a test fails, its name will immediately suggest what aspect of the software isn't working.

    Tests may assert states of objects, values in databases or return values. Alternatively, tests may assert messages that are exchanged between two blocks of code. Whether state-based testing or interaction-based testing, adopt what makes sense for the project.

  • What are some best practices for TDD?

    Management and developers must first commit to TDD. This commitment can help overcome old habits and migrate from test-last workflows. Avoid partial adoption where only some developers in the team use TDD.

    TDD doesn't imply that we don't need a QA team. While TDD covers unit testing, the QA team can look at integration and system testing. In fact, TDD may not be the best way to test for concurrency and security.

    Code refactoring can be about changing structure or improving design. Refactor in a controller manner and in small steps. Refactor to change internal structure without affecting external behaviour. Moving from one design pattern to another is an example of refactoring.

  • What are some criticisms of TDD?

    TDD requires initial effort. Since developers don't write application code until later, progress can be slow. TDD involves extra effort due to refactoring.

    Even 15 years after the birth of TDD, studies have failed to observe the benefits of TDD. There's no strong evidence that TDD improves code quality and productivity. Lack of testing skills among developers is a limitation.

    One study found that testing first doesn't contribute to the benefits of TDD. The main contributing factor is TDD's incremental process.

    Often requirements at the start of a project are vague, even from the client's perspective. They evolve as the project progresses. By following TDD, tests will need to be rewritten often as requirements evolve. This is extra effort. TDD advocates that developers write tests. The developer could make the same mistake in both application code and test code. This defeats the purpose of testing.

    By focusing on unit tests, TDD compromises system-level design. It leads to complex interactions, indirections, conceptual overheads, command patterns, and more. Hard-to-unit-test code is not necessarily bad design. In fact, integration tests are better than unit tests for controllers under the MVC pattern. System tests are better for views.

  • What are some variations of TDD?
    TDD can overlap with BDD and DDD. Source: Satya 2020.
    TDD can overlap with BDD and DDD. Source: Satya 2020.

    TDD unit tests verify isolated pieces of code. But do these parts satisfy high-level requirements? This issue is addressed by Acceptance TDD (ATDD). Tests are derived from specification and requirements. ATDD works at the system level whereas TDD works at the implementation step of each feature. ATDD improves external quality whereas TDD improves internal quality.

    Behaviour-Driven Development (BDD) is derived from TDD. It shifts the focus from testing to behaviour, requirements and design. BDD could be seen as TDD done right. BDD has been called by other names including Story TDD (STDD), executable acceptance testing, and specification by example.

    In the world of microservices, Contract-Driven Development (CDD) can help test interfaces from the perspective of both consumers and providers of service APIs. This mitigates the problem of finding problems later during integration testing. An earlier form of CDD was called Agile Specification-Driven Development that combined TDD and Design by Contract.

    Domain-Driven Design (DDD) can work with TDD for experimentation and iterative design. One suggested approach is to think outside in (BDD), view the big picture (DDD) and then think inside out (TDD).

  • What tools are available to practice TDD?
    Tools and frameworks that support TDD. Source: Erdogmus et al. 2010, table 2.
    Tools and frameworks that support TDD. Source: Erdogmus et al. 2010, table 2.

    Many programming languages and IDEs support TDD. There are frameworks that support unit testing, mocking, end-to-end testing, or acceptance testing. Other frameworks support variations of TDD such as BDD. The figure shows a selection of these.

    Consider a Node.js project as an example. We note some useful tools: Node Version Manager (NVM) for versioning, Jest for unit testing, ESLint for linting, Prettier for formatting, and lint-staged for linting on staged files via the pre-commit Git Hook.

    Some IDEs can generate stub methods or modify method arguments based on the usage in test cases.

Milestones

1957

McCracken writes in his book Digital Computer Programming that tests may be written before coding. Moreover, it's advisable that such tests are written by the customers rather than by the programmers themselves. This helps to bring out misunderstandings and logical errors.

1960

In the early 1960s, programmers at NASA working on the Mercury Space Program write test cases on punched cards before writing the program code. They work on half-day iterations doing test-first micro-increment cycles. Their approach is top-down with stubs. Engineers on this project were doing incremental development as early as 1957.

1994

Kent Beck codes the first version of SUnit test framework for Smalltalk. About a year later he demos TDD to Ward Cunningham at the OOPSLA conference.

1998

Kent Beck coins the term TDD in his book Extreme Programming Explained: Embrace Change. A point to note is that TDD has always been a part of Extreme Programming although only now it's being named TDD.

2002
Cover of Kent Beck's book on TDD. Source: Beck 2002.
Cover of Kent Beck's book on TDD. Source: Beck 2002.

Kent Beck publishes a book titled Test Driven Development: By Example. He describes TDD as "a proven set of techniques that encourage simple designs and test suites that inspire confidence." The goal is "clean code that works." He also notes that the idea of writing tests first is not new. Years later he notes that he "rediscovered" TDD rather than invented it.

Mar
2003

An early study of TDD shows that it produces code that passes 18% more black box test cases compared to the Waterfall model. TDD tends to produce more tests. However, developers took 16% more time.

Sep
2003

Dan North starts working on JBehave as a replacement for JUnit. He introduces a vocabulary around behaviours rather than tests. Tests should actually describe behaviours. This becomes the starting point for Behaviour-Driven Development (BDD). Inspired by DDD, BDD introduces (in 2004) a common language to describe user stories and acceptance criteria.

2004

Story TDD (STDD) is born as an XP practice. It brings together developers, testers and customers to discuss the requirements before any code is written. Chunks of functionality are grouped into stories. Stories are detailed. Tests are written for them. Thus, everyone arrives at a common understanding of what's to be built. In 2010, Park and Maurer review the literature on STDD.

2008
TDD detects mistakes earlier and their fixes are often obvious. Source: Wingman Software 2023.
TDD detects mistakes earlier and their fixes are often obvious. Source: Wingman Software 2023.

In a blog post, Grenning presents a useful visualization of how TDD saves time and cost. He compares it to the traditional approach of coding first and testing later, something he calls "Debug Later Programming". He makes the point that bugs in code are unavoidable and therefore testing is essential. When a test fails in TDD, we usually know the problem since only small changes have been made. Feedback is immediate. This avoids long debugging sessions.

2010

In a survey of Agile practitioners, 53% claim to use TDD. At an Agile webinar, 50% claim to use TDD. A Forrester survey shows only 3.4% adoption of TDD among IT professionals. However, TDD was not well-defined in this survey. It was listed alongside Scrum and XP when in fact TDD is a practice that can used within these methodologies. Likewise, results from other studies during this period must be analysed in the context of how TDD was defined or interpreted.

2014
Reported effects of TDD. Source: Mäkinen and Münch 2014, fig. 1.
Reported effects of TDD. Source: Mäkinen and Münch 2014, fig. 1.

Mäkinen and Münch analyze current literature and learn that TDD reduces defects, makes code more maintainable and improves external quality. However, it doesn't seem to improve productivity or internal code quality. Meanwhile, Martin Fowler, Kent Beck, and David Heinemeier Hansson engage in a series of discussions asking "Is TDD Dead?" They address TDD's limitations and how not to do TDD.

Sample Code

  • // Source: https://www.baeldung.com/java-test-driven-list
    // Accessed 2022-12-31
     
    // Class to be implemented (we start with a skeleton)
    public class CustomList<E> implements List<E> {
        private Object[] internal = {};
     
        @Override
        public boolean isEmpty() {
            return false;
        }
    }
     
    // Test is written first (test will fail)
    @Test
    public void givenEmptyList_whenIsEmpty_thenTrueIsReturned() {
        List<Object> list = new CustomList<>();
     
        assertTrue(list.isEmpty());
    }
     
    // Quick implementation to make the above test pass
    public class CustomList<E> implements List<E> {
        private Object[] internal = {};
     
        @Override
        public boolean isEmpty() {
            return true;
        }
    }
     
    // A new failing test
    @Test
    public void givenNonEmptyList_whenIsEmpty_thenFalseIsReturned() {
        List<Object> list = new CustomList<>();
        list.add(null);
     
        assertFalse(list.isEmpty());
    }
     
    // Implementation to make the above tests pass
    public class CustomList<E> implements List<E> {
        private Object[] internal = {};
     
        @Override
        public boolean add(E element) {
            internal = new Object[] { element };
            return false;
        }
     
        @Override
        public boolean isEmpty() {
            if (internal.length != 0) {
                return false;
            } else {
                return true;
            }
        }
    }
     
    // Refactor the above while checking that tests continue to pass
    public class CustomList<E> implements List<E> {
        private Object[] internal = {};
     
        @Override
        public boolean add(E element) {
            internal = new Object[] { element };
            return false;
        }
     
        @Override
        public boolean isEmpty() {
            return internal.length == 0;
        }
    }
     
    // The TDD process is then applied:
    // - for improving add() to handle more than one element in the list
    // - for implementing other methods such as size(), get()
     

References

  1. Aniche, M. 2019. "Test-Driven Development in Practice." Slides, CSE1110 - Software Quality and Testing, Delft University of Technology. Accessed 2022-12-31.
  2. Beck, K. 2002. "Test Driven Development: By Example." Addison-Wesley Professional. Accessed 2023-01-05.
  3. Buchan, J., L. Li, and S. G. MacDonell. 2011. "Causal Factors, Benefits and Challenges of Test-Driven Development: Practitioner Perceptions." 2011 18th Asia-Pacific Software Engineering Conference, pp. 405-413. doi: 10.1109/APSEC.2011.44. Accessed 2022-12-31.
  4. Bunardzic, A. 2019. "Best practices in test-driven development." Opensource.com, Red Hat, Inc., October 23. Accessed 2022-12-31.
  5. C2 Wiki. 2014. "Ten Years Of Test Driven Development." C2 Wiki, June 21. Accessed 2023-01-04.
  6. Canfora, G., A. Cimitile, F. Garcia, M. Piattini, and C. A. Visaggio. 2006. "Evaluating advantages of test driven development: a controlled experiment with professionals." ISESE '06: Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering, pp. 364-371, September. doi: 10.1145/1159733.1159788. Accessed 2022-12-31.
  7. Dang, A. T. 2020. "Test-Driven Development: A Very Short Introduction." DataDrivenInvestor, on Medium, April 29. Accessed 2023-01-05.
  8. Erdogmus, H., G. Melnik, and R. Jeffries. 2010. "Test-Driven Development." In: Laplante, P. A. (ed), Encyclopedia of Software Engineering, CRC Press, November. doi: 10.1081/E-ESE-120044180. Accessed 2023-01-04.
  9. Fowler, M. 2014. "Is TDD Dead?" Articles, May-June. Accessed 2022-12-31.
  10. Fox, C. 2019. "Test-Driven Development is Fundamentally Wrong." HackerNoon, November 3. Accessed 2022-12-31.
  11. Fucci, D., H. Erdogmus, B. Turhan, M. Oivo, and N. Juristo. 2017. "A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?" IEEE Transactions on Software Engineering, vol. 43, no. 7, pp. 597-614, July, doi: 10.1109/TSE.2016.2616877. Accessed 2022-12-31.
  12. George, B. and L. Williams. 2003. "An initial investigation of test driven development in industry." SAC '03: Proceedings of the 2003 ACM Symposium on Applied Computing, pp. 1135-1139, March. doi: 10.1145/952532.952753. Accessed 2022-12-31.
  13. Grenning, J. 2008. "Physics of Test Driven Development." Blog, Wingman Software, June 7. Accessed 2023-01-04.
  14. Hammond, S. and D. Umphress. 2012. "Test driven development: the state of the practice." ACM-SE '12: Proceedings of the 50th Annual Southeast Regional Conference, pp. 158-162, March. doi: 10.1145/2184512.2184550. Accessed 2022-12-31.
  15. Hansson, D. H. 2014a. "TDD is dead. Long live testing." Blog, April 23. Accessed 2022-12-31.
  16. Hansson, D. H. 2014b. "Test-induced design damage." Blog, April 29. Accessed 2022-12-31.
  17. Jain, N. 2022. "Microservices Integration Done Right Using Contract-Driven Development." InfoQ, December 29. Accessed 2023-01-04.
  18. Janzen, D. and H. Saiedian. 2005. "Test-driven development concepts, taxonomy, and future direction." Computer, IEEE Computer Society, vol. 38, no. 9, pp. 43-50, September. doi: 10.1109/MC.2005.314. Accessed 2022-12-31.
  19. Karac, I. and B. Turhan. 2018. "What Do We (Really) Know about Test-Driven Development?" IEEE Software, vol. 35, no. 4, pp. 81-85, July/August. doi: 10.1109/MS.2018.2801554. Accessed 2022-12-31.
  20. Koskela, L. 2007. "Test Driven: Practical TDD and Acceptance TDD for Java Developers." August. Manning Publications. Accessed 2022-12-31.
  21. Kralj, K. 2021. "11 Test Driven Development Best Practices – Tests That Think!" Blog, MethodPoet, August 31. Updated 2022-02-23. Accessed 2022-12-31.
  22. Larman, C. and V. R. Basili. 2003. "Iterative and Incremental Development: A Brief History." Computer, IEEE Computer Society, vol. 36, pp. 47-56, June. doi: 10.1109/MC.2003.1204375. Accessed 2022-12-31.
  23. Małaszkiewicz, A. 2020. "TDD - Basics: Test-Driven Development for beginners." Woman on Rails, February 5. Accessed 2022-12-31.
  24. Miller, J. D. 2022. "Real Life TDD Example." The Shade Tree Developer, October 4. Accessed 2022-12-31.
  25. Mäkinen, S., and J. Münch. 2014. "Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies." In: Winkler, D., Biffl, S., Bergsmann, J. (eds), Software Quality. Model-Based Approaches for Advanced Software and Systems Engineering. SWQD 2014. Lecture Notes in Business Information Processing, vol 166. Springer, Cham. doi: 10.1007/978-3-319-03602-1_10. Accessed 2022-12-31.
  26. Nam Thai, N. 2018. "How to TDD a List Implementation in Java." Baeldung, March 3. Updated 2019-05-07. Accessed 2022-12-31.
  27. North, D. 2006. "Introducing TDD." September 20. Accessed 2023-01-05.
  28. Park, S., and F. Maurer. 2010. "A Literature Review on Story Test Driven Development." In: Sillitti, A., Martin, A., Wang, X., Whitworth, E. (eds), Agile Processes in Software Engineering and Extreme Programming, XP 2010, Lecture Notes in Business Information Processing, vol 48. Springer, Berlin, Heidelberg. doi: 10.1007/978-3-642-13054-0_20. Accessed 2022-12-31.
  29. Peterson, K. 2012. "Why does Kent Beck refer to the "rediscovery" of test-driven development?" Quora, May 11. Updated 2022-02-14. Accessed 2023-01-06.
  30. Reppert, T. 2004. "Don't Just Break Software. Make Software." Better Software, pp. 18-23, July/August. Accessed 2023-01-05.
  31. Satya, D. 2020. "The Value at the Intersection of TDD, DDD, and BDD." Blog, mobileLIVE Inc., November 17. Updated 2022-11-18. Accessed 2022-12-31.
  32. Steinfeld, G. 2020. "5 steps of test-driven development." IBM Developer, IBM, February 6. Accessed 2022-12-31.
  33. Straub, R. 2022. "The Real Reasons for Doing Test-Driven Development." Blog, CodeCraftr, September 13. Updated 2022-09-14. Accessed 2022-12-31.
  34. Tacker, T. 2017. "Best Practices for Test Driven Development." Thesis, Governors State University. Accessed 2022-12-31.
  35. Wingman Software. 2023. "Test-Driven Development For C Training." Wingman Software. Accessed 2023-01-04.

Further Reading

  1. Erdogmus, H., G. Melnik, and R. Jeffries. 2010. "Test-Driven Development." In: Laplante, P. A. (ed), Encyclopedia of Software Engineering, CRC Press, November. doi: 10.1081/E-ESE-120044180. Accessed 2023-01-04.
  2. Fucci, D., H. Erdogmus, B. Turhan, M. Oivo, and N. Juristo. 2017. "A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?" IEEE Transactions on Software Engineering, vol. 43, no. 7, pp. 597-614, July, doi: 10.1109/TSE.2016.2616877. Accessed 2022-12-31.
  3. Wake, B. 2021. "TDD: Balancing Progress with Stability." Blog, XP123, February 17. Accessed 2023-01-04.
  4. Straub, R. 2022. "The Real Reasons for Doing Test-Driven Development." Blog, CodeCraftr, September 13. Updated 2022-09-14. Accessed 2022-12-31.
  5. Miller, J. D. 2022. "Real Life TDD Example." The Shade Tree Developer, October 4. Accessed 2022-12-31.

Article Stats

Author-wise Stats for Article Edits

Author
No. of Edits
No. of Chats
DevCoins
4
0
1487
2116
Words
0
Likes
52
Hits

Cite As

Devopedia. 2023. "Test-Driven Development." Version 4, January 6. Accessed 2023-01-06. https://devopedia.org/test-driven-development
Contributed by
1 author


Last updated on
2023-01-06 10:30:31
  • Unit Testing
  • Continuous Integration
  • Agile Software Development
  • Behaviour-Driven Development
  • Extreme Programming
  • Waterfall to Agile Transformation