Technical Debt

Discussion

• Why do we call technical debt a "debt"?

  Cunningham explains the debt metaphor. Source: Cunningham 2009.

  The debt metaphor comes from finance. It's common for companies to borrow money to grow faster and capture the market quickly. However, this debt has to be eventually repaid with interest. The longer the company delays the repayment, the more it pays in terms of interest. Ward Cunningham applied this debt metaphor to technology, more specifically to software development.

  Due to various reasons, developers might not adopt the best approach to build their product. They might release the product even when they don't understand some parts of it. This is the debt they acquire in the hope that they will fix these issues in a future release. Bad code and poor understanding of that code lead to more time and effort to add new features. This is the interest paid on the debt. An hour saved today would require more than an hour tomorrow to fix the problem.

  Code refactoring leads to better design that's easy to understand and maintain. However, the longer they postpone this refactoring, the more difficult it becomes to refactor, just as interest payments continue to grow in financial debt.

• What's principal and interest with respect to technical debt?

  In finance, when we borrow money, the principal has to be paid along with interest. The longer we delay the payment of principal, the more interest we pay.

  In technical debt, suppose we keep working with poor code, design or documentation. Interest is the extra effort we pay to maintain the software. As the software gets more complex with each release, interest keeps going up.

  Instead, we could refactor the code incrementally and make it better for future maintenance. Principal is therefore the cost of refactoring. Paying the principal today reduces future interest payments (extra effort).

• How does technical debt arise in a project?

  

  Technical debt in a vicious cycle. Source: Sharma 2018.

  Technical debt could arise due to "business pressure, incorrect design decisions, postponing refactoring indefinitely, updating dependencies or simply the lack of experience of the developer". Bad practices can lead to technical debt: starting development without proper design, lack of testing, poor documentation, or poor collaboration.

  A software product may start with a clean design but incur technical debt as requirements change. Sometimes a software component could undergo a number of incremental changes made by multiple developers. These developers may not fully understand that component or its original purpose. Some call this "bit rot technical debt". Coding by copy-paste is a symptom.

  Sometimes a third-party software is upgraded in an incompatible manner, such as Magento 1.x to 2.0. All websites using Magento 1.x now have a technical debt since there's no simple upgrade path.

  Consider a web application. Though Ajax might be the right approach, it would take longer to develop. So developers use frames instead. This is a conscious decision that incurs technical debt. Developers must still implement a clean solution that can be migrated to Ajax easily in a future release.

• What are the different types of technical debt?

  

  The four quadrants of technical debt. Source: Fowler 2009.

  Back in 2009, Martin Fowler identified four types of technical debt:

  • Reckless & Deliberate: Developers are aware of good design practices but deliberately choose to ignore them and produce messy code. This leads to excessive interest payments and long time to payback the principal.
  • Reckless & Inadvertent: Developers are clueless about good design practices and produce messy code.
  • Prudent & Deliberate: Developers willing incur debt because they estimate that interest payments are small (rarely touched code). They do a cost-benefit analysis and accept the debt if it can be overcome.
  • Prudent & Inadvertent: This is often retrospective in nature. Developers release clean code but later realize that they could have done it differently. This is inadvertent debt. Developers are constantly learning and trying to improve their code. As systems evolve and requirements change, developers may find better designs.

  From the perspective of interest payment, we could classify technical debt as eventually repaid (refactoring on a daily basis, debt tends to zero), sustainable (refactoring regularly, debt is constant) or compound growth (adding new features but not refactoring, exponential growth of debt).

• Could you share some case studies of technical debt?

  

  Everyday indicators of technical debt. Source: Zazworka and Seaman 2012, slide 16.

  According to a study by Stripe, developers spend nearly half their time fighting with technical debt. They estimated that this amounts to $85 billion in annual cost.

  Another study of large software organizations showed that 25% of development time is the cost of technical debt. Only some used backlogs and static analyzers to manage technical debt. Very few had a systematic process for addressing technical debt.

  Twitter's platform was built on Ruby on Rails that was hard to optimize for search performance. Twitter solved this by eventually switching to a Java server, thus paying off its technical debt.

  A Canadian company successfully released a product locally. When they expanded to the rest of Canada, they had to cater to 20% of French-speaking Canadians. They quickly solved this by using a global flag and lots of if-else statements in code. Later they got an order from Japan. Had they made their software multilingual earlier, they could have easily updated their software for Japanese or any other language.

• What are the consequences of technical debt?

  If not managed, technical debt has long term effects on the product. It becomes difficult to add features or improve the product. More time is spent on paying off the interest. Any change implies higher costs. As product quality deteriorates, system outages or security breaches can lead to lost sales or fines. The cost of a quick release might be poor design, more bugs, volatile performance, and insufficient testing.

  Technical debt has a human cost too. Developers become unhappy. Adding new features becomes a pain. Even when new developers are hired, it takes time to explain what the code does or why it's so messy. New developers could start blaming older developers for the technical debt. Teamwork suffers.

  Crippling technical debt might force the team to postpone big changes. They might not adopt latest technologies or upgrade to the latest versions of third-party libraries. Developers might get stuck with outdated frameworks and have no opportunity to upgrade their skills. They may even leave for better opportunities elsewhere.

  Ultimately, these problems can be related to business risk, cost, sales, and employee retention. Technical debt gives developers a language to communicate clearly with business folks.

• How can I manage technical debt within my project?

  

  Technical debt if not managed grows over time. Source: Frederick 2018.

  Technical debt when addressed early requires only some code refactoring. If this is postponed, a more expensive rewrite may be needed.

  As in financial debt, it's better to pay off the principal to save on future interest payments. In other words, developers must continuously refactor code. If a new feature takes three days, a developer could take an extra day to refactor. This might simplify the current feature and make the code easier to work with for the future.

  Once code is shipped to customers, developers are reluctant to change it, for the fear of breaking system behaviour. The solution is to add more tests. In fact, technical debt implies a more disciplined approach towards refactoring and testing.

  Developers should follow good design practices: reuse rather than duplicate code; design highly cohesive and loosely coupled modules; name variable, classes and methods to reveal intention; document the code; organize code reviews.

  Unlike bugs, technical debt is often invisible. Tools such as Designite can help identify and track debt. Prioritize high-interest debts for refactoring. Motivate and reward developers for refactoring.

Milestones

1992

Ward Cunningham coins the term Technical Debt while working on WyCASH+, a financial software written in Smalltalk and employing object-oriented programming. He notes that it's important to revise and rewrite code towards better understanding. He also notes,

A little debt speeds development so long as it is paid back promptly with a rewrite. … The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt.

2001

The Agile Manifesto is signed. In the following years, as the Agile software movement gains adoption, it also brings more visibility to technical debt. Technical debt becomes an essential concept of software engineering. Agile is about faster development and responding quickly to customer needs. This focus on short-term deliveries should not compromise long-term goals. It's in this context that technical debt becomes relevant in Agile.

Sep
2009

Robert C. Martin (commonly called Uncle Bob) makes the point that technical debt is incurred due to real-world constraints. It's risky but could be beneficial. It requires discipline and clean coding. Messy code is not technical debt. Messy code is due to laziness and unprofessionalism.

Oct
2009

Martin Fowler expands on Uncle Bob's explanation of technical debt. In the process, he identifies the four quadrants of technical debt. He also notes that technical debt is a useful metaphor when communicating with non-technical people.

2016

Frequency of types of technical debt in embedded systems. Source: Ampatzoglou et al. 2016, fig. 1.

At the IEEE 8th International Workshop on Managing Technical Debt, researchers present their findings on technical debt in embedded systems. They find that test, architecture and code debts are more common. In embedded systems, runtime aspects are prioritized higher than design aspects. When the expected lifetime of a component is more than ten years, its maintainability is more seriously considered.

References

1. Ampatzoglou, Areti, Apostolos Ampatzoglou, Alexander Chatzigeorgiou, Paris Avgeriou, Pekka Abrahamsson, Antonio Martini, Uwe Zdun, and Kari Systa. 2016. "The Perception of Technical Debt in the Embedded Systems Domain: An Industrial Case Study." IEEE 8th International Workshop on Managing Technical Debt, IEEE Computer Society, pp. 9-16. Accessed 2020-07-05.
2. Bollinger, Isaiah. 2016. "The Consequences of Accumulating Technical Debt." The Trellis Blog, April 19. Accessed 2020-07-05.
3. Bulaty, Wojciech. 2019. "The Three Secret Types of Technical Debt." DZone, January 29. Accessed 2020-07-05.
4. Codabux, Zadia, and Byron Williams. 2013. "Managing technical debt: An industrial case study." Proceedings of 4th International Workshop on Managing Technical Debt, pp. 8-15. doi: 10.1109/MTD.2013.6608672. Accessed 2020-07-05.
5. Coding Sans. 2020. "Technical Debt: What You Need to Know, and How to Manage It." Blog, Coding Sans. Accessed 2020-07-05.
6. Cunningham, Ward. 1992. "The WyCash Portfolio Management System." Experience Report, OOPSLA'92, March 26. Accessed 2020-07-07.
7. Cunningham, Ward. 2009. "Debt Metaphor." YouTube, February 15. Accessed 2020-07-06.
8. Dietrich, Erik. 2016. "The Human Cost of Tech Debt." Blog, Infragistics, June 29. Accessed 2020-07-07.
9. Eland, Matt. 2019. "Technical Debt as Risk." Kill All Defects, December 24. Accessed 2020-07-05.
10. FirstMark. 2018. "There are 3 main types of technical debt. Here’s how to manage them." Hackernoon, January 25. Accessed 2020-07-05.
11. Fowler, Martin. 2009. "Technical Debt Quadrant." ThoughtWorks, October 14. Accessed 2020-07-07.
12. Fowler, Martin. 2019. "Technical Debt." ThoughtWorks, May 21. Accessed 2020-07-05.
13. Frederick, Erik. 2018. "The Financial Implications of Technical Debt." Toptal, March 29. Accessed 2020-07-05.
14. Jarosz, Paweł. 2019. "Technical debt: What it means and how to deal with it." Software Brothers, October 29. Accessed 2020-07-05.
15. Jensen, Orion. 2018. "5 Ways Technical Debt Impacts Your Business." Clear Launch, May 8. Updated 2018-05-18. Accessed 2020-07-05.
16. Kolakowski, Nick. 2018. "Technical Debt and Bad Code Have a Bigger Impact Than You Think." Dice, December 6. Accessed 2020-07-05.
17. Martin, Robert C. 2009. "A Mess is not a Technical Debt." Clean Coder, September 22. Accessed 2020-07-05.
18. Martini, Antonio, and Jan Bosch. 2015. "The Danger of Architectural Technical Debt: Contagious Debt and Vicious Circles." Proceedings of 12th Working IEEE/IFIP Conference on Software Architecture, May 5. Accessed 2020-07-05.
19. Martini, Antonio, Terese Besker, and Jan Bosch. 2018. "Technical Debt tracking: Current state of practice: A survey and multiple case study in 15 large organizations." Science of Computer Programming, Elsevier, vol. 163, pp. 42-61, October 1, Accessed 2020-07-05.
20. Ozkaya, Ipek. 2019. "Managing the Consequences of Technical Debt: 5 Stories from the Field." Blog, Software Engineering Institute, May 13. Accessed 2020-07-05.
21. Rawsthorne, Dan. 2015. "The 4 Types of Technical Debt." 3Back, LLC, May 21. Accessed 2020-07-05.
22. Sharma, Tushar. 2018. "Four Strategies for Managing Technical Debt." Design Smells, June 22. Accessed 2020-07-05.
23. Smith, Andrew. 2019. "What Is Tech Debt and How to Explain It to Non-Technical People?" DZone, December 18. Accessed 2020-07-05.
24. Wikipedia. 2020. "Technical debt." Wikipedia, May 27. Accessed 2020-07-05.
25. Wolff, Eberhard, and Sven Johann. 2013. "Managing Technical Debt." InfoQ, May 6. Accessed 2020-07-15.
26. Zazworka, Nico and Carolyn Seaman. 2012. "Identifying and Managing Technical Debt." SlideShare, June 12. Accessed 2020-07-05.

Further Reading

1. Watts, Stephen. 2017. "Technical Debt: A Beginner’s Guide." BMC Blogs, September 29. Accessed 2020-07-05.
2. Garvin, Rick. 2015. "The Impact of Technical Debt: A Guide for Business Managers." Ten Mile Square, October 13. Accessed 2020-07-05.
3. Built In. 2019. "How 19 Software Engineering Teams Deal With Technical Debt." Built In, October 15. Updated 2020-05-27. Accessed 2020-07-05.
4. Brown, Kyle. 2020. "Pay back technical debt." IBM Corporation. Accessed 2020-07-05.
5. Frederick, Erik. 2018. "The Financial Implications of Technical Debt." Toptal, March 29. Accessed 2020-07-05.
6. Abad, Zahra Shakeri Hossein, Reza Karimpour, Jason Ho, S.M. Didar-Al-Alam, Guenther Ruhe, Edward Tse, Kevin Barabash, and Ian Hargreaves. 2016. "Understanding the Impact of Technical Debt in Coding and Testing: An Exploratory Case Study." Proceedings of the 3rd International Workshop on Software Engineering Research and Industrial Practice, May 17. Accessed 2020-07-05.

Article Stats

Author-wise Stats for Article Edits

Author

No. of Edits

No. of Chats

DevCoins

arvindpdmn

3

4

1985

nagkumar

2

1

112

devbot5S

1

1

23

DevCoins due to articles, chats, their likes and article hits are included.

Cite As