Bad Requirements

Discussion

• What's a bad requirement?

  

  Attributes of a good requirement. Source: Lind et al. 2023, fig. 4.

  Requirements are bad if they're ambiguous, incomplete, inconsistent, incorrect, outdated or technically infeasible. Sometimes they're written in highly technical jargon that only a few understand. Some bad requirements describe system internals without relating them to visible interfaces or scenarios.

  Badly structured requirements are hard to trace. Multiple requirements written as a single descriptive sentence is also problematic. Unstructured requirements are hard to verify or validate. Requirements not essential or relevant to the system are equally bad. This practice is sometimes called "gold plating".

  Some bad requirements describe how to build the system, thus imposing inappropriate constraints. Others describe system operations. Instead, they should set the expectation from subsequent phases (design, coding, integration, testing).

  Requirements without suitable metadata is also a problem. Priority, status, estimated cost/schedule, and unique identification number are examples of metadata.

• What's the cost of bad requirements?

  Bad requirements lead to more work, rework, and lower morale. This in turn causes cost overruns, delayed deliveries, missing features, and poor quality products. Requirements that impose constraints or describe a solution prevent innovative thinking. When requirements are not traced, impact analysis becomes difficult. Lack of requirements tracing is a technical debt. It becomes difficult to evolve the system. Highly volatile requirements incur expensive rework.

  One study found that requirements and design phases contributed 64% of defect costs. Engineers spent 19 hours per day addressing requirements-related defects. Test engineers spent a lot of time clarifying requirements. As much of 10% of testing budget could have been saved with proper requirements engineering.

  It's been found that fixing a requirements defect during integration and testing is as high as 78 times more expensive. Other studies showed that defects discovered in the field cost 50-200 times more. Rework is 50% of total development cost. 50% of defects and 80% of rework effort are due to poor requirements. Poor requirements have caused 40% of accidents in safety-critical systems.

• Could you share examples of bad requirements?

  

  Examples showing how bad requirements can be improved. Source: Devopedia 2024.

  Here are some illustrative examples:

  • The system must have good usability: It's vague. It's unclear how to measure usability. Each user group or role interacts with the product differently.
  • Response time should be less than 5 seconds: Five seconds is an objective measure but there's no context. Product feature, scenario, system state or configuration are examples of context.
  • The system has to be bug-free: No system can guarantee this. Instead there should be well-defined processes for testing and change management.
  • Round-the-clock availability: Unrealistic and could be too costly in practice. Requirements should balance business, user, technical, and economic perspectives.
  • The system shall work just like the previous one, but on a new platform: Requirements can't easily be transferred across platforms or technologies. Newer technologies often offer better ways to implement features.
  • Reporting: Too brief. It's unknown what reports should include, the level of detail, who'll read them and towards what objectives.
  • Authenticate with user IDs and passwords: Imposes a particular solution rather than specifying a requirement.
• What words characterize bad requirements?

  

  Terms that spoil requirements and how to improve them. Source: Wiegers 2021, table 1.

  Even when requirements are correctly identified, they may not be written down correctly. Engineers are not necessarily good at writing. The choice of words matters. Words that're not precise enough should be avoided. One study grouped these and called them requirements smells. Here are some examples:

  • Unquantified: all, always, every, never.
  • Ambiguous: most, many, several, some, sometimes, usually, normally, ideally, optionally.
  • Vague Adjectives: easy, appropriate, user-friendly, fast, graceful, flexible, versatile, efficient, robust, adequate.
  • Vague Verbs: handle, improve, provide, reject, support, maximize, optimize.
  • Implied Multiple/Missing Requirements: and, another, etc., other, including, not limited to, such as, including, for example.
  • Pronouns: it, this.
  • Implied Uncertainty: should, can, could, if possible, may.

  Context is important. Consider the use of "and" in "DADS shall display the number of requests pending and requests processed." There are two requirements here. They must be separated. However, the following is a single requirement even when "and" is used: "DADS shall display the combined number of requests pending and requests processed."

• How can I avoid bad requirements?

  Requirements development should involve all stakeholders, including testers and operational experts. Discussions should promote diverse perspectives. Involve all user groups. The idea is to fix problems early on. Formal inspections and informal reviews should complement each other.

  Techniques for functional requirements may not suit non-functional requirements. Use different techniques to elicit and develop requirements: storyboards, interviews, prototypes, use case diagrams, class diagrams, data flow diagrams, state transition diagrams, etc. Consider both normal and abnormal scenarios. Use these techniques to identify and analyze requirements.

  Use checklists and templates. Use tools to identify ambiguous words and phrases. Use tools to structure, track and trace requirements. Store requirements in a single repository. Link models and diagrams to requirements. Store metadata. Tools with these capabilities include Borland CaliberRM, IBM/Rational's RequisitePro, and Telelogic's DOORS.

  Requirements and even project scope can change, particularly with long-running projects. Requirements engineering and project schedules must embrace and manage this change. Adopt iterative and incremental development methodologies.

Milestones

1990

The U.S. Government Accountability Office recognizes that poor requirements are causing projects to fail. Through the 1990s, studies identify poor requirements as the main cause of missed deadlines and cost overruns.

1993

Hooks proposes the following uses for words: "shall" for requirements; "will" for statements of fact; "should" for goals. In 1995, the MIL-STD 961D standard reserves the word "shall" to identify requirements. Words "will", "should" and "must" can be used in descriptions but not in requirements.

1996

Alvarez et al. apply syntactic, semantic and domain analysis to specifications. While they develop some tools to do this, the focus of their research is to develop metrics to identify poor requirements. In the late 1990s, a similar research at NASA results in the development of the Automated Requirements Measurement (ARM) tool. This tool analyses text structure, specification depth and readability.

2005

System utility as a function of required attribute. Source: Arnold and Martin 2005, fig. 6.

Arnold and Martin give the example "The system shall transmit data at least 100 kilobits per second." If the system delivers 99.8 kbps, it clearly fails the requirement. This binary approach to defining requirements unnecessarily constraints the solution space and forces designers to make the wrong trade-offs. Hooks observed this in 1993 and recommended the use of allowable tolerances.

2006

Kasser et al. propose a way to define metrics for requirements. They adapt metrics used to assess code quality and apply them to requirements. They also consider metrics or indices commonly used to assess readability of English text. They use the presence of poor words to compute what they call the Figure of Merit (FOM). These ideas are incorporated into tools named FRED, TIGER and TIGER Pro.

Sep
2009

The Requirement Analysis and Modeling Process (RAMP) project is started as a collaboration between academia and industry. It's objectives are to improve the writing of requirements in natural language, requirements analysis and context analysis. This research finds a gap between theory and practice. There are problems in formalization and consistency. Many requirements actually describe solutions.

2010

Hooks observes that people are still writing bad requirements despite the availability of many tools and techniques. Twenty years earlier he wrote an article titled Why Johnny Can't Write Requirements. Over these years he has learned that many things have to come together to produce good requirements: tools, techniques, training, good metrics, continuous validation, inspections, ask-not-assume approach.

2017

Use of smell detection in QA. Source: Femmer et al. 2017, fig. 5.

Femmer et al. coin the Requirements Smell. They identify nine smells and relate them to ISO 29148. They note that many automated tools currently available are inadequate in catching these smells. They develop a tool called Smella. Among it's techniques are POS tagging, morphological analysis, lemmatization, and dictionary search.

2022

A Deloitte white paper talks about the role of a Requirements Engineer. This role overlaps partially with the role of a software architect. The requirements engineer must be from a testing background. He/she must look at user scenarios and testability. He/she must address any gaps between developers and testers. In one project, Deloitte reduces requirements-related defects from 23% to 4%. This includes improvements to requirements, design, test data and test scripts.

References

1. Arnold, S. and J. N. Martin. 2005. "Requirements – The Good, the Bad and the Ugly." INCOSE International Symposium, Rochester, NY, pp. 1466-1479, July 10-14. doi: 10.1002/j.2334-5837.2005.tb00763.x. Accessed 2024-01-08.
2. Fanmuy, G., J. Llorens, and A. Fraga. 2012. "Requirements Verification in the Industry." In: Hammami, O., D. Krob, JL. Voirin (eds), Complex Systems Design & Management, Springer, Berlin, Heidelberg. doi: 10.1007/978-3-642-25203-7_10. Accessed 2024-01-08.
3. Femmer, H., D. M. Fernández, S. Wagner, and S. Eder. 2017. "Rapid quality assurance with Requirements Smells." J. of Systems and Software, Elsevier, vol. 123, pp. 190-213, January. Accessed 2024-01-08.
4. Firesmith, D. 2007. "Common Requirements Problems, Their Negative Consequences, and the Industry Best Practices to Help Solve Them." J. of Object Technology, vol. 6, no. 1, pp. 17-33, January-February. Accessed 2024-01-08.
5. Grove, L. 2007. "7 Words You Can Never Use in a Requirements Specification." 25th Annual Pacific Northwest Software Quality Conference, Portland, Oregan, October 9-10. Accessed 2024-01-08.
6. Hooks, I. 1990. "Why Johnny Can't Write Requirements." Space Programs and Technologies Conference, September 25-27. doi: 10.2514/6.1990-3561. Accessed 2024-01-08.
7. Hooks, I. 1993. "Writing Good Requirements." In: Proceedings of the Third International Symposium of the NCOSE - Volume 2. Accessed 2024-01-08.
8. Hooks, I. 2010. "Why Johnny Still Cannot Write Requirements." Slides, Compliance Automation, Presented at INCOSE. Accessed 2024-01-08.
9. Jacobs, D. 2007. "Requirements Engineering so Things Don't Get Ugly." 29th International Conference on Software Engineering (ICSE'07 Companion), Minneapolis, MN, USA, pp. 159-160. doi: 10.1109/ICSECOMPANION.2007.60. Accessed 2024-01-08.
10. Kasser, J. E. 2004. "The First Requirements Elucidator Demonstration (FRED) tool." Systems Engineering, J. of the INCOSE, vol. 7, no. 3, pp. 243-256. doi: 10.1002/sys.20005. Accessed 2024-01-09.
11. Kasser, J., W. Scott, X.-L. Tran, and S. Nesterov. 2006. "A Proposed Research Programme for Determining a Metric for a Good Requirement." Conf. on Systems Engineering Research. Accessed 2024-01-08.
12. Lind, E., J. Gonzalez-Huerta, and E. Alégroth. 2023. "Requirements Quality vs. Process and Stakeholders’ Well-Being: A Case of a Nordic Bank." In: Mendez, D., D. Winkler, J. Kross, S. Biffl, and J. Bergsmann (eds), Software Quality: Higher Software Quality through Zero Waste Development, Lecture Notes in Business Information Processing, vol. 472. Springer, Cham. doi: 10.1007/978-3-031-31488-9_2. Accessed 2024-01-08.
13. Rajkumar, A. 2022. "Your bad requirements are costing you money." White paper, Deloitte, January. Accessed 2024-01-08.
14. Reqtest. 2016. "10 of the Worst Requirements I have Ever Seen." Reqtest, March 3. Accessed 2024-01-08.
15. Westfall, L. 2010. "The Certified Software Quality Engineer Handbook." New Age International, New Delhi.
16. Wiegers, K. 2021. "When Bad Requirements Happen to Good People." Blog, Analyst's Corner, On Medium, October 23. Accessed 2024-01-08.

Further Reading

1. Hooks, I. 1993. "Writing Good Requirements." In: Proceedings of the Third International Symposium of the NCOSE - Volume 2. Accessed 2024-01-08.
2. Hooks, I. 1990. "Why Johnny Can't Write Requirements." Space Programs and Technologies Conference, September 25-27. doi: 10.2514/6.1990-3561. Accessed 2024-01-08.
3. Grove, L. 2007. "7 Words You Can Never Use in a Requirements Specification." 25th Annual Pacific Northwest Software Quality Conference, Portland, Oregan, October 9-10. Accessed 2024-01-08.
4. Firesmith, D. 2007. "Common Requirements Problems, Their Negative Consequences, and the Industry Best Practices to Help Solve Them." J. of Object Technology, vol. 6, no. 1, pp. 17-33, January-February. Accessed 2024-01-08.
5. Femmer, H., D. M. Fernández, S. Wagner, and S. Eder. 2017. "Rapid quality assurance with Requirements Smells." J. of Systems and Software, Elsevier, vol. 123, pp. 190-213, January. Accessed 2024-01-08.

Article Stats

Author-wise Stats for Article Edits

Author

No. of Edits

No. of Chats

DevCoins

arvindpdmn

2

0

1098

DevCoins due to articles, chats, their likes and article hits are included.

Cite As