Upfront Analysis: The Payoff
Camilo Fitzgerald
Sign A – Restaurant Closed For Business?

Sign A – Restaurant Closed For Business?

The question of how much upfront RE analysis to perform is one of considerable complexity that has always plagued software projects – proposed answers to have come from both ends of the spectrum. Research in requirements engineering typically suggests that more needs to be done [1], and that the cost ratio of fixing defects at the requirements level to later stages is between 5 and 10 for smaller projects and between 10 and 100 for larger one [2]. In practice, however, many a manager says, “I know that we should work out the requirements in detail, but we don’t have time. We have to get started on the programming because we have a short deadline to deliver the code!” [3].

The upfront analysis question is prevalent in many fields – When do we stop exploring a problem and begin taking action to solve it?

During my stay in Japan in the past six months I have seen many examples of signs designed to be understood by foreigners that illustrate the problem. Some of these have answered this question well by clearly communicating the desired message; some have done poorly; while others have simply left me baffled.

I encountered Sign A on the entrance to one of my favourite Tokyo restaurants. At first I was disappointed, thinking that I would have to satisfy my hunger somewhere else. I grew suspicious, however, when observing that the sign was permanently fixed to the door and on enquiring discovered that the establishment was in fact open. I later succeeded in translating the Japanese characters on the sign, revealing that its desired purpose was to state ‘that the door should be kept closed’. The restaurant manager had not realised the more obvious interpretation of the sign, which I suspect may have lost the custom of many a foreigner.

This sign is an all to common example of when a lack of upfront analysis can lead to unobserved negative consequences. Cutting corners on requirements analysis may result in a software product that performs its functions correctly, but in doing so has a strong negative impact on other business goals. Exploring negative impact with techniques like stakeholder analysis [4] can often result in an increase in product value that would otherwise be missed.

screen-capture-16

Signs B and C – Washroom Information

Signs B and C come from the walls of washrooms and tell contrasting stories. The first I found in a restaurant and was purchased from a bric-a-brac shop at the cost of a sticker. The shop owner did not know whether it would reduce clients accustomed to traditional Japanese toilets using their ‘western-style’ one in the same fashion. In contrast Sign C can be found in many metro stations and appears to have had a great deal spent on its sleek minimalist design and the mass manufacture of its metal cuttings. I first imagined it to signal a baby-changing station, but alas the sign appeared in many units where none was to be found. Its true meaning remains a riddle to me.

A comparison of these two signs exemplifies some of the factors to take into consideration when analysing the upfront analysis trade-off.

In the case of Sign B the costs of design and implementation, defects, and later fixes are low – buying and positioning a sticker, cleaning the toilet, and replacing the sticker if negative consequences result. We can assume, therefore, that the restaurant manager might be right in putting little consideration into how effective the sign would be.

With Sign C, meanwhile, these costs are high – design, distribution and installation of the metallic cut-outs across the metro network, many customers failing to understand the desired message, and replacing all such signs. A rough estimate of these values might well have lead to a strong case for further upfront analysis, such as extended assessment of the proposed sign’s clarity on test subjects.

The cost of upfront analysis and the probability of its success need also to be taken into consideration. The key factors can be defined as:

  • Cimp: cost of design and implementation.
  • Cdef: cost of product defects that could result from a lack of upfront analysis.
  • Cfix: cost of fixing defects at a later date.
  • P: probability that further upfront analysis will reduce later defects.
  • Cact: cost of further upfront analysis.

The first four points relate to the benefits of upfront analysis, and the net value of a unit of upfront analysis can be estimated to be its benefit minus its cost. An oversimplified equation for calculating the net value of a unit of further upfront analysis is as follows:

P*( Cdef + Cfix + Cimp) – Cact

Such an equation can provide an answer to the upfront analysis question. P will invariably decrease as more upfront analysis takes place, and upfront analysis should continue until the equation gives us a negative value.

In a complex software projects it is a difficult task to construct the upfront analysis equation and estimate values for its variables that accurately model the realities of specific domains. Costs, for example, can have multiple measurement criteria such as man-hours, budget, damage to a product’s reputation and loss of stakeholder satisfaction. Further, other factors may constrain the equation such as deadlines and the availability of manpower. The complexity of these models makes their construction and maintenance for software projects a non-trivial side-project in its own right.

Sign D – Upmarket Hair Saloon

Sign D – Upmarket Hair Saloon

If we were to have a pool of such equations that could be called on for specific domains this task would be made much easier. In a recent paper I provided such a cost model conducting extra upfront requirements analysis in online feature request management systems [5]. A set of ready-made models for other domains may well aid us in our efforts to answer the age-old upfront analysis question.

I leave you with Sign D from an upmarket hair saloon in central China that tells a slightly different story. It goes to show that even when the individual components, or in this case words, of a solution are carefully constructed the solution as a whole can behave very differently. Admittedly though, foreigners in the area were few and far between.

References:

[1] Daniela Damian, James Chisan – “An Empirical Study of the Complex Relationships between Requirements Engineering Processes and Other Processes that Lead to Payoffs in Productivity, Quality, and Risk Management” – IEEE Transactions on Software Engineering 2006
[2] S. McConnell – “Code Complete: A Practical Handbook of Software Construction, volume 2” – Microsoft Press, 2004
[3] Daniel Berry et al. – “To do or not to do: If the requirements engineering payoff is so good, why aren’t more companies doing it?” – International Conference on Requirements Engineering, 2005
[4] Ian Alexander - “A Taxonomy of Stakeholders: Human Roles in System Development”  – International Journal of Technology and Human Interaction, 2005
[5] Camilo Fitzgerald et. al – “Early Failure Prediction in Feature Request Management Systems” – International Conference on Requirements Engineering, 2011
Share Button

Leave a Reply