Meeting the Spec and other fables

Meeting the Spec and Other Software Fables

Michael P. Meier October, 1993

Philip Crosby, a noted quality guru, has put forth the following definition of quality:

A measure of Crosby's influence is the extent to which this definition has been adopted.

Software development has never seemed to fit readily into any of the quality paradigms. The best software has been developed by one or two people who were meeting their own specification. A brief anecdote will illustrate why this definition is not applicable to software.

I was once involved in a walkthrough of the design for a system being developed for a national defense customer. Also participating were two people who had direct experience in the activity being automated. I had related experience (hence my involvement as an end user stand-in).

During the course of the walkthrough, one of the people who had formerly been affiliated with the customer made a statement to the effect that the design wasn't giving him any "warm fuzzies." (I happened to agree -- the design, although technically competent and no doubt implementable, did not give evidence that the designer understood the context in which the system would be used.)

The individual conducting the walkthrough took apparent offense and replied in a somewhat smug tone that,

He no doubt felt confident that this was a correct position because the company we were all working for at the time had fairly recently embarked on a quality campaign based on the Crosby Quality College teachings.

The customer, adhering to Department of Defense policies and regulations, had taken great pains to develop a specification that was extremely detailed and (apparently) left nothing to chance. They had done their part.

The vendor/supplier had also done its part in that all involved were confident that they could show that the system to be delivered met the specification in every way. Nonetheless, the customer was about to receive a product that contained no "warm fuzzies." Is this the picture of quality that we want to paint?

Any product considered to be of high quality must give the customer warm fuzzies. Products lacking warm fuzzies will never be considered more than adequate at best.

Even a product as mundane as a hammer must have warm fuzzies built in. The professional carpenter will immediately reject a hammer that doesn't “feel right” (warm fuzzies) even though, to the untrained eye and hand it appears the same as the preferred hammer and both can be used to drive nails.

Software products depend to an even greater extent on warm fuzzies, factors which will never appear in any specification.

What are the warm fuzzies that determine the level of acceptance (quality?) in a software product? How can we characterize the intangibles that determine how the client will perceive our product offering? A short time spent in contemplation will probably yield one or more of the following:

-- does the product consistently (every time) produce the same result, does it maintain the integrity of the data within its influence

-- is the result produced by the product that expected by the client, does the output have the correct relationship to the input

-- when business conditions change (as they will), can the product be simply and quickly adapted to the new requirements

-- is the data controlled by the product directly accessible by other than the product itself, does the organization and management of the data support or permit uses other than those for which the product was designed

The qualities above have been recognized and defined within the software engineering discipline for several years (although the definitions above may differ in some respects from the "accepted" ones). It is recognized that these warm fuzzies are required in all systems in order for the system to be valued and long-lived.

As you look over the list you might notice that many of the qualities seem to be under the direct control of the analyst-programmer, either in whole or in part. This is an erroneous impression.

Each and every one of these qualities has a dimension that is entirely subjective, that is, its value is completely determined by the beholder (user, client). It is probably dangerous and certainly inappropriate for the programmer, no matter how familiar with the enterprise, to believe that he/she has the ideal solution.

Certainly there are aspects of some qualities about which the programmer is the best authority. The design of error-resistant algorithms is one such area. Error recovery might be another.

Still, the definition of the error condition itself is completely in the hands of the client. Words such as "normal use," "intended use," "correct," "easy" and "intuitive" imply subjectivity and do not lend themselves well to unambiguous specification.

In short, then, it is not reasonable to expect the client to be able to completely specify the warm fuzzies. Adding fuel to the fire, it is not reasonable to assume that one individual from the client organization is capable of evaluating warm fuzzies for the entire organization.

If one person can't speak for the organization in this matter, can two? How many will it take to arrive at an evaluation of warm fuzzies in the new system that will represent the opinions of all within the organization?

This brings us to another "quality" slogan that is appealing on the surface but causes no end of difficulties for software system developers.

By now, it should be apparent that zero defects is unattainable in any real sense. Those who subscribe to the "quality is free," "quality without tears" school [these happen to be titles of books by Philip Crosby] will be saying, "Of course zero defects is an unattainable goal. The slogan simply means that we should be satisfied with nothing less." This seems to be worthwhile in the way that a carrot held in front of a horse is worthwhile. It provides a destination for any movement towards it.

The effect of such a slogan, adopted wholeheartedly by the client (and by information industry management) is absolutely demoralizing on those responsible for producing systems. Each and every such person, after his/her first experience in delivering a system to a client, understands implicitly that "defects" are not something within their control.

More exactly, of all the kinds of defects possible, only a very small percentage is within the sphere of control of the system producer. The vast majority are defined by and attributed by someone else, usually within the client ("user") community. In Deming terms, the defects (variations) are part of the system.

With no power to control the "quality" of their product, the workers' morale can only go in one direction -- down. The Deming "red beads" experiment demonstrates this.¹ Deming's conclusion also applies.

The "defects" are caused by and inherent in the system. To eliminate these defects, the system itself must be changed.

The difficulties described here indicate that we need a different way of specifying and producing systems. In fact, largely unremarked by software developers, a paradigm shift has occurred. The client has come to expect that the first offering will be inadequate. He/ she/they plan from the outset that they will have to endure several cycles of specify, implement, test (use), specify before they will have the system that they want. This sounds something like what the software development world has come to call ("rapid") prototyping.

The client, however, has learned that, from the very beginning, it is foolish to demand less than the best that the programmer has to offer. This lesson is being reinforced every day around the world. The client agrees to a programmer suggestion of a prototyping tactic on a new system. He/She becomes excited about the development of screen layouts at an early date, only to have his/her hopes dashed upon finding that the screens are part of the specification rather than part of the system.

There is an impression of progress but the system doesn't get delivered any sooner and it's still only the first iteration of the cycle. This bad situation is made worse by the fact that “users” always confuse the presentation with the system -- understandable since, of the two sides of any system, inside and outside, the only one that they ever see is the outside.

The client now demands some critical subset of the total system's functionality (utility) at an early date. He does this knowing full well that it is the first (of probably many) iterations but determined to have something that can be used while the warm fuzzy qualities are being added in subsequent iterations.

Our current system of computer software production is a fairly stable one and prototyping has brought some improvements. Except for the fact that nobody is happy, we might be content to simply let things go on as they "always" have.

System developers are often made unhappy because they feel they are being rushed into delivering something that isn't ready. (“It's only a prototype!”)

The client is unhappy because the system looks done. (Remember that the outside of a system is the only side that the client ever sees.) By the time the system is delivered in its stable, “quality” form, it is usually obsolete due to technological advances or business changes.

Is there any answer to this problem? What is the problem? The following is a list of some of the distinct problems that have led us into the system of balanced unhappiness described above.

If you think this list reads like a prescription for trouble -- welcome aboard! Do you believe that this list is a good representation of the problems? Some of the items require a little serious reflection but basically each one results in or contributes to poor decisions. You will notice that I don't place all the responsibility on one party -- there's plenty to go around.

"Well," you say, "we (I) don't develop anything. We simply act as an intermediary between the client and the vendor who is actually doing the development." Does anyone believe that this fact changes in the slightest anything that has been said so far here?. If anything, the reverse is true. All of the problems discussed above are amplified by the introduction of a third party to develop the "system."

There may be very good reasons for going to a third party for system development. The reasons might involve support considerations, better development expertise, experience, time and fear of the unknown to name but a few.

Time is, as it turns out, one of the foremost reasons for going to a third party. "Vendor X has a product already and we will save time by using what already exists." So goes the rationale.

Prompt delivery may, indeed, be a warm, fuzzy attribute of quality. We should not, however, confuse instant gratification with promptness. If we get what we need and what we want, then it is appropriate to talk about promptness.

If we get something we don't need, don't want and can't use, then a discussion of promptness may demonstrate that the participants are out of touch with reality.

All of this goes to show that software quality (or more specifically, system quality) is an extremely complex issue and one that will require much careful thought and discussion.

One thing is clear -- warm fuzzies will always be essential components of quality. Given that fact, a prototyping or phased approach to system development seems to be a good choice.

Reuse of system components is essential if we are to deliver warm fuzzies promptly. We can no longer afford to have cycle times measured in months and years. Times measured in these units will guarantee the delivery of obsolete systems.

We must be able to deliver in hours or days. The only way to do this is by stocking our shelves with components that can be combined and recombined into new business functionality by nonprogrammers.

Of course, building such components will take exceptional programmers as well as analysts. It will involve some additional cost as the shelves are initially stocked. Someone must plan the inventory. Someone must make the difficult decisions about what is reusable and what isn't, which parts will be stocked and which won't.

Each new system project must have a goal to reuse whenever possible and to produce new reusable inventory where it is not. This burden must be shared across all system projects and should be managed by non-project resources answerable only to the highest level of the production organization.

The need for reusability has been recognized for some time. It has not been recognized to the extent that a client organization has been willing to underwrite it and dedicate the resources to make it happen. Should we wait for the client to insist on this? Don't we benefit at least as much as the client?

We (the information industry) have, for too long, been telling ourselves that we are powerless to affect our own position. We take the easy way out every time by saying, "That's what the user wanted."

Friends, the "user" wants what is best for business. The "user" is willing to pay a price to be able to do business more effectively, more competitively. We, on the other hand, are unable or afraid to incur any cost at all (except for new technology). We can't/won't do anything the "user" won't pay for.

We can't even develop a vision for ourselves because we can't tear our attention away from the keyboard (or mouse or track ball or light pen) long enough. Our "strategic plans" read like a technology catalog. Nowhere in them is there evidence of a vision -- something to give shape to the technology.

In the absence of vision, we are forced to examine every new technology that comes on the scene. "Can we use this?" "How?" "For what?"

Without a vision, we must be content to do things as we always have. Doing things as we always have means

Take a long, hard look at the list above. Why should we be considered a partner in the client's business? Would you want a partner who operated like this? Can you deny that this is the profile of the information/software industry.?

The first question to answer is "Do we want to change?" The changes required barely affect the programming side of things at all. What we must change is our view of the world in which programming and technology are the hammer and the whole world looks like a big collection of nails.

CASE vendors will not be able to help very much because their products are designed to automate the shift from one level of abstraction to another, less abstract, level.² The end result of the use of these tools is a software program. Unfortunately for us these tools are of marginal utility for describing the real world of our client(s). This is true because

In order to gain the flexibility we need, we must work in units bigger and more complex than lines of code or "statements." We must be free to deal in "objects" whose level of abstraction matches that of the client world.

Spreadsheets have become an indispensible business tool because they enable someone with virtually no knowledge of computers or programming to make use of the computer's speed so solve business problems. What we need is something like a spreadsheet but which is capable of more than numerical calculations.

Object Oriented has some promise along these lines but only if the "objects" are comprehensible and usable by non-IS persons. To the extent that OO environments require knowledge of programming, they are a failure. [Note that OO environments are developed by programmers so the programming knowledge required to use them may seem negligible to the developers but daunting to others.]

The future of OO is inseparable from the future of reusable (software) system components. If we can't find a way to stock our component inventory -- with or without the active (financial) participation of the client -- we will have failed as an industry.

In order to make reuse of software system components practical, it is necessary to thoroughly understand the system that is the client enterprise. Part of this understanding is the recognition of those portions of the enterprise that are unique to the particular client and those which are common to all "clients" engaged in the same business activity.

The careful reader will note at this point that such an understanding will require a significant analytical effort whose product will be -- what?

Here is the precise point at which the client gets to feeling for his wallet. To the client, this sounds like a scam. He is asked to pay for a large scale analysis of his enterprise including a comparison of his enterprise to those of his competitors.

What will he receive for the expenditure of his precious resources? He will get some diagrams (models) and some associated documentation. Where is the payoff? How does this affect the quarterly bottom line? How does it affect the fiscal year bottom line?

It is the exceptional client who is willing to take this leap of faith. Even more rare is the client who will support the analysis through to completion. This is true for two major reasons:

The difficulties are compounded by the apparent fact that software systems producers and their clients have difficulty in understanding and communicating their motives to one another. In consequence, all motives must be reduced to the lowest common denominator -- money.

It is very unfortunate that warm fuzzies do not have price tags. It is impossible to hang a price tag on, "I can't describe it, but I'll know it when I see it." In actual practice, the client doesn't "know it when he sees it." He/she will, after some period of time, "know it when he doesn't see it."

System quality, warm fuzzies and reusable components are undeniably related. Their relationship defies analysis -- at least by any means currently at our disposal. It may be that system quality has more in common with the non-deterministic "real-world" problems faced by our clients.

How, then, are these problems to be solved? Well, we could wait for a silver bullet to kill the monster. Mr. Brooks ³ has assured us, however, that there is no silver bullet. I believe that we have been waiting for someone to give us a magic carpet. We don't really expect to kill monsters -- we only want to avoid them.

We would all be happy with a magic carpet to help us bypass the problems and take us straight to our destination. So far we have hopped onto a lot of CASE throw rugs, bought into a bunch of methodolgy magic and waited. We have eagerly anticipated our arrival at the point where extremely short cycle time, end user satisfaction and low cost systems come together.

Unfortunately for all, problems aren't solved that way in the real world. In our world, problems are solved by:

We must make it possible for our clients to assemble their own systems. We must provide them with system Leggo® blocks that can be combined and recombined without programming.

In doing these things (or planning to do them) we should bear in mind that much of the complexity of today's business systems is due to the mindless accretion of undocumented system to undocumented system. If this is all done in the context of business reengineering, we can avoid mindboggling complexity in components and everyone will benefit.

How can we make this happen? What should we do first? What should we avoid at all costs? Who among us has no role to play? What additional roles must be filled? What is your part?

1 A good description of this experiment is found in The Deming Management Method by Mary Walton

2 For additional background on this problem, please request “The Legacy of CASE” (M.Meier, Aug. 1993)

3 Brooks, Frederick, No Silver Bullet, IEEE Computer, April 1987

4 Thanks to Dr. W. Edwards Deming

5 Not the appearance, but the functionality -- a graphical interface would be superb -- the more it behaves like Leggo blocks, the better

Michael P. Meier