Preface to Problem Frames

Software development problems

The central goal in a software development problem is to create the software for a computer system that will serve some useful purpose in the world. This book is about analysing and structuring problems of this kind.

These problems are found in many different contexts and forms. If you are doing software development you might be building a system to act as a repository and access mechanism for information about bank accounts and loans. You might be developing a telephone exchange to switch local calls. You might be creating a tool for writing and editing texts and diagrams; or a control device to maintain cruising speed in a car; or a compiling machine to transform Java programs into bytecode. Almost any part of the human and physical world can furnish the raw material and the context for a software development problem.

Because computers can serve so many purposes, and play the central role in solving so many different kinds of problem, the practice of software development is less specialised than the established engineering disciplines. For the individual developer, and the individual development project, there's a lot more variety. That's why you should usually start by describing and structuring your problem in a way that's rarely necessary in other engineering disciplines, where the diversity of problems to be solved is much smaller. The automobile engineer designing a sports car does not need to ask whether the car must be capable of carrying 15 people, travelling underwater, carrying a ten-ton load, or moving backwards at 100mph. The phrase 'sports car' specifies both the problem and its acceptable solutions closely enough to answer those questions and many others.

But as a software developer you are rarely solving an immediately recognisable and well understood problem. Usually, you must begin by asking: What kind of problem is this? What, exactly, is the problem about? What purpose is being served in the world? What behaviour and properties must the computer have to achieve that purpose? Often, software development seems to start at square one.

Problem frames

When you analyse a problem you see what kind of problem it is, and identify the concerns and difficulties you will have to deal with to solve it. The concerns in a Java compiler will be very different from the concerns in a car braking system. You have different things to think about. Different kinds of descriptions must be made, and fitted together into differently shaped arguments. Problem analysis takes you from the level of identifying the problem to the level of making the descriptions needed to solve it.

But most realistic problems are too big and complex to handle in just two levels like this. Another level is needed: structuring the problem as a collection of interacting subproblems. If your structuring is successful, the subproblems will be smaller and simpler than the problem you started with, and their interactions will be clear and understandable. Then you can analyse each subproblem separately, as a simple problem on its own, and make the descriptions it needs.

The central idea of this book is to use problem frames in problem analysis and structure. They help you by defining different simple problem classes. When you structure a larger, realistic problem, you choose the subproblems so that each one is a problem of the simple kind defined by some problem frame. Then, when you analyse the subproblem, you see what concerns it raises according to the problem frame that fits it. The frame shows you what you must do to solve it.

A problem frame defines the shape of a problem by capturing the characteristics and interconnections of the parts of the world it is concerned with, and the concerns and difficulties that are likely to arise. So problem frames help you to focus on the problem, instead of drifting into inventing solutions. They do this by emphasising the world outside the computer, the effects that are required there, and the relationships among things and events of the world by which your customer will ultimately judge whether those effects have been achieved.

Problem frames share much of the spirit of design patterns. Design patterns look inwards towards the computer and its software, while problem frames look outwards to the world where the problem is found. But they both identify and describe recurring situations. They provide a taxonomy within which each increment of experience and knowledge you acquire can be assigned to its proper place in a larger scheme, and can be shared and accessed more effectively. So just as in object-oriented design a familiarity with design patterns allows you to say 'we need an instance of the decorator pattern here', so in problem decomposition a familiarity with problem frames allows you to say 'this part of the problem is a workpieces problem'. Having identified a part of a problem with a recognised problem frame, you can draw on experience associated with the frame.

The problem frame idea was first published in book form in my book Software Requirements & Specifications, where it was sketched in outline as one of a small number of related topics. This book puts more flesh on that skeleton. A number of elementary and composite problem frames are discussed and illustrated, along with a number of flavours and variants, and some of the concerns they raise are examined. The use of problem frames in decomposing realistic problems into subproblems is also explored and illustrated.

A focused view

Some people think that this notion of software development problems derives from a perspective that is too sharply focused and too narrow. They point out that computer systems, and the process of software development itself, almost always exist in a complex and fluid social, political, ethical and economic environment. When you are discovering the requirements for a system you are likely to be engaged in a process of social negotiation among conflicting groups of stakeholders; when you make a decision about system functionality you may be implicitly favouring one group over another; when you enlarge the system scope you are often giving political power to one group at the expense of another; when you analyse the purposes to be served by the system you are exploring its economic and organisational consequences.

So, from this point of view, the major concerns in software development are social and political, organisational and economic. Thinking in terms of problems doesn't do justice to these concerns. It's true that these concerns are important in many developments, but nonetheless we are going to ignore them in this book. We are not going to discuss how to elicit requirements, how to make the business case, how to manage the project, facilitate meetings, or negotiate compromise. We will ignore these things, not because they are unimportant but simply because they are not the subject matter of this book. If you want to understand anything, you mustn't try to understand everything.

Instead, we are going to try to understand some more sharply focused ideas in problem structure and analysis in the context of software development. Our chief topics will be the material, observable effects that the system should bring about in the world, the computer behaviour that will achieve those effects, and the connection between them. In short, the topics that are often called functional requirements, software specifications, and the path by which you get from one to the other.

Focusing on problems

It is not easy to focus on problems in software development. One reason is that they have some precise and some imprecise aspects, all competing for your attention. Like Odysseus on his ship coming home from Troy, you are sailing between Scylla and Charybdis, and must try to steer a middle course.

If you are attracted by the arguments of the people who regard our view of problems as far too formal and narrow, you may be dragged off to the right, into the world of purely human problems - the imprecise world of sociology and ethnography, where nothing is ever completely certain or completely exact. That's an important world, but it neglects the software and its development.

However, if you find that problems are less exciting than their software solutions, you are more likely to veer off to the left, towards the much more precise world of programming - of variables and methods and object classes, where boolean values are always either True or False, and never anything in between. Progamming is also important, but it won't be useful if no one has analysed the problem and worked out what the programs must do.

In the problems discussed in this book, there are plenty of precise and plenty of imprecise ingredients. We will try to do justice to both, to steer the proper course for understanding and analysing problems.

Formality and informality

If you steer the proper course between Scylla and Charybdis, and succeed in focusing on the problem, your descriptions must sometimes be quite formal, and sometimes quite informal. Formality, of course, is in the eye of the beholder. Some software developers think they are being extremely formal when they draw a diagram according to some rule rather than according to their fancy of the moment. Some think that a development is hopelessly informal if it does not rely on rigorous mathematical proofs of correctness.

Software development problems are about the real world where the system must have its effect. So the formality or informality of much of what you must do in problem analysis is governed by the formality or informality of the parts of the world you are dealing with. It's good to be as precise and formal as the subject matter allows and the task demands: why be vague when you can be exact? But it's also important to recognise that much of what is in the physical world can't be described faithfully or even adequately in purely formal terms.

In Engineering and the Mind's Eye, a wonderful book about mechanical and structural engineering, Eugene Ferguson complains that many engineering disasters have happened because modern engineers have been taught to pay too much attention to calculation and formal analysis of structures and too little to the physical reality of the world of which those structures are a part. He writes:

'The real problem of engineering education is the implicit acceptance of the notion that high-status analytical courses are superior to those that encourage the student to develop an intuitive "feel" for the incalculable complexity of engineering practice in the real world.'

Perhaps as practising software developers we are not guilty of paying too much attention to 'high-status analytical courses'. But we do pay a great deal of attention to techniques that are essentially syntactic and notational. That leaves us - like the engineers whose education Ferguson is criticising - paying too little attention to the incalculable complexity of the real world. Our problems and requirements are in the world, not in the computer. We must focus on them directly, and describe them conscientiously.

Are there really problems?

Some people have another reason for not thinking of software development in terms of problems. They see every system and its environment in a symbiotic relationship, perpetually evolving and adjusting to each other. Installing and using a system - even proposing to install and use a system - brings about changes in the environment, which in turn affect the evolution of the system in its later versions. The 'problem' is being continually modified from the moment it is proposed. A problem description is, at best, a snapshot of a problem at one moment in time that will soon pass away. There is no such thing as a 'requirement'. And if there is, it's not useful.

This is often a sound argument for incremental deployment and evolutionary development. And it can also be a sound argument for a determined attempt to forecast the effects of the system, to calculate the changes it will bring about in the environment, and to take account of those changes in the system development. It's not enough to design the new bridge to handle the immediate and currently foreseen traffic demands; you must design it to handle the much more uncertain increase in demand that the bridge itself will call forth.

But it's never a sound argument for refusing to think in terms of problems and requirements. When a computer system - or an increment, or an evolutionary modification, to a system - is installed and put into operation, it will certainly have some particular behaviour, determined by its software. That particular behaviour will have effects and consequences in the problem world, which may be desirable or undesirable. To capture the requirement, and to structure and analyse the problem, is simply to do the best you can to decide what consequences are wanted and to create software that will bring them about. To say that different consequences, and different software, will be wanted next year doesn't justify getting it wrong this year.

The proper response to uncertainty and fluidity of requirements is not to throw up your hands and abandon the task. It is to work at a higher level of generality. One technique is to encode some of the functionality in explicit descriptions that the machine will interpret at execution time. Then the users can modify the system behaviour by modifying these descriptions. This approach, of increasing the level of generality, falls entirely within the discipline of problem analysis discussed in this book: it uses a variant frame that has a description domain. It is also falls, of course, entirely within a common practice of design patterns, where it is manifested in patterns with names such as reflection or metadata or type object.

What method?

The use of problem frames does not imply a particular development method. One reason is that different frames demand different methods and the use of different concepts and notations. The idea of a panacea - a universal cure - for the difficulties of software development is now rightly discredited. But sadly it persists. Surprisingly, universal methods, implicitly claiming to be applicable to every possible problem, are still being devised and sold, and seem to flourish. But if a method offers help with every possible problem, you mustn't expect it to help much with any particular problem.

Another reason for not advocating a particular development method is that most of the methods widely used today are strongly solution-oriented. You need solution-oriented methods to help you to produce solutions. But they give little help - and are often a positive hindrance - in structuring and analysing problems.

Problem frames, and the related ideas, are meant to be used as a front end to what you would do anyway; or to suggest how you might extend or modify your practice; or, perhaps, just to clarify it. If you do adopt and use these ideas, do it gradually in a piecemeal process, not in a sea-change. You may perhaps find that many of them are just giving names to ideas and intuitions that you already have. Naming them should make them more consciously accessible when you need them. And where they're new to you, they may shed light on some difficulties that you had felt but had never articulated.

Structure of the book

The key idea of the book is the decomposition of complex and realistic problems into structures of simple subproblems of kinds that fit recognised problem frames. So there's a mixture of larger and smaller problems. Some of the smaller problems are trivial, especially the problems used to illustrate the most basic elementary problem frames. Don't spurn trivial problems. They show the stripped-down essence of problem classes in their barest form: that makes it much easier for you to recognise them when they appear in fancy dress concealed by the trappings of a larger problem.

The problems are not presented in ascending order by size and complexity. That would be rational and tidy, but it would make the book frustrating to read. Instead, larger and smaller problems are interleaved, and different aspects of each problem are discussed wherever they seem relevant and provide a good illustration of the current topic.

Some topics, such as descriptive languages and notations, run through the whole book. Again, aspects of these topics are introduced where they arise.

This very informal structure is supplemented by two appendices. One summarises the descriptive languages and notations used; the second provides a glossary of the terminology. There is a consolidated list of all the bibliographic references, and an index.

Is this book for you?

I hope this book will be useful to teachers, students and practitioners of systems analysis, system specification, and software and requirements engineering, and to anyone else with an interest in concepts and intellectual tools for software development. It is not a course textbook. But it may prove useful as a supplementary text in some courses on software development.

Any questions?

Who or what are Scylla and Charybdis?

Scylla was a monster of Greek mythology who lived in a cave on the Straits of Messina, between Sicily and Italy. She dragged passing seafarers into her cave and devoured them - rather like the way that the energy needed for solving a problem is consumed by the technicalities of programming. Charybdis was a lethal whirlpool on the opposite side of the straits that drew seafarers down to the depths of the sea - rather like the way that a problem can be whirled round and round in the eddying currents of politics and sociology. Both Odysseus and the Argonauts sailed successfully between these two perils.

What are the two books you mentioned that discuss problem frames?

Michael Jackson, Software Requirements & Specifications: A Lexicon of Practice, Principles, and Prejudices, Addison-Wesley, 1995.

Ben Kovitz, Practical Software Requirements: a Manual of Content and Style, Manning, 1998.

What is the book by Ferguson about mechanical and structural engineering?

Eugene S Ferguson, Engineering and the Mind's Eye, MIT Press, 1992.

Where can I find other ideas for classifying and structuring problems?

There is not a lot in the practitioners' world. But there is some interesting research work in the area of system requirements that is very relevant and has some close parallels to the problem frame ideas.

The Requirements Apprentice provides a taxonomy of requirements and a library of requirement cliches. Cliches fall into three areas: environment, needs and system. A system may be, for example, an information system, a tracking system or a display system. The Requirements Apprentice itself is a tool that helps you to explore the cliche library and use the cliches for your particular problem. You can read about this work in:

Howard B Reubenstein and Richard C Waters, The Requirements Apprentice: Automated Assistance for Requirements Acquisition, IEEE Transactions on Software Engineering, Volume 17, Number 3, pages 226-240, March 1991.

Object system models and information system models capture abstractions of requirements engineering problems. The highest level of classification of object system models includes, for example, resource allocation, work item manipulation, and financial object exchange. This work is described in:

NAM Maiden and AG Sutcliffe, Analogical Retrieval in Reuse-Oriented Requirements Engineering, Software Engineering Journal, Volume 11 Number 5, pages 281-292, September 1996.

The KAOS method sees requirements in terms of goals. The goals of the system to be constructed are decomposed and elaborated, and assigned to agents. An agent may be a component consisting of hardware and software, or a human being, or another part of the physical problem environment. Goals are classified into information, safety, and other kinds. Conflicts between goals are identified and resolved; obstacles to satisfaction of a goal are detected and overcome. KAOS is described in:

A van Lamsweerde, R Darimont and Ph Massonet, Goal-Directed Elaboration of Requirements for a Meeting Scheduler: Problems and Lessons Learnt in Proceedings of RE95, the Second IEEE International Symposium on Requirements Engineering, pages 194-203, May 1995.

R Darimont and A van Lamsweerde, Formal Refinement Patterns for Goal-Driven Requirements Elaboration in Proceedings of FSE4, Fourth ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 179-190, October 1996.

Axel van Lamsweerde, and Emmanuel Letier, Integrating Obstacles in Goal-Directed Requirements Engineering in Proceedings of the 20th International Conference on Software Engineering, April 1998.