Discovering the design problem

Discovering the design problem Peter Lloyd and Peter Scott, Department of Psychology, University of Sheffield, Western Bank, Sheffield S10 2UR, UK

The design disciplines of architecture, engineering, and computer science have provided three distinct models of the design process. We hypothesize that these models merely indicate three approaches to design, and that fragments of all three models will be found in any one discipline. To discuss this issue we present a discipline-independent cognitive framework which we then apply to a protocol analysis study of five engineering designers. Our results indicate that the designer's experience plays a key role in determining the design process.

Keywords: design models, cognition, protocol analysis, engineering, design

1 s~on, H A 'The structure of ill structured problems' Artificial in- telligance Vol 4 (1973) 181-201 2 Rlttel, H W J and Webber, M M 'Planning problems are wicked problems' Po//cy Sc/en- ces Vol 4 (1973) 155-169 3 Cross, N Engineering Design Methods Wiley, Chichester, UK. (1989) 4 Jones, J e 'A method of sys- tematic design' in Deve/opments in Design Methodology N. Cross (ed) Wiley, Chichester, UK (1963) pp 9-31 5 Jones, J C Design Methods; Seeds of Human Futures Wiley, Chichester, UK (1980) 6 I~hl, G and Betlz, W En- gineering Design The Design Council, London (1984) 7 P, adcllffe, D F and Lee, Y L 'Design methods used by under- graduate engineering students' Design Stud/es Vol 10 (1989) 199-207

I n attempting to characterize the nature of the design process theorists

tend to focus on what makes design problems and design thinking qualitatively different from other forms of problem solving and

thinking. Simon 1 for example has referred to design problems as ill

structured (as opposed to well structured problems) and suggests that ill

structured problems can be thought of in the sense of a number of

sequential well structured problems. Rittel and Webber 2 refer to design

problems as 'wicked', that is a design problem and its solution are linked

in such a way that in order to think about the problem the designer has to

commit themselves to some sort of solution.

More recent theoretical and empirical models of design have originated

from three general areas; engineering, architecture, and computer prog-

raming. Each discipline concentrates on different aspects of the design

process. Engineering provides models based on the analysis of problems preceding the synthesis of solutions 3-7, design is described as a number of

phases; each phase being completed before the next can begin.

Architectural models are based around the idea that solution concepts precede problem analysis; that is, the designer needs to generate a solution

0142-694X/94/020125-16 (~ 1994 Butterworth-Heinemann Ltd 125

8 Darke, J 'The primary generator and the design process' De- sign Studies Vol 1 No 1 (1979) 36-44 9 HllUer, B Muegrove, J and O'Sullivan P 'Knowledge and design' in Environmental Design: Research and Practice W.J. Mitchell (Ed) University of Califor- nia, Los Angeles (1972) 10 March, LJ 'The logic of design' in The Architecture of Form L.J. March (Ed) CUP, Cam- bridge, UK (1976) 11 Ade~on, e a.d Solway E 'The role of domain experience in software design' IEEE Transac- tions on Software Engineering Vol 11 (1985) 1351-1360 12 Dev~, s P 'Characterising the program design activity: neither strictly top-down nor glo- bstly opportunistic' Behaviour and Information Technology Vol 10 (1991) 173-190 13 Gulndon, R 'Designing the design process: exploiting opportunistic thoughts' Human Compu- ter Interac~on Vol 5 (1990) 305- 344 14Jeffrlee, R, Turner, AA, Polaon, P G and Atwood, M E 'The processes involved in designing software' in Cognitive Skills and their Acquisition J.R. Anderson (Ed) Lawrence Erlbaurn, Hillsdale, NJ (1981) 1 8 VkNmr, W 'More or less fol- lowing a plan during design: opportunistic deviations in specification' International Journal of Man-Machine Studies Vol 33 (1990) 247-278 16 Roozonburg, N F M and Cross, N G 'Models of the design process: integrating acroes the disciplines' Design Studies Vol 12 No 4 (1991) 215-220 17 Go~, v and Plrolll, P 'Moti- vating the notion of generic design within information proces- sing theory: the design problem space' AI Magazine (spring 1989) 18-36 18 Seh6n, D A The Reflective Practitioner Temple Smith, Lon- don (1983) 19 Sch6n, D A Educating the Reflective Practioner Jossey- Bass Inc, San Francisco (1987) 20 Lawson, B R How Desig- ners Think ButtenNorths, London (1980)

to begin to think about the problem s-l°. Computer programing models

describe designers negotiating the structure of design problems; either in opportunistic ways, regular ways, or a combination of the two 11-15.

Roozenburg and Cross 16 have referred to engineering models of the

design process as analysis-synthesis models of design, and architectural

models as conjecture-analysis models.

As these distinct models have originated in different disciplines one might

assume that, in designing, an architect behaves qualitatively differently

from an engineer or computer programer. However, other research

suggests that we can identify common behaviour in designers regardless of discipline 17-19. What seems likely is that models derived from a particular

discipline capture the primary nature of design problems within that

discipline, but not designer behaviour; which may fuse a combination of

the approaches listed as the design process proceeds.

This provides us with a division in the design situation: the design problem

and designer behaviour. The design problem contains a passive agenda in

terms of its organization and representation. The designer provides an

active agenda dependent on their experience and goals. The two agendas

are woven together in the design process: the designer projects their

unique experience on to the design problem to form a unique design

solution. We hypothesize that we can describe the way in which this

happens (the design process) in terms of the three general design models

introduced earlier. We have carried out an empirical study of five

engineering designers to illustrate this.

I Developing a framework for the analysis of design Our observational method is based on protocol analysis; in our study we

have instructed subjects to 'think aloud' when solving design tasks. From

a verbalized report of mental activity during the design process transcripts

are made that are then segmented into a sequence of utterances. Each

utterance is then coded. By analysing the order of coded utterances

statistical indicators of the individual's design process can be arrived at.

These indicators can then be interpreted, with the three models of the design process in mind, within our general framework.

Our analytical framework is an operationalization of two design theories: Lawson 2° and March 1°. Lawson presented a constraint based model of

design that allows mapping of the design process using three dimensions: generator, the person or body proposing the constraint; domain, to what

part of the design the constraint applies; and function, the purpose of the constraint. The cognitive aspects of Lawson's model were extended in an

126 Design Studies Vol 15 No 2 April 1994

empirical study of student architects 21. This cognitive extension had much

in common with the work of March 1° whose production-deduction-

introduction model stated that design progresses by cycles of these three

types of human reasoning.

Our framework is a fusion of these schemes consisting of three basic

utterance categories: generative utterances, deductive utterances and

evaluative utterances.

Generative utterances

Generative utterances bring something new to the design situation. This

can be in the form of a partial solution, or the bringing together of several

partial solutions. Generative utterances involve creating something to

reason about and advancing the solution. They can occur at various levels

of commitment; solutions and partial solutions can also be transient.

Examples

'I ' ll put the washing machine there'

'Put the sink in front of the window'

'Have a long work surface'

'Make that a general work surface'

Deductive utterances

Deductive utterances understand the specific needs of the problem, in

some way making the problem clearer. This can vary from simply

clarifying a point, to identifying a need not explicit in the specification.

Deductive utterances involve perceiving and representing the problem.

Examples

'So how big is this room?'

'What sort of light will this window produce?'

'It 's pretty insulated so heat loss shouldn't be a problem'

'If this is an outside wall it'll be hard brick to drill into'

21 Agll~nl, F A 'Cognitive aspects of architectural design problem solving' Ph.D. Thesis Examples Sheffield University, Sheffield, UK 09e0) ' I like that'

Evaluative utterances

Evaluative utterances are general comments about the discipline, design-

ing, or the situation. They occur both as utterances of intent and strategy,

and utterances of subjective reflection. In many ways evaluative utter-

ances are 'meta' comments organizing, planning, and reflecting on the design situation according to experience and preference.

Discovering the design problem 127

TII start by listing all the things I think I need'

'That looks kind of funny'

'I'11 do this half first then the other one'

2 Empirical study

Domain The study was carried out at a company specializing in the design of

electrical motor control systems for a wide range of industrial applica-

tions. For example paper-making, though a mechanical process of mould-

ing, forming, pressing, drying and winding, is controlled electrically by a

motor system that regulates the speed and sequence of operation of the

mechanical equipment. Wire-making again a mechanical process of

unwinding, stranding, twisting, extruding, insulating and rewinding is

controlled electrically by a motor system.

Designers working at the company complete their designs alone. Designs

take between two hours and twelve months to complete depending on the

complexity and novelty of the problem. Every motor control problem

involves specifying and configuring a wide range of electrical equipment;

relays, motor controllers, analogue electronics units etc. are combined to

achieve the function and performance required in the customer specifica-

tion.

Task The task involved designing a motor control system for a paper embossing

machine (see Figure 1). Embossing is a process that puts patterned

indentations onto paper producing products such as wallpaper and toilet

roll.

The specification consisted of general features of the factory for which the

embossing machine was intended, project notes about the mechanical

features of the machine (containing a technical drawing of the machine,

Figure 1), and a description of the intended machine operation. A price for the control system was also given. The design problem was a typical

task that any one of the designers could have received from their company and would usually take around 15 hours to complete to a detailed,

manufacturable design. The experiment took an average of 40 minutes to

complete.

In Figure 1 the motors drive the gearboxes which drive the rolls. Plain paper is unwound from the left and kept taut by a tension device. The


Figure 1 Embossing machine

F.mbosmg mils

l 'q~r m be U

1~lyir~ ~ ' r ~ c ~

oi12

)

paper then passes through the embossing rolls and is wound up using a

'flying splice' mechanism.

The flying splice mechanism allows the machine to continue running when

changing the winding roll from 1 to roll 2; when roll 1 is full, the large

wheel is rotated so that roll 2 is brought into contact with the paper. The

paper is then cut from roll 1 and spliced on to empty roll 2 which then

continues to wind up the embossed paper. A solution to this problem

would contain equipment to control the four motors (labelled M in Figure

1) according to the speed, weight, and size of the machinery in Figure 1.

Subjects

Five male designers were used in the study, varying in domain experience

from three to ten years. Of the five designers, two had experience of the

task administered (see Table 1).

Table 1 Engineering designer experience

General experience Experience of Age Education /years problem?

Designer 1 27 University 3 No Designer 2 28 University 3 Yes Designer 3 30 University 5 No Designer 4 34 Apprenticeship 8 No Designer 5 36 Apprenticeship 11 Yes


Procedure The design sessions took place at the desk of the designer and were

recorded on audio tape. Designers were instructed to verbalize their

thoughts at all times during the design session and produce design

schemes where possible. This presented no problems for any of them.

The conditions of the experiment were natural to the designer; in all

instances designers used pencil and paper although it had been stated that

they were free to use their CAD system. Designers were told that they

were free to refer to any previous projects, books and other designers.

The experimenter had direct experience of the design task and acted as a

company sales engineer resolving any points in the specification. This is

normal procedure for the designer.

Treatment of protocols Protocols were segmented by the experimenter into utterances that could

be distinguished as one of the three coding categories outlined earlier.

Sometimes this was easy, for example when paralinguistic activity could

be used to distinguish the beginnings and ends of utterances, but

sometimes chunking is more problematic, for example when semantics

dictate segmentation (requiring empathy with the designer). Often, in

coding the protocols, ambiguities in utterance type were only resolved by

the experimenter recalling the actual design session, the progression of

the solution, and the general behaviour of the subject. It is impossible to

provide this data for independent judges, however, coding agreements of

70% with judge 1 and 60% with judge 2 were obtained.

3 Results Distribution of coded utterances

To obtain an overall picture of the composition of the protocols Figure 2 shows graphs of the total number of the three utterance types in each

protocol. The designer number axis relates to the designer numbers given

in Table 1. Figures 2(a) and 2(b) show two general trends. Figure 2(a)

reveals that an increase in design experience means an increase in

generative utterances, Figure 2(b) for deductive utterances, indicates the opposite effect. Figure 2(c) displays no general trends for evaluative utterances.

Statistical analysis of the graphs in Figure 2 would not be meaningful due

to the difference in the length of protocols in terms of the number of

utterances. Designers were told to complete their designs to their satisfaction, the protocols thus relate a complete design process. To cut off the end of protocols, so that each protocol contained equal numbers of


Figure 2 Distribution of

coded utterances. (a) genera-

rive utterances; (b) deductive

utterances; (c) evaluative

utterances.

30.

w ,

lO,

Deduction Evaluation

SO 30

, 0 ~ o ': 1 2 $ 4 $ 1 2 3 4 S 1 2 3 4

Dea~ Designer Designer

a b c

utterances, seemed to miss the point of one of the fundamental issues of

design; the end is never well defined and only exists in the mind of the designer. Figure 2 illustrates underlying trends.

3.1 Reasoning mode by designer Figure 3 shows graphs of the cumulative total of each reasoning mode plotted against time (represented by protocol utterance number) for each designer. The graphs illustrate the interaction between different modes of reasoning. Three general patterns can be observed in the graphs. Long

periods of horizontal lines represent inactivity in a mode of reasoning, conversely a continuously rising line indicates undivided activity in a

reasoning mode. A stepped pattern in a line indicates short periods of activity followed by short periods of inactivity in a particular reasoning mode.

The graph of designer l 's design process shows that the dominant mode of reasoning is deduction. At the start of the protocol the deductive gradient is pronounced, and only begins to drop at the end of the protcol.

Generative reasoning does not start for 15 utterances and is much less in

evidence through the protocol, although at the end generation begins to increase (perhaps as one would expect).

Designer 2's initial dominant phase is one of deduction but this drops off (as with designer 1) and gives way to evaluation. Again generative reasoning occurs far less than the other two modes.

In the graph of designer 3's design process evaluation has become the dominant mode of reasoning. As with designer 1 and designer 2 deduction starts strongly and tails-off while for generation the reverse happens. Overall generation again, occurs the least often.


Figure 3 Reasoning mode by

designer: a) designer 1; b)

designer 2; c) designer 3; d)

designer 4; e) designer 5

Total

a

Total

b

Total

Total

Total

D e d u c t i ~ / revaluation

~ , , Generation

30 60 90 Utterance no

40'

30"

20"

10"

0 0

4O

30 Evaluatio~

20' ~ Deduction Deductio.~

10 ~ Generatio~n 0 , - , - , , . ,

0 15 30 45 60 75 Utterance no

40

30" Evaluati~~ 20" / j / ~ D e ~

1 O" Generation

0 . z / - . • 0 30 60 90

Utterance no

40

30

20

10

0

Evaluatio~

/ j J _ Gene~ration

~ Deduction • . , . _ , . , - ,

0 15 30 45 60 75 Utterance no

30' ~Evaluation

20 Generatic~r~/ 10' f D e d u c t i o n

o '-~O-'-~O-'-~O" Utterance no


The graphs of designer 4 and designer 5 show much more coherence than

the first three. Gradients of all reasoning modes are similar and vary much

less frequently. Initially in the graph of designer 4's design process all three reasoning modes are employed. However, about half way through the protocol, deductive reasoning drops-off leaving two dominant modes of evaluation and generation. For the first time in all the protocols

deduction occurs less than generation.

In designer 5's design process generation is the dominant mode of reasoning and overall, the reasoning modes are close in terms of cumulative totals. Even towards the end of the protocol we see no drop in

any one reasoning mode.

In summary the graphs of Figure 3 show evidence of trends that, at least

on the surface, seem to depend on experience. As designer experience increases (going from designer 1 to designer 5) generation becomes the dominant mode of reasoning and deduction becomes less important. The graphs of designers 4 and 5 show a coherence lacking in the first three

designers.

3.2 Design phases Figures 2 and 3 provide a qualitative analysis of the design process, they

highlight general trends but do not indicate fragments of the three models discussed earlier. All three models suggest that the design process consists of a number of phases. The conjecture-analysis model predicts generative phases before deductive phases. The analysis-synthesis model predicts

deductive phases followed by generative phases and finally evaluative phases. Structural models of the design process, stemming from computer

science, suggest that design problems are composed of units that can be solved independently. If these units are solved in regular ways we would expect to see phases of 'macro' design processes, but if they are solved in opportunistic ways we would expect to find irregular patterns in reasoning

modes. To identify fragments of these design models in our protocols we need to divide them into phases and then to interpret these phases. We thus need a means of distinguishing and characterizing phases.

Protocols were divided into phases of equal length. The minimum possible length of a phase was found to be 15 utterances due to the nature of the

statistical test (the chi-square test) used to determine phase type, (utterances were assumed independent). Protocols were thus divided up into segments of 15 utterances, analysed to obtain an idea of the phase type, and recompiled into a succinct description of the original protocol. Instances at the end of protocols where phases of less than 15 utterances


Table 2 Abstracted protocols

Phase

Reasoning mode

G D E Chi-square (p value) Notes

Designer 1 1 2 3 4 5

Designer 2 1 2 3 4




0 5 10 10 2 11 2 10.8 3 9 3 4.8 7 6 2 2.8 3 4 8 2.8 3 9 3 4.8 5 2 8 3.6 0 9 6 8.4 0 12 3 15.6 0 6 9 8.4 5 4 6 0.4 2 4 9 5.2 2 11 2 10.8 9 1 5 6.4 4 5 6 0.4 2 7 6 2.8 9 1 5 6.4 6 1 8 5.2 4 1 10 8.4 4 6 5 0.4 3 7 5 1.6 6 2 7 2.8 8 1 6 5.2 4 6 5 0.4

0.05* 0.01" 0.1

ns

ns

0.1 ns

0.05* 0.01" 0.05*

ns*

0.1 0.01" 0.05*

as*

ns

0.05* 0.1 0.05*

ns*

ns

ns

0.1 ns*

Evaluation Deduction Deduction

Generative trend Evaluative trend

Deduction Generative/Evaluative trend

Deduction Deduction Evaluation

G/D/E phase Evaluation Deduction Generation

G/D/E phase Deductive/Evaluative trend

Generation Generation/Evaluation

Evaluation G/D/E phase G/D/E phase

Generative/Evaluative trend Generation/Evaluation

G/D/E phase

* denotes outstanding phases

occurred were ignored. The average number of phases per protocol was

five, this is considered sufficient to make an object ive measure of designer

strategy. The abstracted protocols are shown in Table 2.

Before explaining what each phase type means, there are first several

possible inconsistencies in the analysis that deserve further thought. All

designers obviously do not design in phases of a uniform 15 utterances.

The reason for this segmentat ion, rather than segmenting according to the

actual content of the protocol , was to provide a unit small enough to

characterize the designing at a local level and also small enough to add to

any larger phases. By treating each protocol the same we are retaining a

measure of consistency that enables us to distinguish be tween design

models. What can be considered a normal distribution of ut terance

codings in one phase? The chi-square statistical test relies on the

difference be tween a normal and a test situation, that is a datum has to be

provided to test the protocol data against - but what does a 'normal '

134 Design Studies Vol 15 No 2 Apri l 1994

designer do? Different design models predict different strategies for

'normal' designers. One possible answer might be to assume one model

normal and test for a deviational measure of protocol data, repeating the

procedure for the second model, but we are then faced with a situation of

predicting, at a local level, what the designer will be doing in any given 15

utterance phase. The models under test simply do not allow this fine

grained prediction. Our preferred solution is to assume that a normal

designer will produce equal numbers of utterance codings in every

segment.

Table 2 shows the phase types exhibited by each designer in their

protocols. Phases were first classified, with respect to an even distribution

of utterance reasoning mode, into four general groups: 1) significantly

different (p <0.05); 2) approaching significance (p <0.1); 3) trend; and 4)

not significant. Secondly, the content of the phase was looked at to decide

the dominant mode(s) of reasoning. For example phase 2 in the protocol

of designer 1 is significantly a phase of deduction.

There is one type of phase that differs from the rest, that is the G/D/E

phase; this is a phase that we have assumed to be 'normal' behaviour, with

an equal distribution of all utterance types. Without becoming more localized, and without specific reference to the actual aspects of the design

'thought aloud' by the designer, it is impossible to assert that this is a

phase in the same sense as the other types of phase. Rather, a G/D/E

phase seems to be a collection of very quick iterations that reflect a rapid

breakdown, or decomposition, of an area of the design. Each iteration

roughly consists of equal numbers of generative, deductive, and elabora-

rive utterances, hence the balanced distribution.

4 Discussion The results of the previous section are presented in three sections. Figures

2 and 3 show general trends in designer behaviour that, at least super-

ficially, depend on experience. Figure 2 shows that, on average, an

increase in experience means an increase in generative utterances with a

corresponding decrease in deductive utterances. The number of evalua-

tive utterances remain fairly constant as the variable of experience

changes.

Figure 3 provides a dynamic illustration of each person's design process showing how reasoning modes interact with one another during any given time period. Figure 3 again shows differences in design processes that

seem to relate to experience; designers 4 and 5 show a marked difference from designers 1-3. Designers could almost be characterized into two


groups, deductors and generators. If we look at the individual graphs for

designers 4 and 5 we see that the reasoning modes seem to be much more

balanced than for designers 1-3. It seems that as generation becomes the

dominant mode of reasoning, the whole design process becomes more

stable.

Can we confidently say that these general trends come down to experi-

ence? Table 1 shows that although designers 4 and 5 are the most

experienced designers they are also the only designers to have received an

apprenticeship education. An apprenticeship education is based on

practical engineering skills rather than theoretical analysis or even design,

it seems far more likely that it would promote the type of generative

thinking, drawing on metaphors and examples, that has been displayed by

designers 4 and 5. There are, however, two reservations: firstly, it seems

an equally valid argument to say that general design experience, rather

than education, can provide the metaphors and examples that yield high

levels of generative utterances. Secondly, the apprenticeship/university

distinction in educational background does not explain the general trends

in the three university educated designers. There are clearly two variables

that could have influenced the content of the protocols, the type of

education, and general experience in design tasks.

Figures 2 and 3 provide a means for analysing the content of the protocols,

that is the numbers of different utterance types, in two ways. Statically in

terms of bar graphs, and dynamically in terms of cumulative frequency

graphs. We have noted general trends from this evidence of protocol

content, but until now have treated the design process as a whole. The

hypotheses under test require some order in the design process, so we now

need to break down the protocols to characterize designers in terms of this

order, or strategy. Table 2 is an attempt to combine the qualitative

elements of Figures 2 and 3 into a quantitative analysis relating directly to

the three design models introduced earlier.

Table 2, to distinguish between design models, breaks down the design

process in terms of fixed phases. It is the order of these phases that

indicate a particular strategy, however, each protocol has its idiosyncra-

sies. The small sample size of our study allows us to discuss each protocol individually and interpret parts of each protocol in terms of the three design models.

4. 1 Individual analysis of design protocols Designer I starts with a significant phase of evaluation, this is a process of

taking specific aspects of the design problem and talking about them in


general terms. For example designer 1 mentions gearboxes, he then goes

on to say what a usual gearbox ratio is, who the best gearbox suppliers are

etc. This is a phase of general orientation, he has no experience of the

type of the problem and has to form links in other ways. The second and

third phases are deductive. Now familiar with the content of the specifica-

tion, designer 1 then goes on to work out the implications of the

specification by expanding on aspects of the design problem, making the

problem clearer in his own mind. There then follows a period of

generation followed by a period of evaluation, this is where designer 1

begins to develop, and reflect on, his solution to the problem. Designer 1

closely follows an engineering model of design, there is clearly deduction

about the design problem before the generation of a solution. What is also

present is an ability to reason in one way for a long period of time, a phase

of 15 utterances in this instance, this seems to indicate more of a global

strategy than a localized strategy based on extensive problem structuring.

Designer 2 starts with a phase of deduction, he has experience of the

particular type of design problem and can use this knowledge to start

analysing the specification. The deductive phase is followed by a

generative/evaluative trend. This ordering of phases follows a similar

pattern to designer 1, deduction followed by generation, which we have

loosely interpreted as evidence for an engineering model of design.

However, designer 2 then goes on to two significantly deductive phases.

This behaviour is more suited to an architectural model of design. The

generative activity of producing an outline solution in phase 2 allowed him

to analyse the design problem more efficiently. It seems that specific

experience of the type of design problem used in the experiment, rather

than just general experience of the design discipline, has helped designer 2

to approach this design problem from the perspective of the solution

rather than the problem, although there are elements of both engineering

and architectural models in the protocol. Designer 2, like designer 1,

follows a globalized strategy that seems to be characterized by long

periods of deduction.

Designers 3 and 4 both show deductive activity before generative activity

which seems to indicate problem analysis before solution synthesis.

However, it is the position of the G/D/E phases that is of interest in these

protocols. As we have mentioned earlier the G/D/E phase represents an

ability to decompose and structure the design problem, to slice it into units that can then be considered individually. In both protocols this G/D/E phase appears early on, presumably when the designers are searching for structure in both problem and solution. Subsequent phases

in both protocols are deductive and elaborative before becoming genera-


tive. Although the general behaviour of designers 3 and 4 coincides with

the engineering model of design, it seems to be an engineering model

applied to a problem structured in the language of the designer rather

than the client. The deduction-generation method is applied to partial

problems rather than the whole problem. For designers 3 and 4, who are

more experienced than designers 1 and 2, it represents a change in strategy from global to localized.

Designer 5 is the most experienced designer and from the evidence so far

we would expect to see not only 'architectural' type behaviour due to

specific experience of the problem type (see Table 1), but also large

amounts of problem/solution structuring resulting from general discipline

experience. The protocol is consonant with these expectations; two

G/D/E phases appear at the beginning of the protocol suggesting a

decompositional process going on. What follows are phases of combined

generation-evaluation. Designer 5 has missed out any period of deduc-

tion. This is a surprise given that the architectural model of design implies that analysis will take place, though only after a solution has been

generated. Designer 5 seems somehow to have encapsulated his analytical

knowledge into the structuring of the problem. He intuitively knows that

his interpretation of the problem and solution, developed from the G/D/E

phases is the correct one. He displays the ability in the latter half of the

protocol to generate solutions and evaluate them without reference to the

problem. One possible explanation for this is the composition of the initial

G/D/E phases, designer 5 may have used an 'architectural' approach

within these phases, not to form a solution or analyse the design problem,

but to structure the design problem requiring specific knowledge of

problem structure.

In summary then, we have identified a number of processes and variables

that appear to influence the way that engineering designers solve design

problems. The general education of a designer may promote certain

attitudes to design problems but is unlikely to be a large factor when

designing. More likely to affect this process are the variables of specific experience of the problem type enabling designers to perceive new

problems through old solutions, and general domain experience which leads to the ability not simply to develop a design problem (deductive

behaviour) but to structure and decompose a design problem (G/D/E

behaviour). In general terms this is a distinction between a globalized attitude to design problems and a localized one. The variable of general design experience does not seem to alter a fundamentally analytical process: engineers educated analytically will remain so, unless they have

138 Design Studies Voi 15 No 2 April 1994

22 Downing, F 'Conversations in imagery' Design Studies Vol 13 No 3 (1992) 291-319 23 Galle, P and Kov=lms, L B 'Introspective observations of sketch design' Design Studies Vol 13 No 2 (1992) 229-272 24 Sehan, D A 'Designing: rules, types and worms' Design Stud/es Vol 9 (1988) 181-190 25 Sch~, D A and Wiggins, G 'Kinds of seeing and their function in designing' Design Studies Vol 13 No 2 (1992) 135-156

specific experience of the problem type. It is then that designers start to

approach design tasks through solutions, rather than through problems.

4.2 Experimental methodology The initial idea behind this paper was to explore what we thought was a

gap, or at least a disparity in the literature. Theories of the design process

had been produced in different design disciplines and this had led to vastly

different views of design. Our aim was to take these views, providing us

with a rich vocabulary for describing the 'design process' as a phe-

nomenon, and apply them to just one design discipline, that of electrical

engineering. Our framework of analysis drew on existing research but our

use of this framework was exploratory. Any such experiment is open to

methodological criticism. By discussing the drawbacks in our present

study we hope to show the typical problems of using a protocol analysis

type of approach to empirical design study.

In quantitative study one has to be objective in order to generalize

behaviour with the use of statistics. In protocol analysis this is commonly

done by the use of 'inter-rater reliability'; getting a judge to agree with

your basic theoretical distinctions. Once this is achieved statistical inter-

pretations are deemed significant. A problem in our study was that there

was much more information in the design situation than was available to

the judges in transcripts of the design session. There were several

utterances given different codings by all three raters; for example the

utterance 'I think if I put an oven there, it would cover this space' seems to

be evaluative, generative, and deductive. Inevitably this means that there

is a difference between the experimenter's view and the judges' view. In

short, it is extremely difficult as an experimenter to retain an unbiased

view of a design process that has been witnessed, however, by the same

token a second-hand account seems to remove meaning from the design

situation. What this seems to suggest is that design study provides a strong

argument for qualitative analysis. This seems to be the direction that recent design studies have taken 22-25. Further evidence in favour of

qualitative methods comes when we consider the problems of interpreting

data.

Once raw data has been coded there remains the problem of interpreta-

tion, working out what the data actually means. In our study we drew

general conclusions based around three possible models of the design process. Although statistics were used in the analysis, they were not

heavily relied upon in the interpretations. These interpretations do not represent a definitive view, but simply a subjective one, a qualitative

judgement and a judgement open to conjecture.


So even though we have basic objectivity in our protocol analysis, by far

the largest part of the analysis remains subjective. In addition each of the

designers in the present study showed remarkably different characteristics

suggesting that an individual 'case study' approach to the design situation

may be more appropriate than treating designers, or engineers, as one

homogenous group.

In our current work we are using the same basic models of the design

process discussed in this paper to look at the domains of architecture and

computer science. Our 'generation-deduction-evaluation' framework is

being used as a general guide, rather than a specific coding mechanism for

the design process (as in the work reported here). Protocols integrate

verbal data, graphical data, and review data to provide a more dynamic

description of the various levels operating in the design situation. In this

form design study yields data that is closely linked with design models,

that is, design activity need not be specifically coded to illustrate basic

cognitive approaches. This removes the problem of discrepancies in

coding, and interpretations of design behaviour can be offered directly

from empirical data.

5 Conclusions We have presented a design study of five electrical engineers that

describes design behaviour in terms of design models originating, not just

from engineering design theorists, but also from architectural and compu-

ter science theorists. In our study we found several variables that seem to

influence the individual's design process.

The variable of general engineering, or domain experience was linked to a

strategy for assimilating the design problem; less experienced designers

utilized a global approach to design problem solving whereas more

experienced designers used a localized approach demonstrating an ability

to decompose and structure the design problem. General domain experi-

ence did not, however, alter the basic analytical approach to problem

solving. It is the variable of specific experience of the problem type that

enables designers to adopt a conjectural approach to designing, that of framing or perceiving design problems in terms of relevant solutions.

We have also discussed the problems of quantitative design study suggesting that the design process provides an excellent opportunity for

qualitative and individual analysis, a direction that is becoming increasing- ly popular in the design literature.


Discovering the design problem

Documents

Transcript of Discovering the design problem