8/18/2019 How Did Software Get So Reliable Without Proof?
1/17
H o w D i d S o f t w a r e G e t S o R e l ia b l e
W i t h o u t P r o o f
C.A.R. Hoare
Oxford University Computing Laboratory,
Wolfson Building, Parks Road, Oxford, OX1 3QD, UK
A b s t r a c t By surveying current software engineering practice, this pa-
per reveals that the techniques employed to achieve reliability are little
different
from those which have proved effective in all other branches of
modern engineering: rigorous management of procedures for design in-
spection and review; quality assurance based on a wide range of targeted
tests; continuous evolution by removal of errors from products already in
widespread use; and defensive programming, among other forms of de-
liberate over-engineering. Formal methods and proof play a small direct
role in large scale programming; but they do provide a conceptual frame-
work and basic understanding to promote the best of current practice,
and point directions for future improvement.
1 I n t r o d u c t i o n
Twenty years ago it was reasonable to predict t ha t the size and ambition of soft-
ware products would be severely limited by the unreliability of their component
programs. Crude estimates suggest t ha t professionally written programs deliv-
ered to the customer can contain between one and ten independent ly correctable
errors per thousand lines of code; and any software error in principle can have
spectacular effect or worse: a subtly misleading effect) on the behaviour of the
entire system. Dire warnings have been issued of the dangers of safety-critical
software controlling health equipment, aircraft, weapons systems and industr ial
processes, including nuclear power stations. The arguments were sufficiently per-
suasive to trigger a significant research effort devoted to the problem of program
correctness. A proportion of this research was based on the ideal of certainty
achieved by mathemat ical proof.
Fortunately, the problem of program correctness has turned out to be far
less serious than predicted. A recent analysis by Mackenzie has shown that of
several thousand deaths so far reliably attributed to dependence on computers,
only ten or so can be explained by errors in the software: most of these were
due to a couple of instances of incorrect dosage calculations in the treatment
of cancer by radiation. Similarly predictions of collapse of software due to size
have been falsified by continuous operation of real-time software systems now
measured in tens of millions of lines of code, and subjected to thousands of
updates per year. This is the software which controls local and trunk telephone
exchanges; they have dramatically improved the reliability and performance of
8/18/2019 How Did Software Get So Reliable Without Proof?
2/17
t e l e c o m m u n i c a t i o n s t h r o u g h o u t t h e w o r l d . A n d a i r c r a f t , b o t h c i v i l a n d m i l i t a r y ,
a r e n o w f ly i n g w i t h t h e a i d o f s o f t w a r e m e a s u r e d i n m i l li o n s o f l in e s - t h o u g h
n o t a l l o f i t i s sa f e ty - c r i ti c a l . Co m p i l e r s an d o p e r a t in g sy s t em s o f a s im i l a r s i ze
n o w n u m b er th e i r sa t i s f i ed cu s to m er s i n m i l l i o n s .
S o t h e q u e s t i o n s a r is e : w h y h a v e t w e n t y y e a r s o f p e s s i m i s t i c p re d i c t i o n s b e e n
f a ls i fi e d ? W a s i t d u e t o s u c c es s fu l a p p l i c a t io n o f t h e r e s u l ts o f t h e r e s e a rc h w h i c h
w a s m o t i v a t e d b y t h e p r e d i c t io n s ? H o w c o u l d t h a t b e , w h e n c l e a r l y l i tt l e s o f tw a r e
h a s e v e r h a s b e e n s u b j e c t e d t o t h e r ig o u rs o f f o r m a l p r o o f ? T h e o b j e c t i v e o f th e s e
e n q u i r ie s i s n o t t o c a s t b l a m e f o r t h e n o n - fu l fi lm e n t o f p r o p h e c i es o f d o o m . T h e
h i s to r y o f sc i en ce an d en g in ee r in g i s l i t t e r ed w i th fa l se p r ed ic t io n s an d b r o k en
p r o m i s e s ; i n d e e d t h e y s e e m t o s e r v e a s a n e s s e n t i a l s p u r t o t h e a d v a n c e m e n t o f
h u m a n k n o w l e d g e ; a n d n o w a d a y s , t h e y a r e n e e d e d j u s t t o m a i n t a i n a d e cl in i n g
f lo w o f f u n d s f o r r e se a r ch . N i x o n s c a m p a i g n t o c u r e c a n c e r w i t h i n t e n y e a r s
w a s a to t a l f a il u re ; b u t i t c o n t r i b u t e d i n i t s t i m e t o t h e u n d e r s t a n d i n g o n w h i c h
t h e w h o l e o f m o l e c u l a r m e d i c i n e is n o w b a s e d . T h e p r o p e r r o le f o r a n h i s t o r ic a l
e n q u i r y i s t o d r a w l e s s o n s t h a t m a y i m p r o v e p r e s e n t p r a c t i c e s , e n h a n c e t h e
accu r acy o f f u tu r e p r ed ic t io n s , an d g u id e p o l i c i e s an d d i r ec t io n s f o r co n t in u ed
r e sea r ch in t h e su b jec t .
T h e c o n c l us i o n o f t h e e n q u i r y w i l l b e t h a t i n s p i t e o f a p p e a r a n c e s , m o d e r n
s o f t w a r e e n g in e e r in g p r a c t i c e o w e s a g r e a t d e a l t o t h e t h e o r e t i c a l c o n c e p t s a n d
i d e a ls o f e a r l y r e se a r ch i n t h e s u b j e c t ; a n d t h a t t e c h n iq u e s o f f o r m a l i s a t i o n a n d
p r o o f h av e p l ay ed an e s sen t i a l r o le in v a l id a t in g a n d p r o g r e ss in g th e r e sea rch .
Ho w ev e r , t e ch n o lo g y t r an s f e r i s ex t r em e ly s lo w in so f twar e , a s i t sh o u ld b e in an y
se r io u s b r an ch o f en g in ee rin g . Bec au se o f t h e b ack lo g o f r e sea r ch r e su l t s n o t y e t
u s e d , t h e r e i s a n i m m e d i a t e a n d c o n t in u i n g r o l e f o r e d u c a t i o n , b o t h o f n e w c o m e r s
t o t h e p r o fe s s io n a n d o f e x p e r ie n c e d p r a c t it i o n e r s. T h e f in a l r e c o m m e n d a t i o n i s
t h a t w e m u s t a i m o u r f u t u r e t h e o r e t i c a l r e s e ar c h o n g o a l s w h i c h a r e a s f a r a h e a d
o f t h e c u r r e n t s t a t e o f t h e a r t a s th e c u r r e n t s t a t e o f i n d u s t r i a l p r a c t i c e l a g s
b e h i n d t h e r e s e a r c h w e d i d i n t h e p a s t . T w e n t y y e a r s p e r h a p s ?
2 M a n a g e m e n t
T h e m o s t d r a m a t i c a d v a n c e s in t h e t i m e l y d e l iv e r y o f d e p e n d a b l e s o f t w a r e a r e
d i r e c t ly a t t r i b u t e d t o a w i d e r re c o g n i ti o n o f t h e f a c t t h a t t h e p r o c e s s o f p r o g r a m
d e v e l o p m e n t c a n b e p r e d i c t e d , p l a n n e d , m a n a g e d a n d c o n t r o l l e d i n t h e s a m e
w a y a s i n a n y o t h e r b r a n c h o f e n g in e e ri n g. T h e e v e n t u a l w o r k i n g s o f t h e p r o g r a m
i t s e lf a r e i n t e rn a l t o a c o m p u t e r a n d i n v is i b le t o t h e n a k e d e y e ; b u t t h a t i s n o
l o n g e r a n y e x c u s e f o r k e e p in g t h e d e s ig n p r o c e s s o u t o f t h e v i e w o f m a n a g e m e n t ;
a n d t h e v i s i b i l i t y s h o u l d p r e f e r a b l y e x t e n d t o a l l m a n a g e m e n t l e v e l s u p t o t h e
m o s t s e n io r . T h a t i s a n e c e s s a ry c o n d i t i o n fo r t h e a l l o c a t i o n o f t i m e , e f fo r t a n d
r e s o u r c es n e e d e d f o r t h e s o l u t i o n o f l o n g e r t e r m s o f t w a r e p r o b l e m s li k e t h o s e o f
re l iab i l i ty .
T h e m o s t p r o f i ta b l e i n v e s t m e n t o f e x t r a e f fo r t i s k n o w n t o b e a t t h e v e r y s t a r t
o f a p r o j e c t , b e g i n n i n g w i t h a n i n t en s if ie d s t u d y n o t o n l y o f t h e r e q u i r e m e n t s
o f t h e u l t i m a t e c u s to m e r , b u t a ls o o f t h e r e l at io n s h ip b e t w e e n t h e p r o d u c t a n d
8/18/2019 How Did Software Get So Reliable Without Proof?
3/17
t h e e n v i r o n m e n t o f i ts u l t i m a t e u s e. T h e g r e a t e s t n u m b e r b y fa r ) o f p r o j e c ts
t ha t ha ve e nd e d i n c a n c e l l a ti on o r f a i lu r e i n de l i ve r y a nd i n s t a l la t i on ha ve a l r e a d y
e g u n t o f a il a t t h i s s t age . O f cou r s e w e ha ve to l ive w i t h t he c on s t a n t c om p l a i n t
t h a t t h e c u s t o m e r s d o n o t k n o w w h a t t h e y w a n t; a n d w h e n a t l a st t h e y s a y th e y
d o , t h e y c o n s t a n t l y c h a n g e t h e i r m i n d . B u t t h a t i s n o e x c u s e f o r a b r o g a t i n g
m a n a g e m e n t r e s p o n s i b i l i t y . I n d e e d , e v e n s t r o n g e r m a n a g e m e n t i s r e q u i r e d to
e x p l o re a n d c a p t u r e t h e t r u e r e q u i r e m e n t s , t o s e t u p p r o c e d u r e s a n d d e a d l in e s
f o r m a n a g e m e n t o f c h a n g e , t o n e g o t i a te a n d w h e r e n e c es s a ry i nv o k e a n e a r l y
c a nc e l l a ti on c l a use in t he c on t r a c t . A bove a ll , t h e s t r ic t e s t m a n a g e m e n t i s ne e de d
t o p r e v e n t p r e m a t u r e c o m m i t m e n t t o s t a r t p r o g r a m m i n g a s so o n a s p o ss ib le .
T h i s c a n o n l y l e a d t o a v o l u m e o f c o d e o f u n k n o w n a n d u n t e s t a b l e u t il it y , w h i ch
w i ll a c t f o r e ve r a f t e r a s a de a d w e i gh t , b l i gh t ing t he s ubs e que n t p r og r e s s o f t he
pro j ec t , i f any .
T he t r a n s i t i on f r o m a n a na l y s i s o f r e qu i r e m e n t s to t he s pe c i f ic a t ion o f a p r o -
g r a m t o m e e t t h e m is t h e m o s t c r u c ia l s t a g e in t h e w h o l e p r o j ec t ; t h e d i sc o v e ry
a t t h i s s t a ge o f on l y a s i ng le e r r o r o r a s i ng le s im p l if i c at ion w ou l d f u l l y r e pa y a ll
the e f fort
e x p e n d e d . T o e n s u r e t h e p r o p e r d i r ec t io n o f e f fo r t, t h e m a n a g e m e n t
r e qu i r e s t ha t a ll p a r t s o f the s pe c i fi c at ion m u s t he s ub j e c t e d t o r e v i e w by t h e
b e s t a n d m o s t e x p e r i e n ce d s o f tw a r e a r c h it e ct s , w h o t h e r e b y t a k e u p o n t h e m -
s e lve s a n a pp r op r i a t e de g r e e o f r e s pons i b il it y f o r the s uc ce s s o f t he p r o j e c t . T ha t
i s w h a t e n a b l e s l a r g e i m p l e m e n t a t i o n t e a m s t o s h a r e t h e h a r d - w o n e x p e r i e n c e
a n d j u d g e m e n t o f t h e b e s t a v a il ab l e e ng i ne e rs .
S uc h i n s pe c t ions , w a l k t h r oughs , r e v i ew s a nd ga t e s a r e r e qu i r e d t o de f ine i m -
po r t a n t t r a n s i t ions be t w e e n a l l s ubs e que n t pha s e s in t he p r o j e c t , f r om p r o j e c t
p l a nn i ng , de s i gn , c ode , t e s t p l a nn i ng , a n d e va l ua t i on o f t e s t r e s u lt s . T he i nd i v i d -
u a l d e s ig n e r o r p r o g r a m m e r h a s t o a c c e p t t h e c h a ll en g e n o t o n l y o f m a k i n g t h e
r i gh t de c is ions , hu t a ls o o f p r e s e n ti ng t o a g r oup o f c o l le a gue s t he a r g um e n t s a nd
reasons fo r conf idence in t h e i r cor rec tness . Th i s i s am az in g ly e ffec ti ve i n i ns t il l -
i ng a nd s p r e a d i ng a c u l t u r e c ond uc i ve t o t he h i ghe s t r e li a b il it y . F u r t he r m or e , i f
t he r e v i ew c om m i t t e e i s no t s a t is f ie d t ha t t he p r o j e c t c a n s a f el y p r oc e e d t o i t s
ne x t pha s e , t he de s i gne r is r e qu i r e d t o r e - w or k t he de s ign a nd p r e s e n t i t a ga i n .
E v e n a t t h e e a r l ie s t s ta g e , m a n a g e m e n t k n o w s im m e d i a t e l y o f t h e s e t b a ck , a n d
a l r e a dy know s , e ve n i f t he y r e f u s e t o be li e ve i t, t h a t t he de l i ve ry w il l ha ve t o be
r e s c he du l e d by e xa c t l y t he s a m e in t e r va l t ha t ha s be e n l o s t. S l a c k f o r one o r t w o
s uc h s l ippa ge s s hou l d be bu i l t i n t o t he s c he du le ; bu t i f t he s l ac k is e xha u s t e d ,
a l t e r na t i ve a n d v i go r ous a c t i on s hou l d be no l onge r de l a ye d .
A t t h e p r e s e n t da y , m os t o f t he d i sc us si on a t r e v i e w m e e t i ngs is c ond uc t e d
i n a n e n t i r e l y i n f o r m a l w a y , u s i ng a l a ngu a ge a nd c onc e p t ua l f r a m e w or k e vo l ve d
l oc a ll y f o r t he pu r pos e . H ow e ve r , t he r e is now i nc r e a si ng e xpe r ie nc e o f the be n -
e fi ts o f i n t r o d u c i n g a b s t r a c t m a t h e m a t i c a l c o n ce p ts a n d r e a so n i n g m e t h o d s i n t o
t he p r oc e s s , r i gh t f r om t he be g i nn i ng . T h i s pe r m i t s t he c ons e que nc e s o f e a c h
p r opos e d f e a t u r e a nd t he i r pos s i b l e c om b i na t i ons t o be e xp l o r e d by c a r e f u l a nd
e x h a u s t i v e m a t h e m a t i c a l re a s o n in g , to a v o id t h e k i n d o f a w k w a r d a n d p e r h a p s
c r i t i c a l i n t e r a c t i ons t ha t m i gh t o t he r w i s e be de t e c t e d on l y on de l i ve r y . A t t he
de s i gn s t a ge , t he m a t he m a t i c s c a n he l p i n e xp l o ri ng the w ho l e o f t he de s i gn
8/18/2019 How Did Software Get So Reliable Without Proof?
4/17
s p a c e a n d s o g iv e g r e a t e r a s s u r a n c e t h a t t h e s i m p l e s t p o s s ib l e s o l u t i o n h a s b e e n
a d o p t e d . E v e n s t r i c t e r f o r m a l i s a t i o n i s r e c o m m e n d e d f o r s p e c if y i n g t h e i n te r f a ce s
b e t w e e n t h e c o m p o n e n t s o f t h e d e s ig n t o b e im p l e m e n t e d p e r h a p s i n d i ff e re n t
p l aces a t d i f fe r en t t im es b y d i f f er en t p eo p le . I d ea lly o n e w o u ld l ik e to see a p r o o f
i n a d v a n c e o f t h e i m p l e m e n t a t i o n t h a t c o r re c t n e s s o f t h e c o m p o n e n t s d e f in e d
in t e r m s o f sa t i s f ac t io n o f t h e in t e r f ace sp ec i f ica t io n s w i l l g u a r an tee co r r ec tn ess
o f t h e i r s u b s e q u e n t a s s e m b l y . T h i s c a n g r e a t l y r e d u c e t h e r is k o f a l e n g t h y a n d
u n p r ed ic t ab le p e r io d o f i n t eg r a t io n t e s t in g b e f o r e d e liv e ry .
A t t h e f i na l r e v ie w o f t h e c o d e j u d i c i o u s u s e o f c o m m e n t a r y i n t h e f o r m
o f a s s e rt io n s p r e c o n d i ti o n s p o s t c o n d i t i o n s a n d i n v a r i a n ts c a n g r e a t l y h e lp in
m a r s h a l l i n g a c o nv i n c in g a r g u m e n t t h a t a p r o g r a m a c t u a l l y w o r k s . F u r t h e r m o r e
i t i s m u ch ea s i e r t o f i n d b u g s in a l in e o f r ea so n in g th an i t i s i n a l i n e o f co d e . I n
p r in c ip l e co r r ec tn ess o f each li n e o f r ea so n in g d ep en d s a t m o s t o n tw o p r eced in g
l in es o f r ea so n in g wh ich a r e ex p l i c i t l y r e f e ren ced . I n p rin c ip le co r r ec tn ess o f
e a c h l i ne o f c o d e d e p e n d s o n t h e b e h a v i o u r o f e v e r y o t h e r l in e o f c o d e i n t h e
s y s t e m .
Su ccess i n t h e u se o f m a th em a t i c s f o r sp ec i f ica t io n d es ig n an d co d e r ev iews
d o e s n o t r e q u ir e s t r i c t f o r m a l i s a t i o n o f al l t h e p r o o fs . I n f o r m a l r e a s o n i n g a m o n g
t h o s e w h o a r e f lu e n t in t h e i d i o m s o f m a t h e m a t i c s i s e x t r e m e l y e ff ic ie n t a n d
r e m a r k a b l y r e li a b le . I t is n o t i m m u n e f r o m f a i lu r e; f o r e x a m p l e s i m p l e m i s p r i n t s
c a n b e s u rp r i s in g l y h a r d t o d e t e c t b y e y e . F o r t u n a t e l y t h e s e a r e e x a c t l y t h e
k i n d o f e r ro r t h a t c a n b e r e m o v e d b y e a r l y t e s ts . M o r e f o r m a l c a l c u l a ti o n c a n b e
r e se r v ed f o r t h e m o s t c r u c i a l i s su es su ch a s i n t e r r u p t s an d r eco v e r y p r o ced u r e s
w h e r e b u g s w o u l d b e m o s t d a n g e r o u s e x p e n s iv e a n d m o s t d i ff ic u lt t o d i a g n o s e
b y t e s t s .
A f ac i l i t y i n f o r m a l i sa t io n an d e f f ect iv e r ea so n in g i s o n ly o n e o f t h e t a l en t s
t h a t c a n h e l p i n a s uc c e s sf u l r e v ie w . T h e r e a r e m a n y o t h e r l e s s f o r m a l t a l e n t s
w h i c h a r e e s s en t ia l . T h e y i n c l u d e a w i d e u n d e r s t a n d i n g o f t h e a p p l i c a t i o n a r e a
a n d t h e m a r k e t p l a c e a n in t u i t iv e s y m p a t h y w i t h th e c u l t u r e a n d c o n c e r n s o f t h e
c u s t o m e r a k n o w l e d g e o f t h e s t r u c t u r e a n d s t y l e o f e x i s ti n g le g a c y c o d e a c q u a i n -
t a n c e a n d p r of es s io n a l r a p p o r t w i t h t h e m o s t a u t h o r i t a ti v e c o m p a n y e x p e r t s o n
each r e l ev an t t o p ic a s ix th sen se f o r t h e ev en tu a l o p e r a t io n a l co n seq u en ces o f
e a r l y d e s ig n d ec i si o ns a n d a b o v e a ll a d e e p s e n se o f p e r s o n a l c o m m i t m e n t t o
q u a l it y a n d t h e p a t i e n c e t o s u r v i v e l o n g p e r i o d s o f i n t e ll e c t u a l d r u d g e r y n e e d e d
to ach iev e a t h o r o u g h ly p r o f e ss io n a l r e su l t . Th ese a t t r i b u te s a r e e s sen t i a l . Th e
a d d i t i o n o f m a t h e m a t i c a l f lu e n c y t o t h e l is t is n o t g o i n g t o b e e a s y ; th e b e s t
h o p e i s t o sh o w th a t i t w i l l en h an ce p e r f o r m an ce in a l l t h e se o th e r way s a s we l l .
3 T e s t i n g
T h o r o u g h t e s t i n g i s t h e t o u c h s t o n e o f r e l ia b i l it y in q u a l i t y a s s u r a n c e a n d c o n t r o l
o f m o d e r n p r o d u c t i o n e n g in e e ri n g. T e s t s a r e a p p l i e d a s e a r l y a s p o s s i b l e a t a l l
s t a t i o n s i n t h e p r o d u c t i o n l i n e . T h e y a r e d e s i g n e d r i g o r o u s l y t o m a x i m i s e t h e
l ik e l ih o o d o f f a i lu r e an d so d e t ec t a f au l t a s so o n a s p o ss ib l e . Fo r ex am p le i f
p a r a m e t e r s o f a p r o d u c t i o n p r o c e s s v a r y c o n ti n u o u sl y t h e y a r e t e s t e d a t t h e
8/18/2019 How Did Software Get So Reliable Without Proof?
5/17
ex t r e m e o f t h e i r i n t en d ed o p e r a t in g r an g e . Sa t i s f ac t io n o f a l l t e s t s i n t h e f ac -
to r y a f f o r d s co n s id e r ab ly in c r ea sed co n f id en ce , o n th e p a r t o f t h e d es ig n e r , t h e
m a n u f a c t u r e r , a n d t h e g e n e ra l p u b li c , t h a t t h e p r o d u c t w i ll c o n t i n u e t o w o r k
w i th o u t f a i l t h r o u g h o u t i t s se rv i ce l i fe t im e . An d th e co n f id en ce i s j u s t i f i ed : m o d -
e r n c o n s u m e r d u r a b l e s a r e f a r m o r e d u r a b l e t h a n t h e y w e r e o n l y t w e n t y y e a r s
ag o .
B u t c o m p u t i n g s c i e n t i s t s a n d p h i l o s o p h e r s r e m a i n s k e p t i c a l . E . W . D i j k s t r a
h a s p o i n t e d o u t t h a t p r o g r a m t e s t in g c a n r e v e al o n l y t h e p r e s e n c e o f b u g s , n e v e r
t h e i r a b s e n c e . P h i l o s o p h e r s o f s c ie n c e h a v e p o i n t e d o u t t h a t n o s e ri e s o f e x p e r i-
m e n t s , h o w e v e r l o n g a n d h o w e v e r f a v o u r a b l e c a n e v e r p r o v e a t h e o r y c o r re c t; b u t
ev en o n ly a s in g le co n t r a r y ex p e r im e n t w i l l c e r t a in ly f a l s i fy i t . An d i t i s a b a s i c
s l o g an o f q u a l i t y a s s u r a n c e t h a t y o u c a n n o t t e s t q u a l i t y i n to a p r o d u c t . H o w
t h e n c a n t e s t i n g c o n t r i b u t e t o r e l ia b i li t y o f p r o g r a m s , t h e o ri e s a n d p r o d u c t s ? I s
th e co n f id en ce i t g iv es i l l u so r y ?
T h e r e s o l u t io n o f t h e p a r a d o x i s w e l l k n o w n i n t h e t h e o r y o f q u a l i t y c o n t r o l.
I t is t o en su r e th a t a t e s t m ad e o n a p r o d u c t i s n o t a t e s t o f t h e p r o d u c t i ts e l f,
b u t r a t h e r o f t h e m e t h o d s t h a t h a v e b e e n u s e d t o p r o d u c e i t - t h e p r o c e ss e s ,
t h e p r o d u c t i o n l i n e s , t h e m a c h i n e t o o l s , t h e i r p a r a m e t e r s e t t i n g s a n d o p e r a t i n g
d i sc ip l ine s . I f a t e s t f a i ls , i t i s n o t e n o u g h to m en d th e f au l ty p r o d u c t . I t i s n o t
e n o u g h j u s t t o t h r o w i t a w a y , o r e v e n t o re j e c t t h e w h o l e b a t c h o f p r o d u c t s i n
wh ich a d e f ec t iv e o n e i s f o u n d . T h e f i r s t p r in c ip l e i s t h a t t h e w h o le p r o d u c t io n
l in e m u s t b e r e - ex am in ed , i n sp ec ted , ad ju s t ed o r ev en c lo sed u n t i l t h e r o o t cau se
o f t h e d e f e c t h a s b e e n f o u n d a n d e l i m i n a te d .
Sc ien t i s t s a r e eq u a l ly sev e r e w i th th em se lv es . To t e s t a t h eo r y th ey d ev i se
a ser i es o f t h e m o s t r i g o r o u s p o ss ib l e ex p e r im en t s , a im ed ex p l i c i t l y an d ex c lu -
s i v e ly t o d i s p r o v e i t. A s i n g l e t e s t w i t h a n e g a t i v e r e s u l t m a y o c c a s i o n a l l y b e
a t t r i b u t e d t o i m p u r e i n g re d i e n ts o r f a u l t y a p p a r a t u s ; b u t i f t h e n e g a t i v e o u t -
c o m e i s r e p e a t e d , p a r t s o f t h e t h e o r y h a v e t o b e r e t h o u g h t a n d r e c a lc u l a te d ;
w h e n t h i s g e t s t o o c o m p l i c a t e d , t h e w h o l e t h e o r y h a s t o b e a b a n d o n e d . A s P o p -
p e r p o in t s o u t , t h e n o n - sc i en t i s t w i l l o f t en d i e w i th ( o r ev en fo r ) h i s fa l se b e l ie f s ;
t h e sc i en t i s t a l lo ws h i s b e l i e fs t o d i e i n s t ead o f h im se l f .
A t e s t i n g s t r a t e g y f o r c o m p u t e r p r o g r a m s m u s t b e b a s e d o n l e s s o n s l e a r n e d
f r o m th e su ccess f u l t r ea tm en t o f f a i lu re in o th e r b r an ch es o f sc i en ce an d en g i -
n ee r in g . Th e f i r s t l e s so n i s t h a t t h e t e s t s t r a t eg y m u s t b e l a id o u t i n ad v an ce
an d in a ll p o ss ib l e d e t a i l a t t h e v e r y ea r li e s t s t ag e in t h e p l an n in g o f a p r o j ec t .
T h e d e e p e s t t h o u g h t m u s t b e g i v e n t o m a k i n g t h e t e s t s a s s e v e r e a s p o s s i b l e ,
s o t h a t i t i s e x t r e m e l y u n l ik e l y t h a t a n e r r o r i n t h e d e s ig n o f t h e p r o g r a m c o u l d
p o s s i b l y r e m a i n u n d e t e c t e d . T h e n , w h e n t h e p r o g r a m is i m p l e m e n t e d a n d p a s s e s
a l l i t s t e s t s t h e f i rs t t im e , i t i s a lm o s t u n b e l i ev ab le th a t t h e r e co u ld b e an y in -
h e r e n t d e f e c t i n t h e m e t h o d s b y w h i c h t h e p r o g r a m h a s b e e n p r o d u c e d o r a n y
sy s t em a t i c lap se in t h e i r ap p l i ca t io n . Th i s is t h e m essag e o f Ha r l an M i l l' s c l ean
r o o m s t r a t e g y .
T h e e a r l i e s t p o s s i b l e d e s ig n o f t h e t e s t s t r a t e g y h a s s e v e ra l o t h e r a d v a n t a g e s .
I t en co u r ag e s ea r ly ex p lo r a t io n , s im p l i f i ca t io n an d c l a r if i ca tio n o f t h e a s su m p -
t i o n s u n d e r l y i n g u s e o f t h e p r o g r a m , e s p e c i al l y a t e d g e s o f i t s o p e r a t i n g r a n ge ;
8/18/2019 How Did Software Get So Reliable Without Proof?
6/17
it facilitates early detection of ambiguities and awkward interaction effects la-
tent in the specification; and it concentrates attent ion from the earliest stage on
central problems of assuring correctness of the system as a whole. Many more
tests should be designed than there will ever be time to conduct; they should be
generated as directly as possible from the specification, preferably automatically
by computer program. Random selection at the last minute will protect against
the danger that under pressure of time the program will be adapted to pass the
tests rathe r than meeting the rest of its specification. There is some evidence
that early attention to a comprehensive and rigorous test strategy can improve
reliability of a delivered product, even when at the last minute there was no time
to conduct the tests before delivery
The real value of tests is not t ha t they detect bugs in the code, but that they
detect inadequacy in the methods, concentration and skills of those who design
and produce the code. Programmers who consistently fail to meet their test-
ing schedules are quickly isolated, and assigned to less intellectually demanding
tasks. The most reliable code is produced by teams of programmers who have
survived the rigours of testing and delivery to deadline over a period of ten years
or more. By experience, intui tion, and a sense of personal responsibility they are
well qualified to continue to meet the highest standards of quality and reliability.
But don't stop the tests: they are still essential to counteract the distracting ef-
fects and the perpetual pressure of close deadlines, even on the most meticulous
programmers.
Tests that are planned before the code is written are necessarily "black box"
tests; they operate only at the outermost interfaces of the product as a whole,
without any cognizance of its internal structure. Black box tests also fulfil an
essential role as acceptance tests, for use on delivery of the product to the cus-
tomer's site. Since software is invisible, there is absolutely no other way of check-
ing that the version of the software loaded and initialised on the customer's ma-
chine is in fact the same as what has been ordered. Another kind of acceptance
test is the suite of certification tests which are required for implementations
of standard languages like COBOL and ADA. They do litt le to increase confi-
dence in the overall reliability of the compiler, but they do at least fairly well
ensure that all the claimed language features have in fact been delivered; past
experience shows th at even this level of reliability cannot be taken for granted.
Another common kind of black box test is regression testing. When main-
taining a large system over a period of many years, all suggested changes are
submitted daily or weekly to a central site. They are all incorporated together,
and the whole system is recompiled, usually overnight or at the week end. But
before the system is used for further development, it is subjected to a large suite
of tests to ensure that it still works; if not, the previous version remains in use,
and the p rogrammer who caused the error has an uncomfortable time unti l it is
mended. The regression tests include all those that have detected previous bugs,
particularly when this was done by the customer. Experience shows that bugs
are often a result o f obscurity or complication in the code or its documen tation;
and any new change to the code is all too likely to reintroduce the same bug -
something that customers find particularly irksome.
8/18/2019 How Did Software Get So Reliable Without Proof?
7/17
Debugg ing
The secret of the success of testing is th at it checks the quality of the process
and methods by which the code has been produced. These must be subjected to
continued improvement, until it is normal to expect tha t every test will be passed
first time, every time. Any residual lapse from this ideal must be tracked to
its source, and lead to lasting and widely propagated improvements in practice.
Expensive it may be, but that too is part of the cure. In all branches of commerce
and industry, history shows dramatic reduction in the error rates when their cost
is brought back from the customer to the perpetrator.
But there is an entirely different and very common response to the discov-
ery of an error by test: just correct the error and get on with the job. This is
known as debugging, by analogy with the att emp t to get rid of an infesta tion
of mosquitoes by killing the ones that bite you - so much quicker and cheaper
and more satisfying than draining the swamps in which they breed. For insect
control, the swatting of individual bugs is known to be wholly ineffective. But
for programs it seems very successful; on removal of detected bugs, the rate of
discovery of new bugs goes down quite rapidly, at least to begin with. The reso-
lut ion of the paradox is quite simple; it is as if mosquitoes could be classified into
two very distinct populations, a gentle kind tha t hardly ever bite, and a vicious
kind that bite immediately. By encouraging the second kind, it is possible to
swat them, and then live comfortably with the yet more numerous swarm that
remains. It seems possible th at a similar dichotomy in software bugs gives an
explanat ion of the effectiveness of debugging.
The first tests o f newly written code are those conducted by the programmer
separately on isolated segments. These are extraordinarily effective in remov-
ing typographical errors, miskeying, and the results of misunderstanding the
complexity of the programming language, the run-time library or the operating
system. This is the kind of error that is easily made, even by the most com-
peten t and diligent programmer, and fortunately just as easily corrected in to-
day' s fast-turnround visual program debugging environments. Usually, the error
is glaringly obvious on the first occasion that a given line of code is executed.
For this reason, the objective of the initial test suite is to drive the p rogram
to execute each line of its code at least once. This is known as a cover ge test;
because it is constructed in complete knowledge of the object under test, it
is classified as an open box test. In hardware design a similar principle is
observed; the suite of tests must ensure tha t every stable element makes at least
one transition from high voltage to low and at least one transition from low
voltage to high. Then at least any element that is stuck at either voltage level
will be detected.
The cheapest way of testing a new or changed module of code in a large
system is simply to insert the module in the system and run the standard suite of
regression tests. Unfortunately, the coverage achieved in this way does not seem
8/18/2019 How Did Software Get So Reliable Without Proof?
8/17
adequate: the proportion of code executed in regression tests has been reported
t
be less than thirty per cent. To improve this figure, a special test harness has
to be constructed to inject parameters and inspect results at the module level.
Unfortunately, for a module with many parameters, options and modes, to push
the coverage towards a hundred percent gets increasingly difficult; in the testing
of critical software for application in space, comprehensive testing is reported
to increase costs by four times as much as less rigorously tested code. Equally
unfortunate ly, total coverage is found to be necessary: more errors continue
t
be discovered right up to the last line tested.
In hardware design, exhaustive testing of stuck-at faults has also become im-
possible, because no sufficiently small par t of a chip can be exercised in isolation
from the rest. Nevertheless, quite short test sequences are adequate to identify
and discard fau lty chips as they come off the production line. It is a fortunate
property o f the technology of VLSI that any faults tha t are undetected by the
initial tests will very probably never occur; or at least they will never be noticed.
They play the role of the gentle kind of mosquito: however numerous, they hardly
ever bite.
Returning to the case of software, when the p rogram or the programmer has
been exhausted by unit testing, the module is subjected to regression testing,
which may throw up another crop of errors. When these are corrected, the regres-
sion tests soon stop detecting new errors. The same happens when an updated
system is first delivered to the customer: nearly all the errors are thrown up
in early runs o f the customer s own legacy code. After tha t, the rate at which
customers report new errors declines to a much lower and almost constant figure.
The reason for this is that even the most general-purpose programs are only
used in highly stereotyped ways, which exercise only a tiny proportion of the to tal
design space of possible paths through the code. Most of the actual patterns of
use are explored by the very first regression tests and legacy tests, and beta
testing enables the customer to help too. When the errors are removed from the
actually exercised paths, the rate at which new paths are opened up is very low.
Even when an anomaly is detected, it is often easier to avoid it by adap ting the
code that invokes it; this can be less effort and much quicker than reporting the
error. Perhaps it is by this kind of mutual adaption that the components of a
large system, evolving over many years, reach a level of natural symbiosis; as
in the world of nature, the reliability and stability and robustness of the entire
system is actually higher than tha t of any of its parts.
When this stable state is reached, analysis of a typical error often leads to an
estimate that, even if the error were uncorrected, the circumstances in which it
occurs are so unlikely that on a stat istical basis they will not occur again in the
next five thousand years. Suppose a hundred new errors of this kind are detected
each year. Crude extrapolation suggests that there must be about half a million
such errors in the code. Fortunately, they play the same role as the swarms of
the gentle kind of mosquito th at hardly ever bites. The less fortunate corollary is
th at if all the errors tha t are detected are immediately corrected, it would take
a thousand years to reduce the error rate by twenty percent. And that assumes
8/18/2019 How Did Software Get So Reliable Without Proof?
9/17
t h a t t h e r e a r e n o n e w e r r o r s i n t r o d u c e d b y t h e a t t e m p t t o c o r r e c t o n e w h i c h h a s
a l r e a d y b e e n d e t e c t e d . A f t e r a c e r t a i n s t a g e , i t c e r t a i n l y p a y s b o t h t h e c u s t o m e r
a n d t h e s u p p l i e r t o l e a v e s u c h e r ro r s u n r e p o r t e d a n d u n c o r r e c t e d .
U n f o r t u n a t e l y , b e f o r e t h a t s t a g e i s r e a c h e d, i t o ft e n h a p p e n s t h a t a n e w
v e r s io n o f t h e s y s t e m i s d e li v e re d , a n d t h e e r r o r r a t e s h o o t s u p a g a i n . T h e c o s t s
t o t h e c u s t o m e r a r e a c c e p t e d a s t h e p r ic e o f p r o g re s s : t h e c o s t t o t h e s u p p l i e r i s
co v e r ed b y th e p r o f i t o n th e p r i ce o f t h e so f twar e . T h e r ea l lo s s t o t h e su p p l i e r i s
t h e w a s t e o f t h e t i m e a n d s k il l o f t h e m o s t e x p e r ie n c e d p r o g r a m m e r s , w h o w o u l d
o t h e rw i s e b e m o r e p r o f i t a b ly e m p l o y e d i n im p l e m e n t i n g f u r t h e r i m p r o v e m e n t s
i n th e f u n c t i o n a l i t y o f t h e s o f t w a r e . A l t h o u g h s u r p ri s in g l y ) t h e f ig u re s a r e o f t e n
n o t o f f ic i al ly r e c o r d ed , t h e p r o g r a m m e r s t h e m s e l v e s e s t i m a t e t h a t n e a r l y h a l f
t h e i r t i m e i s s p e n t i n e r r o r c o r re c t io n . T h i s i s p r o b a b l y t h e s t r o n g e s t c o m m e r c i a l
a r g u m e n t f o r s o f t w a r e p r o d u c e r s t o i n c r e a s e i n v e s t m e n t i n m e a s u r e s t o c o n t r o l
r e l i ab i l i t y o f d e l iv e r ed co d e .
5 O v e r - e n g i n e e r i n g
Th e co n ce p t o f a sa f e ty f ac to r i s p e r v as iv e in en g in ee rin g . A f t e r ca l cu la t in g th e
w o r s t c a s e l o a d o n a b e a m , t h e c i v i l e n g i n e e r w i l l t r y t o b u i l d i t t e n t i m e s
s t r o n g e r , o r a t l e a s t tw ice a s s t r o n g , wh en ev e r t h e ex t r a co s t i s a f f o r d ab le . I n co m -
p u t i n g , a c o n t i n u i n g f al l i n p r i c e o f c o m p u t e r s t o r a g e a n d i n c re a s e i n c o m p u t e r
p o w e r h a s m a d e a l m o s t a n y t r a d e - o f f a c c e p t a b l e t o r e d u c e t h e r is k o f s o f t w a r e
e r ro r , a n d t h e s c a l e o f d a m a g e t h a t c a n i n c r ea s i n gl y r e s u lt f r o m i t. T h i s l e a d s t o
th e s am e k in d o f o v e r - en g in ee rin g a s is r eq u i r ed b y l aw f o r b r id g e - b u i ld in g ; an d
i t is ex t r e m e ly e f f ec t iv e , ev en th o u g h th e r e i s n o c l ea r w ay o f m easu r in g i t b y a
n u m e r i c f a c t o r .
T h e f ir s t b e n e f it o f a s u p e r a b u n d a n c e o f r e s ou r c e is t o m a k e p o s s i b l e a d e -
c is io n t o a v o i d a n y k i n d o f s o p h i s t i c a t i o n o r o p t i m i s a t i o n i n t h e d e s ig n o f a lg o -
r i t h m s o r d a t a s t r u c tu r e s . C o m m o n p r o h i b it i o ns a r e: n o d a t a p a c k in g , n o o p t i m a l
c o d i n g , n o p o i n t e r s , n o s h a r i n g , n o d y n a m i c s t o r a g e a l l o c a t i o n . T h e m a x i m u m
co n ce iv a b ly n ecessa r y s i ze o f r eco r d o r a r r ay i s a l l o ca t ed , an d th e n so m e m o r e .
S i m i l a r p r o h i b i ti o n s a r e o f t e n p la c e d o n p r o g r a m s t r u c t u re s : n o j u m p s , n o i n te r -
r u p t s , n o m u l t i p r o g r a m m i n g , n o g lo b a l v a r ia b l e s. A c c e ss to d a t a i n o t h e r m o d u l e s
i s p e r m i t t e d o n l y t h r o u g h c a r e fu l ly re g u l a t e d r e m o t e p r o c e d u r e c a l ls . I n t h e p a s t ,
t h e se d es ig n r u l e s we r e f o u n d to in v o lv e ex cess iv e lo ss o f e f fi c ien cy ; u p to a f ac to r
o f a h u n d r e d h a s b e e n r e c o r d ed o n f i rs t t ri a l s o f a r ig o r o u s l y s t ru c t u r e d s y s t e m .
T h i s f a c t o r h a d t o b e r e g a i n e d b y r e l a x in g t h e p r o h i b i ti o n s , m a s s a g i n g t h e i n te r -
f a c e s b e t w e e n m o d u l e s , e v e n to t h e e x t e n t o f v i o l a ti n g t h e s t r u c t u r a l i n t e g r i ty
o f t h e w h o l e s y s t e m . A p a r t f r o m t h e o b v i o u s i m m e d i a t e d a n g e r s , t h i s c a n l e a d
t o e v e n g r e a t e r r i sk a n d e x p e n s e in s u b s e q u e n t u p d a t i n g a n d e n h a n c i n g o f t h e
sy s t em . Fo r tu n a te ly , ch eap e r h a r d war e r ed u ces th e co n ce r n f o r e f f i c i en cy , an d
i m p r o v e d o p t i m i s a t i o n t e c h n o l o g y f o r h i g h e r l e v e l l a n g u a g e s p r o m i s e s f u r t h e r
a ss i s t an ce in r eco n c i l in g a c l ea r s t r u c tu r e o f t h e so u r ce co d e wi th h ig h e f f ic i en cy
i n t h e o b j e c t c o d e .
P r o f l ig a c y o f r e s o u rc e s c a n b r i n g b e n e f i ts i n o t h e r w a y s . W h e n c o n s i de r in g
8/18/2019 How Did Software Get So Reliable Without Proof?
10/17
]0
a p o s s i b le e x c e p t i o n a l c a se , t h e p r o g r a m m e r m a y b e q u i t e c o n f id e n t t h a t i t h a s
a l r e a d y b e e n d i s c r i m i n a t e d a n d d e a l t w i t h e l s e w h e r e in s o m e o t h e r p i e c e o f c o d e ;
a s a r e su l t i n f ac t t h e ex cep t io n can n ev e r a r i se a t t h i s p o in t . Nev e r th e l e s s , f o r
s a f e t y , i t i s b e t t e r t o d i s c r i m i n a t e a g a i n , a n d w r i t e f u r t h e r c o d e t o d e a l w i t h
i t. M o s t l i ke ly , t h e e x t r a c o d e w i ll b e t o t a l l y u n r e a c h a b l e . T h i s m a y b e p a r t
o f t h e e x p l a n a t i o n w h y i n n o r m a l t e s t in g a n d o p e r a t i o n , l es s t h a n t w e n t y p e r
c e n t o f t h e c o d e o f a l a r g e s y s t e m i s e v e r e x e c u te d ; w h i c h s u g g e s t s a n o v e r -
e n g i n ee r in g f a c t o r o f f iv e . T h e e x t r a c o s t i n m e m o r y s iz e m a y b e l o w , b u t t h e r e
i s a h i g h c o s t in d e si g n in g , w r i t i n g a n d m a i n t a i n i n g s o m u c h r e d u n d a n t c o d e .
Fo r ex am p le , t h e r e i s t h e to t a l ly p o in t l e s s ex e r c i se o f d e s ig n in g co v e r ag e t e s t s
f o r t h i s o t h e r w i s e u n r e a c h a b l e c o d e .
A n o t h e r p r o f li g a t e u s e o f r e s o u rc e s i s b y c l o n i n g o f c o d e . A n e w f e a t u r e t o b e
a d d e d t o a l a rg e p r o g r a m c a n o f te n b e c h e a p ly i m p l e m e n t e d b y m a k i n g a n u m b e r
o f s m a l l c h a n g e s t o s o m e p i e c e o f c o d e t h a t i s a l r e a d y t h e r e . B u t t h i s i s f e l t t o
b e r i s k y : t h e e x i s t i n g c o d e is p e r h a p s u s e d i n w a y s t h a t a r e n o t a t a ll o b v i o u s
b y j u s t l o o k i n g a t i t , a n d a n y o f t h e s e w a y s m i g h t b e d i s r u p t e d b y t h e p r o p o s e d
ch an g e . So i t s eem s sa fe r t o t ak e an en t i r e ly f r e sh co p y o f t h e ex i s t i n g co d e ,
a n d m o d i f y t h a t i n s t e a d . O v e r a p e r i o d o f y e a r s th e r e a r is e a w h o l e f a m i l y o f
su ch n ea r - c lo n es , ex t en d in g o v e r sev e ra l g en e r a t io n s . Eac h o f t h e m i s a q u ick an d
e f f i c i e n t s o l u t i o n t o a n i m m e d i a t e p r o b l e m ; b u t o v e r t i m e t h e y c r e a t e a d d i t i o n a l
p r o b l e m s o f m a i n t e n a n c e o f t h e l a r g e v o l u m e s o f c o d e . F o r e x a m p l e , i f a c h a n g e
i s m ad e in o n e v e r s io n o f t h e c lo n e, it i s q u i t e d i f f icu l t ev en to d ec id e wh e th e r
i t s h o u l d b e p r o p a g a t e d t o t h e o t h e r v e r si o n s, s o i t u s u a l l y is n t . T h e e x p e n s e
a r is e s w h e n t h e s a m e e rr o r o r d e f ic i en c y h a s t o b e d e t e c t e d a n d c o r r e c t e d a g a i n
in th e o th e r v e r s io n s .
A n o t h e r w i d e s p r e a d o v e r -e n g i ne e r in g p r a c t i c e i s k n o w n a s d e f e n s iv e p r o g r a m -
m i n g . E a c h i n d i v i d u a l p r o g r a m m e r o r t e a m e r e c ts a d e f e n si v e b a r r i e r a g a i n s t e r -
r o rs a n d i n s t a b il i ti e s i n t h e r e s t o f t h e s y s t e m . T h i s m a y b e n o t h i n g m o r e t h a n a
p r i v a t e l i b r a r y o f s u b r o u t i n e s t h r o u g h w h i c h al l c a ll s a r e m a d e t o t h e u n t r u s t e d
f e a t u re s o f a s h a r ed o p e r a t i n g s y s t e m . O r i t m a y t a k e t h e f o r m o f s t a n d a r d
c o d i n g p r a c t i c e s . F o r e x a m p l e , i t i s r e c o m m e n d e d i n a d i s t r i b u t e d s y s t e m t o
p r o t e c t e v e r y c o m m u n i c a t i o n w i t h t h e e n v i r o n m e n t , o r w i t h a n o t h e r p r o g r a m ,
b y a t i m e o u t , w h i c h w i ll b e i n v o k e d if t h e e x t e r n a l r e s p o n s e i s n o t s u f f i c ie n t ly
p r o m p t . C o n v e r s e l y , e v e r y m e s s a g e a c c e p t e d f r o m t h e e n v i r o n m e n t i s s u b j e c t e d
to r ig o r o u s d y n a m ic ch eck s o f p l au s ib i l it y , an d th e s l i g h te s t su s p ic io n wi l l c au se
t h e m e s s a g e t o b e j u s t i g n o r e d, i n th e e x p e c t a t i o n t h a t i t s s e n d e r i s s i m i l a r ly
p r o t e c t e d b y t i m e o u t .
A s i m i l a r t e c h n i q u e c a n b e a p p l i e d t o t h e g l o b a l d a t a s t r u c t u r e s u s e d t o
c o n t r o l t h e e n t i re s y s t e m . A n u m b e r o f c h e c ki n g p r o g r a m s , k n o w n a s s o f t w a r e
a u d i t s , a r e w r i t t e n t o c o n d u c t p l a u s i b i l it y c h ec k s o n a l l t h e r e c o r d s in t h e g l o b a l
sy s t em t ab le s . I n t h i s ca se , su sp ic io u s en t r i e s a r e r en d e r ed h a r m less b y a r e in i -
t i a l i s a t i o n t o s a f e v a l u e s . S u c h a u d i t s h a v e b e e n f o u n d t o i m p r o v e m e a n t i m e
b e t w e e n c ra s h es o f a n e m b e d d e d s y s t e m f r o m h o u rs t o m o n t h s . T h e o c c a s io n a l
l o ss o f d a t a a n d f u n c t i o n i s u n n o t i c e d i n a te l e p h o n e s w i t c h i n g a p p l i c a t i o n : i t
c o u l d h a r d l y b e r e c o m m e n d e d f o r a i r t r a f f i c c o n t r o l , w h e r e i t w o u l d c e r t a i n l y
8/18/2019 How Did Software Get So Reliable Without Proof?
11/17
11
cause quite a different kind of crash.
The ul timate and very necessary defenee of a real time sys tem agains t arbi-
trary hardware error or operator error is the organisation of a rapid procedure
for restarting the entire system. The goal of a restart is to restore the system
to a valid state that was current some time in the recent past. These warm
starts can be so efficient that they are hardly noticeable except by examining
the historical system log. So who cares whether the trigger for a restart was a
rare software fault or a transient hardware fault? Certainly it would take far too
long to record information that would permit them to be discriminated.
The limitation of over-engineering as a safety technique is that the extra
weight and volume may begin to contribute to the very problem that it was
intended to solve. No-one knows how much of the volume of code of a large
sys tem is due to over-engineering or how much this costs in terms of reliability.
In general safety engineering it is not unknown for catastrophes to be caused
by the very measures that are introduced to avoid them.
6 P r o g r a m m i n g M e t h o d o l o g y
Most o f the measures described so far for achieving reliability of programs are the
same as those which have proved to be equally effective in all engineering and in-
dustrial enterprises from space travel to highway maintenance from electronics
to the brewing of beer. But the best general techniques of managemen t quality
control and safety engineering would be totally useless by themselves; they are
only effective when there is a general understand ing of the specific field of en-
deavour and a common conceptual framework and terminology for discussion
of the relationship between cause and effect between action and consequence in
that field. Perhaps initially the understanding is based just on experience and
intuit ion; but the goal of engineering research is to complement and sometimes
replace these in formal judgements by more systematic methods of calculation
and optimisation based on scientific theory.
Research into programming methodology has a similar goal to establish a
conceptual framework and a theoretical basis to assist in systematic derivation
and justification of every design decision by a rational and explicable train of
reasoning. The primary method of research is to evaluate proposed reasoning
methods by their formalisation as a collection of proof rules in some completely
formal system. This permits definitive answers to the vital questions: is the
reasoning valid? is it adequate to prove everything that is needed? and is it
simpler than other equally valid and adequate alternatives? It is the provably
positive answer to these simple questions that gives the essential scientific basis
for a sound methodological reeo rnmendation- certainly an improvement on mere
rhetoric speculation fashion salesmanship char latanism or worse.
Research into prog ramming methodology has already had dramatic effects
on the way that people write programs today. One of the most spectacular
successes occurred so long ago that it is now quite non-controversial. It is the
almost universal adoption o f the practice of structu red programming otherwise
8/18/2019 How Did Software Get So Reliable Without Proof?
12/17
2
known as avoidance of jumps or gotos). Millions of lines of code have now been
written without them. But it was not always so. At one time, most programmers
were proud of their skill in the use of jumps and labels. They regarded struc tured
notat ions as unnatural and counter-intuitive, and took it as a challenge to write
such complex networks of jumps that no structu red notations could ever express
them.
The decisive breakthrough in the adoption of structured programming by
IBM was the publication of a simple result in pure programming theory, the
Bohm-Jacopini theorem. This showed that an arbitrary program with jumps
could be executed by an interpreter written without any jumps at all; so in
principle any task whatsoever can be carried out by purely structured code.
This theorem was needed to convince senior managers of the company that no
harm would come from adopting structu red programming as a company policy;
and project managers needed it to protect themselves from having to show their
programmers how to do it by rewriting every piece of complex spaghetti code
that might be submitted. Instead the programmers were just instructed to find
a way, secure in the knowledge that they always could. And after a while, they
always did.
The advantages of structured programming seem obvious to those who are
accustomed to it: programs become easy to write, to unders tand, and to modify.
But there is also a good scientific explanation for this judgement. It is found
by a formalisation of the methods needed to prove the correctness of the pro-
gram with the aid of assertions. For structured programs, a straightfo rward proof
always suffices. Jumps require a resort to a rather more complex technique of
subsidiary deductions. Formalisation has been invaluable in giving objective sup-
port for a subjective judgement: and th at is a contribution which is independent
of any att empt to actually use the assertional proof rules in demonstra ting the
correctness of code.
Another triumph of theory has been widespread appreciation of the benefits
of data types and strict type-checking of programs. A type defines the outer limits
of the range of values for a program variable or parameter. The range of facilities
for defining types is sufficiently restricted t ha t a compiler can automatically
check that no variable strays outside the limits imposed by its declared type.
The repertoire of operations on the values of each type are defined by simple
axioms similar to those which define the relevant branch of mathemat ics. Strict
typechecking is certainly popular in Universities, because of the help it gives in
the teaching of programming to large classes of studen ts with mixed abilities; it
is even more widely beneficial in modern mass consumer languages like Visual
Basic; and in very large programs which are subject to continuous change, it
gives a vital assurance of global system integrity that no programmer on the
project would wish to forego.
Another t riumph of theoretical research has been widespread adoption of the
principles of information hiding. An early example is found in the local variables
of ALGOL 60. These are introduced by declaration and used as workspace for
internal purposes of a block of code which constitutes the scope of the declara-
8/18/2019 How Did Software Get So Reliable Without Proof?
13/17
13
t i on ; t h e v a r i a b l e n a m e i t s i d e n t it y a n d e v e n i ts e x i s te n c e i s t o t a l l y c o n c e a l ed
f r o m o u t s id e . T h e c o n c e p t o f d e c l a r a t i o n a n d l o c a l it y i n a p r o g r a m w a s b a s e d
o n t h a t o f q u a n t i f i c a t io n a n d b o u n d v a r i a b le s in p r e d i c a t e l o gi c; a n d s o a r e t h e
p r o o f m e t h o d s f o r p r o g r a m s w h i c h c o n ta i n t h e m .
T h e i n f o r m a t i o n h i d i n g i n t r o d u c e d b y t h e A L G O L 6 0 l o c a l v a r i a b l e w a s
g e n e r a li s e d t o t h e d e s ig n o f l a rg e r -s c a le m o d u l e s a n d c la s se s o f o b j e c t - o r i e n t e d
p r o g r a m m i n g i n t r o d u c e d i n t o A L G O L 6 0 b y S I M U L A 6 7 . A g a i n t h e sc i en t if ic
b a s i s o f t h e s t r u c t u r e w a s e x p l o r e d b y f o r m a l i s a t i o n o f t h e r e l ev a n t p r o o f t e ch -
n iq u es in v o lv in g an ex p l i c it i n v a r i an t wh ich l i n k s an ab s t r ac t co n cep t w i th
t s
c o n c r e t e r e p r e s e n t a t i o n a s d a t a i n t h e s to r e o f a c o m p u t e r .
T h e v a l u e o f a f o u n d a t i o n i n fo r m a l l o gi c a n d m a t h e m a t i c s i s i l l u s t r a te d b y
t h e c o m p a r i s o n o f A L G O L 6 0 w i t h t h e C O B O L l a ng u a g e b r o u g h t i n to e x is t en c e
a n d s t a n d a r d i s e d a t a b o u t t h e s a m e t im e b y t h e U . S . D e p a r t m e n t o f D e f en c e.
B o t h l a n g u a g e s h a d t h e h i g h ly c o m m e n d a b l e a n d e x p li ci t o b j e c ti v e o f m a k i n g
p r o g r a m s e a s ie r t o u n d e r s t a n d . C O B O L t r ie d t o d o t h i s b y c o n s t r u c ti n g a c ru d e
a p p r o x i m a t i o n t o n o r m a l n a t u r a l E n g l is h w h e r e a s A L G O L 6 0 t r i e d to g e t cl o se r
t o t h e l a n g u a g e o f m a t h e m a t i c s . T h e r e i s n o d o u b t w h i c h w a s t e c h n ic a l l y m o r e
s u c ce s s fu l : t h e i d e a s o f A L G O L 6 0 h a v e b e e n a d o p t e d b y m a n y s u b s e q u e n t l a n -
g u a ge s i n cl u di n g e v en F O R T R A N 9 0. C O B O L b y c o m p a r i so n h as t u r n e d o u t
t o b e a n e v o l u t i o n a r y d e a d e n d .
Con c l u s i o n
T h i s r e vi e w o f p r o g r a m m i n g m e t h o d o l o g y r ev e a ls h o w m u c h t h e b e s t o f c u rr e n t
p r a c t i c e o w e s t o t h e i d e a s a n d u n d e r s t a n d i n g g a i n e d b y re s e a rc h w h i c h w a s
c o m p l e t e d m o r e t h a n t w e n t y y e a r s a g o . T h e e x i s te n c e o f s u c h a l a rg e g a p b e t w e e n
t h e o r y a n d p r a c t i c e is d e p l o r e d b y m a n y b u t I t h i n k q u i t e w r o n g l y . T h e g a p i s
a c t u a l l y a n e x t r e m e l y g o o d s i g n o f t h e m a t u r i t y a n d g o o d h e a l t h o f o u r d i sc i p li n e
an d th e o n ly d ep lo r ab le r e su l t s a r e t h o se th a t a r i se f r o m f a i lu r e t o r eco g n ise i t .
T h e p r o p e r r e s p o n s e t o t h e g a p i s t o f ir s t c o n g r a t u l a t e t h e p r a c t i t i o n e r s f o r
t h e i r g o o d s e n s e. E x c e p t i n t h e n a r r o w e s t a r e as a n d f o r t h e s h o r t e s t p o s s i b l e
p e r i o d s o f t i m e i t w o u l d b e c r a z y fo r i n d u s t r y t o t r y t o k e e p p a c e w i t h t h e l a t e s t
r e su l t s o f p u r e r e sea rch . I f t h e r e sea rch f a i ls t h e in d u s t r y f a il s w i th i t ; an d i f
t h e r e sea r ch co n t in u es to su cceed th e in d u s t r y wh ich is f i rs t t o i n n o v a te r u n s
t h e r i s k o f b e i n g o v e r t a k e n b y c o m p e t i t o r s w h o r e a p t h e b e n e f it s o f t h e l a t e r
i m p r o v e m e n t s . F o r t h e s e r e a s o n s i t w o u l d b e g r o s s ly i m p r o p e r t o r e c o m m e n d
i n d u s t r y o n i m m e d i a t e i m p l e m e n t a t i o n o f r e s u lt s o f t h e i r o w n re s e a rc h t h a t i s
s t i l l i n p r o g r e ss . I n d eed S i r R ich a r d D o l l p o in t s o u t t h a t sc i en t i s t s wh o g iv e
s u c h a d v i c e n o t o n l y d a m a g e t h e i r c l ie n ts ; t h e y a l s o lo s e t h a t m o s t p r e c i o u s o f
a l l a t t r i b u te s o f g o o d re sea r ch th e i r sc ien t if i c o b jec t iv i ty .
T h e t h e o r i s t s a l s o s h o u l d b e a c c o r d e d a f u l l s h a r e o f t h e c o n g r a t u l a t i o n s ;
f o r i t i s t h e y w h o h a v e a c hi e v e d r e se a rc h r e s u lt s t h a t a r e t w e n t y y e a r s a h e a d o f
th e f i e ld o f p r ac t i ce . I t i s n o t t h e i r f a i l in g b u t r a th e r t h e i r d u t y to ach iev e an d
m a i n t a i n s u c h a n u n c o m f o r t a b l e le a d a n d t o s p r e a d i t o v e r a b r o a d f r o n t a c ro s s
a w i d e r a n g e o f t h e o ri e s . N o - o n e c a n p r e d i c t w i t h a n y c e r t a i n t y o r a c c u r a c y o f
8/18/2019 How Did Software Get So Reliable Without Proof?
14/17
14
d e t a i l , t h e t i m e s c a l e s o f c h a n g e i n t e c h n o l o g y o r in t h e m a r k e t p l a c e . T h e d u t y
o f t h e r e s e a r c h er i s n o t t o p r e d i c t t h e f u t u r e m o r e a c c u r a t e l y t h a n t h e b u s i -
n e s s m a n , b u t t o p r e p a re t h e b a s i c u n d e r s t a n d in g w h i c h m a y b e n e e d e d t o d e a l
w i t h t h e u n e x p e c t e d c h a ll e ng e s o f a n y p o s s ib l e f u t u r e d e v e l o p m e n t . P r o v i d e d
t h a t t h i s g o a l h a s b e e n m e t , n o r e s ea r c he r s h o u l d b e b l a m e d f o r f a i lu r e o f e a r l y
p r e d i c t i o n s m a d e t o j u s t i f y i t s o ri g in a l f u n d i n g o f t h e r e s e a rc h . M i s t a k e s m a d e
b y b u s i n e s s m e n a n d p o l i ti c i a n s a r e fa r m o r e e x p e n s iv e .
T h e r e c og n i ti o n o f t h e a p p r o p r i a t e t im e s c a l e to m e a s u r e t h e g a p b e t w e e n t h e
t h e o r y a n d p r a c t i c e o f a d i sc i p li n e i s a n e s s e n ti a l t o t h e a p p r o p r i a t e p l a n n i n g o f
r e s e ar c h a n d e d u c a t i o n , b o t h t o fill t h e g a p b y im p r o v i n g p ra c t i c e , a n d t o e x t e n d
i t a g a i n b y a d v a n c i n g t h e t h e o r y . I w o u l d r e c o m m e n d t h a t t h e b e s t r e s e ar c h e rs i n
t h e f i e ld s h o u l d s i m u l t a n e o u s l y t r y t o d o b o t h , b e c a u s e t h e i n fl u e n ce o f p r a c t i c e
o n t h e d e v e l o p m e n t o f t h e o r y i s m o r e b e n ef ic i al a n d a c t u a l l y q u i c k e r t h a n t h e
o t h e r w a y r o u n d .
A t t h e e x t r e m e o f t h e p r a c t ic a l e n d , I w o u l d r e c o m m e n d t h e t h e o r is t t o
a l t e r n a t e t h e o r e t i c a l p u r s u i t s w i t h m u c h c l o s e r o b s e r v a t i o n a n d e x p e r i m e n t a t i o n
o n a c t u a l w o r k i n g p r o g r a m s , w i t h a ll t h e m a s s o f d o c u m e n t a t i o n a n d h i s t o ri c a l
d e v e l o p m e n t lo g s t h a t h a v e a c c u m u l a t e d i n t h e l a s t t e n y e a rs . T h e s e s y s t e m s
a r e n o w su f f i c i en t ly s t ab l e , an d h av e su ff ic i en t co m m e r c ia l p r o sp ec t s , t o j u s t i f y
q u i t e p r a c t i c a l r e s ea r c h t o a n s w e r q u e s t i o n s t h a t w i ll g u i d e r e c o m m e n d a t i o n s f o r
f u t u r e b e n e fi c ia l c h an g e s in t h e ir s t r u c t u r e , c o n t e n t o r m e t h o d s o f d e v e l o p m e n t .
F o r e x a m p l e , i t w o u l d b e v e r y i n te r e s ti n g t o f i n d a w a y o f e s t i m a t i n g t h e p r o -
p o r t i o n a l c o s t o f c l o n in g a n d t h e o t h e r o v e r -e n g i ne e r in g p r a c t i c es . B y s a m p l i n g ,
i t w o u l d b e i n t e r e st i n g t o t r a c e a n u m b e r o f e r ro r s t o t h e i r r o o t c a u s e , a n d s e e
h o w t h e y m i g h t h a v e b e e n a v o i d e d , p e r h a p s b y b e t t e r s p e c i f i c a t i o n o r b y b e t t e r
d o c u m e n t a t i o n o r b y b e t t e r s t r u c t u r i n g o f c o d e. Is m y c o n j e c t u re d d i c h o t o m y
o f e r ro r p o p u l a t i o n s o b s e r v ed i n p r a c ti c e ? A n y r e c o m m e n d a t i o n f o r i m p r o v e d
f o r m a l i s a t i o n o r i m p r o v e d s t r u c t u r e w i l l p r o b a b l y b e b a s e d o n o t h e r p e o p l e s r e-
s e a r c h i d e a s t h a t a r e u p t o t w e n t y y e a r s o l d . E v e n s o , t h e y m u s t b e b a c k e d u p b y
t r i a l r eco d in g o f a r an g e o f ex i s t i n g m o d u le s , s e l ec t ed o n th e sc i en t if i c p r in c ip l e
o f b e i n g t h e m o s t l ik e ly t o r e v e a l t h e f a ll a ci e s i n t h e r e c o m m e n d a t i o n , r a t h e r
t h a n i t s m e r i t s . S t r a n g e t o r e l a t e , i t h a s b e e n k n o w n f o r a b u s i n e s s t o s p e n d
m a n y m i l l i o n s o n a c h a n g e t h a t h a s n o t b e e n s u b j e c t e d t o a n y p r i o r s c i e n t i f i c
t r i a l s o f t h i s k in d .
F o r m a l m e t h o d s r e s e a r c h e r s w h o a r e r e a l l y k e e n o n r i g o r o u s c h e c k i n g a n d
p r o o f s h o u l d i d e n t if y a n d c o n c e n t r a t e o n t h e m o s t c r i t ic a l a r e a s o f a l a r g e s o f t-
w a r e s y s t e m , f o r e x a m p l e , s y n c h r o n i s a t i o n a n d m u t u a l e x c l u s i o n p r o t o c o l s , d y -
n am ic r e so u r ce a l lo ca t io n , an d r eco n f ig u r a t io n s t r a t eg ie s f o r reco v e r y f r o m p a r -
t i a l s y s t e m f a i lu r e . I t is k n o w n t h a t t h e s e a r e a r e a s w h e r e o b s c u r e t i m e - d e p e n d e n t
e r ro r s , d e a d l o c k s a n d l iv e lo c k s ( t h ra s h i n g ) c a n l u r k u n t e s t a b l e f o r m a n y y e a r s,
a n d t h e n t r ig g e r a f a il u re c o s t i n g m a n y m i ll io n s . I t i s p o s s i b l e t h a t p r o o f m e t h o d s
a n d m o d e l c h e c k i n g a r e n o w s u f f i c i e n t l y a d v a n c e d t h a t a g o o d f o r m a l m e t h o d -
o l o g i s t c o u ld o c c a s i o n a l ly d e t e c t s u c h o b s c u r e l a t e n t e r ro r s b e f o r e t h e y o c c u r i n
p r a c t ic e . P u b l i c a t i o n o f s u c h a n a c h i e v e m e n t w o u l d b e a m a j o r m i l e s t o n e in t h e
a c c e p t a n c e o f f o r m a l m e t h o d s i n so l v in g t h e m o s t c r i ti c a l p r o b l e m s o f s o f t w a r e
re l iab i l i ty .
8/18/2019 How Did Software Get So Reliable Without Proof?
15/17
15
I h a v e s u g g e s t e d t h a t p e r s o n a l i n v o l v e m e n t i n c u r r e n t p r a c t i c e s a n d i n sp e c -
t i o n o f l e g a c y c o d e m a y l e a d t o q u i t e r a p i d b e n e fi ts b o t h t o t h e p r a c t i t i o n e r a n d
t o t h e t h e o r i s t. B u t t h i s is n o t t h e r i g h t p e r m a n e n t r e l a ti o n s h i p b e t w e e n t h e m ;
i n a p r o p e r p o l i c y o f t e c h n o l o g y tr a n s f e r i t i s f o r t h e p r a c t i t i o n e r t o r e c o g n is e
p r o m i s i n g r e s u lt s o f r e se a r ch a n d t a k e o v e r a ll t h e h a r d w o r k o f a d a p t i n g t h e m
f o r w i d e s p r e a d a p p l ic a t i o n . In s o f t w a r e u n f o r t u n a t e l y t h e g a p b e t w e e n p r a c t i c e
a n d t h e o r y i s n o w s o l a rg e t h a t t h i s is n o t h a p p e n i n g . P a r t o f t h e t r o u b l e i s
t h a t m a n y o r m o s t o f t h e p r a c t it io n e r s d i d n o t s t u d y f o r m a l m e t h o d s o r e v e n
c o m p u t i n g s ci e n ce a t U n i v e r s i ty . T h i s l e a v es a l a rg e e d u c a t i o n a l g a p t h a t c a n
o n l y b e f il le d b y p r o g r a m m e o f i n -s e rv i ce e d u c a t i o n w h i c h w i l l a c q u a i n t s o m e
o f t h e b e s t s o f t w a r e e ng i n ee r s i n in d u s t r y w i t h s o m e o f t h e i m p o r t a n t i d e a s o f
c o m p u t i n g s c ie n c e. S in c e m a n y o f t h e m h a v e d eg r e es in m a t h e m a t i c s o r a t l e a s t
i n s o m e m a t h e m a t i c a l b r a n c h o f s c ie n ce t h e y h a v e t h e n e c e s s a r y b a c k g r o u n d
a n d a b i l i ty : s i n c e t h e y d o n o t h a v e d eg r e es i n c o m p u t i n g t h e y n e e d t o s t a r t
r i g h t a t t h e b e g i n n in g f o r e x a m p l e w i t h c o n t e x t fr e e l a n g u a g e s a n d f in i te s t a t e
m a c h i n e s a n d s i m p l e i d e a s o f t y p e s a n d f u n c t i o n a l p r o g r a m m i n g .
A n o th e r h ig h b a r r i e r t o t e ch n o lo g y t r an s f e r i s t h e f a i lu r e o f so f twa re en g i -
n e e r in g t o o l s e t s t o in c l u d e a m o d i c u m o f s u p p o r t f o r f o r m a l i t y f o r e x a m p l e t o
a l lo w m a t h e m a t i c a l n o t a t i o n s i n w o r d pr o c e ss o r s t o i n c o r p o r a t e t y p e c h e c k i n g
fo r sp ec i f i ca tio n s an d h y p e r t e x t t e ch n iq u es fo r q u ick c ro ss - re f e r en c in g b e twe en
f o r m a l a n d i n f o r m a l d o c u m e n t a t i o n . I m p r o v e d t o o l s s h o u l d c o n c e n t r a t e f i r s t o n
v e ry s im p le o ld t ech n iq u es l i k e ex ecu t io n p ro f i l e s an d se l ec t iv e co m p i l a t io n o f
a s s e r t io n s b e f o r e g o i n g o n t o m o r e a d v a n c e d b u t l e s s m a t u r e t e c hn o l o g y s u c h a s
m o d e l c h e c k in g o r p r o o f a s s is t a n ce . T h e a c t u a l c o n s t r u c t i o n o f i n d u s t r ia l q u a l i t y
t o o l s m u s t b e d o n e i n c o l l a b o r a t i o n w i t h t h e i n d u s t r ia l s u p p l i e rs o f t h e s e t o o l s .
O n l y t h e y h a v e t h e k n ow l e d g e a n d p r o fi t m o t i v e t o a d a p t t h e m a n d t o c o n t in u e
a d a p t i n g t h e m t o t h e ra p i d l y c h a n g in g f a s h io n s a n d n e e d s o f t h e m a r k e t p l a c e .
F o r l o n g - t e r m r e s ea r ch m y a d v i c e i s e v e n m o r e t e n t a t i v e a n d c o n t ro v e r s ia l .
I t p u rs u e s a h o p e t o c o m p l e m e n t t h e m a n y s t re n g t hs a n d c o m p e n s a t e t h e si n-
g l e w e a k n e s s o f c u r r e n t t h e o r e t i c a l r e se a rc h i n f o rm a l m e t h o d s . T h e s t r e n g t h s
a r is e f ro m t h e d e p t h a n d t h e r a n g e o f t h e s p e c i a li s a t io n o f m a n y f l o u r is h i ng r e -
s e a rc h s c h o o l s i n a ll t h e r e l e v a n t a r e as . F o r e x a m p l e i n p r o g r a m m i n g l a n g u a g e
s e m a n t i c s w e h a v e r e a s o n in g b a s e d o n d e n o t a t i o n a l a l g e b r a ic a n d o p e r a t i o n a l
p r e s e nt a ti o n s . A m o n g p r o g r a m m i n g p a r a d i g m s w e h a v e b o t h t h e o r e t ic a l s t u d -
i es a n d a p p l i c a t i o n s o f f u n c t io n a l p r o c e d u r a l l o g ic a l a n d p a r a l l el p r o g r a m m i n g
l a n g u a g e s . E v e n a m o n g t h e p a r a ll e l l a n g u a g e s t h e r e i s a g r e a t v a r i a t i o n b e t w e e n
t h o s e b a s e d o n s y n c h r o n o u s o r a s y n c h r o n o u s c o n tr o l s h a r e d s t o r e o r d i s t r i b u t e d
m e s s a g e p a s s i n g u n t i m e d o r w i t h t i m i n g o f v a r i o u s k i n ds ; e v en h a r d w a r e a n d
so f tware h av e d i f f e r en t m o d e l s .
S p e c i a l i s a ti o n i n v o l v es a d e e p c o m m i t m e n t t o a n a r r o w s e l ec t io n o f p r e se n -
t a t i o n r e a s o n in g m e t h o d s p a r a d i g m l a n g u a g e a n d a p p l i c a t i o n a r e a o r e v e n a
p a r t i c u l a r a p p l i c a t io n . T h e w h o l e p o i n t o f t h e s p e c i a l is a t io n i n f o r m a l m e t h o d s
i s t o r e s t r i c t t h e n o t a t i o n a l f r a m e w o r k a s fa r a s n e c e s s a ry t o a c h ie v e s o m e fo r -
m a l g oa l b u t n e v e r t h e le s s t o sh o w t h a t t h e r e s t ri c t io n s d o n o t p r e v e n t s u c c e s sf u l
8/18/2019 How Did Software Get So Reliable Without Proof?
16/17
16
application to a surpris ingly wide range of problems. This is the reason why spe-
cialist research into formal methods can run the risk of being very divisive. An
individual researcher or even a whole community of researchers becomes wholly
commi tted to a particular selection of specialisations along each of the axes: say
an operational or an algebraic presentation of semantics bisimulation or term
rewriting as a proof method CCS or OBJ as a design notation. The att rac tion of
such a choice can be well illustrated in certain applications such as the analysis
of the alternating bit protocol or the definition of the stack as an abs tract da ta
type. The perfectly proper challenge of the research is to push outwards as far as
possible the frontiers of the convenient application of the particular chosen for-
malism. But that is also the danger: the rush to colonise as much of the available
territo ry can lead to imperialist claims that deny to other specialisms their right
to existence. Any suggestion of variation of standard dogma is treated as akin
to treason. This tendency can be reinforced by the short-sightedness of funding
agencies~ which encourage exaggerated claims to the universal superiority of a
single notation and technique.
The consequences of the fragmentation of research into rival schools is in-
evitable: the theorists become more and more isolated both from each other
and from the world of practice where one thing is absolutely certain: tha t there
is no single cure for all diseases. There is no single theory for all stages of the
development of the software or for all components even of a single application
program. Ideas concepts methods and calculations will have to be drawn from
a wide range of theories and they are going to have to work together consis-
tently with no risk of misunderstanding inconsistency or error creeping in at the
interfaces. One effective way to break formal barriers is for the best theorists to
migrate regularly between the research schools in the hope that results obtained
in one research specialisation can be made useful in a manner acceptable by the
other. The interworking of theories and paradigms can also be explored from the
practical end by means of the case study chosen as a simplified version of some
typical application. In my view a case study th at constructs a link between two
or more theories used for different purposes at different levels of abstraction
will be more valuable than one which merely presents a single formalisat ion in
the hope th at its merits compared with rival formalisations will be obvious.
They usually are but unfortunately only to the author.
Since theories will have to be unified in application the best help tha t ad-
vanced research can give is to unify them in theory first. Fortunately unification
is something that theoretical research is very good at and the way has been
shown again and again in both science and mathematics. Examples from science
include the discovery of the atomic theory of mat ter as a unified framework for
all the varied elements and components of chemistry; similarly the gravi tational
field assimilates the movement of the planets in the sky and cannon balls on
earth. In mathematics we see how topology unifies the study of continu ity in
all the forms encountered in geometry and analysis how logic explains the valid
methods of reasoning in all branches of mathemat ics. I would suggest the current
streng th of individual specialisation in theoretical computing science should be
8/18/2019 How Did Software Get So Reliable Without Proof?
17/17
17
balanced by a commitment from the best and most experienced researchers to
provide a framework in which all the specialisations can be seen as just aspects
or variations of the same basic ideas. Then it will be clear how both existing
and new specialisations are all equally worthy of effort to deepen the theory or
broaden its application. But the aim is no longer to expand and colonise the
whole space but rather to find the natural boundaries at which one theory can
comfortably coexist and cooperate with its neighbours. Closing a gap between
one theory and another is just as important as closing the gap between theory
and practice; and just as challenging.
A c k n ow l e d gm e n t s
I am very grateful to many programmers and managers working in industry who
have made available to me the benefits of their judgment and long experience.
In particular I would like to praise the leading practitioners of the sta te of
the a rt in IBM at Hursley in BNK at Maidenhead and in Digital at Nashua.
Many contributions to my thinking are due to members of IFIP WG2.3 on
Programming Methodology and to its chairman Cliff Jones who made useful
suggestions to the previous draft of the paper. Finally thanks to those named
in the paper with apologies for lack of more formal reference.
Top Related