From Legacy to State-of-the-art; Architectural Refactoring · code and complexity, no added value...
Transcript of From Legacy to State-of-the-art; Architectural Refactoring · code and complexity, no added value...
From Legacy to State-of-the-art; ArchitecturalRefactoring
-
TV domain HW Computin
g HW
TV domain platform
TV computing
Infra- structure
TV applications
TV Hybrid TV Digital TV
"All-in-one" combi TV
Set Top Box domain HW
Set Top Box Platform
M H P
Set Top Box
functions 3 rd party stack(s)
Computing HW TV domain HW
Computin g HW
TV domain platform
TV computing
Infra- structure
TV applications
Digital Video Platform SW
Set Top Box domain HW
M H P
Set Top Box
functions 3 rd party stack(s)
Computing HW TV domain HW
TV domain platform
TV computing
Infra- structure
TV applications
Digital Video Platform SW
Set Top Box Platform
Digital TV UI
Set Top Box domain HW
TV domain HW Computing HW
Digital Video Platform SW
M H P
Set Top Box functions
TV domain platform
TV computing
Infra- structure
Set Top Box Platform
3 rd party stack(s)
TV applications
storage domain HW
Storage applications
Digital TV UI
Storage srvices
PDP DVR
DVD RW
Set top
elec. Digital Cable
Gate way
ADSL TV2
re- ceiver
portable media- screen
trans mitter
game console
Gerrit MullerUniversity of South-Eastern Norway-NISE
Hasbergsvei 36 P.O. Box 235, NO-3603 Kongsberg Norway
Abstract
The market of electronic appliances shows a fast increasing diversity. Manufac-turers must be able to combine existing functions and new applications in a shorttime frame. A large amount of accumulated SW code (legacy) has to be reused innew ways.The architecture(s) must be adapted to these new ways of working. Revolu-tionary adaptations have proven to be extremely risky. Opportunistic extensionand integration decrease the quality of the code base, making it increasingly moredifficult to continue. Architectural refactoring is a feedback based method to evolvean architecture.
DistributionThis article or presentation is written as part of the Gaudí project. The Gaudí project philosophy is to improveby obtaining frequent feedback. Frequent feedback is pursued by an open creation process. This document ispublished as intermediate or nearly mature version to get feedback. Further distribution is allowed as long as thedocument remains complete and unchanged.
All Gaudí documents are available at:http://www.gaudisite.nl/
version: 1.3 status: finished September 9, 2018
1 The problem
1.1 Market trends
Consumer Electronics Products are a large variety of products, which have evolvedfrom straightforward electronic devices, such as radios, into complex softwareintensive systems. Figure 1 shows a typical set of today’s audio and video products.
DVD
VCR
Audio Mini
Audio
Audio Portable
CD man
TV
Walkman Hard Disk
From
: CO
PA
tuto
rial,
Rob
van
Om
mer
ing
Figure 1: Today’s Audio Video Consumer Products
Technological advances and business opportunities result in a convergence ofseparate worlds. The worlds of telecommunications, computers and consumerelectronics are converging, see figure 2.
Telecom
Consumer
Computer
Figure 2: Trend: Convergence of separate worlds
This convergence means that functions from the different domains are integratedin new types of appliances. These appliances are optimized towards the intendeduse. User, form factor, function and environment all together determine what anoptimal appliance looks like. The wide variety in users, form factors, functions andenvironments requires a very rich variety of appliances.
Figure 3 shows at the left hand side a small subset of existing devices belongingin one of the three domains. More to the right some of the form factors are shown,
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 1
while the right hand side shows some of the environments. The number of usefulcombinations of functions, form factors and environments is nearly infinite!
mp3
dvd
set top box
flat display
pen
speech
cable
modem
firewall
Ambient Intelligence
living room
car
car navigation
pda
surveillance
camera
camera
GSM phone
computerCommunicator
television
games
sailboat
audio
microset
headphone
garment
watch
Figure 3: Integration and Diversity
In this presentation video entertainment will be used as the application area.Figure 4 shows a typical diagram of the set up of video products in our homes.We see products to connect with the outside world (set top box), storage products(Video Cassette Recorder abbreviated as VCR), and conventional TV’s and remotecontrols, which are the de facto user interface.
Conventional TV
Set Top Box
Video Recorder
DVD Player
Cable Modem
PC
Figure 4: Today’s Video Products
This chain of video products is slowly evolving, as depicted in figure 5. Inthe past a straightforward analog chain was used. The elements in this chain arestepwise changed into digital elements. The introduction of the Large Flat TV’sbreaks open the old paradigm of an integrated TV, which integrates tuner andrelated electronics with the monitor function.
In the near future many more changes can be expected, such as the introductionof the Digital Video Recorder (DVR), a gateway to alternative broadband solutions,
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 2
wireless inputs and outputs, home networks enabling multiple TV’s.
TV VCR
DVD
Set top
TV VCR
DVD
PDP VCR
DVD
Set top
elec.
Digital Cable
Digital Cable
Analog Cable
PDP DVR
DVD RW
Set top
elec. Digital Cable
Gate way
ADSL TV2
Multiple inputs Multiple stores Multiple displays Multiple applications Multiple formfactors
1
2
3
4 re-
ceiver
portable media- screen
trans mitter
game console
Figure 5: Evolution of Video Products
The function allocation for this last stage of figure 5 and the network topologycan be solved in several ways. Figure 6 shows four alternatives, based on a client-server idiom.
The expectation is that all alternatives will materialize, where the consumerchooses a solution which fits his needs and environment.
Network
Thin Clients
All-in-one Server
All-in-one Combi's
Thin Servers
Clients
Network
Network
B A C
Network
Server
Server
Server
Client
Client
Client
D "All-in-one"
Combi's "Thin Servers"
"All-in-one" Home server
"Modular"
Figure 6: Distribution Scenario’s
These alternatives require different packaging of functions into products, asshown in figure 7.
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 3
TV2
PDP DVR
DVD RW
Set top
elec. Digital Cable
Gate way
ADSL TV2
4A re-
ceiver
portable media- screen
trans mitter
game console
PDP DVR
DVD RW
Set top
elec. Digital Cable
Gate way
ADSL TV2
4B re-
ceiver
portable media- screen
trans mitter
game console
PDP DVR
DVD RW
Set top
elec. Digital Cable
Gate way
ADSL
4D re-
ceiver
portable media- screen
trans mitter
game console
PDP DVR
DVD RW
Set top
elec. Digital Cable
Gate way
ADSL TV2
4C re-
ceiver
portable media- screen
trans mitter
game console
"All-in-one" Digital TV
"Thin Servers" "All-in-one" Home server
"Modular"
TV2
Figure 7: Product Packaging Options
1.2 Technology trends
The major trend for electronic devices is Moore’s law, roughly stating that theavailable amount of transistors in an integrated circuit doubles every 18 months.Devices based on IC’s follow this trend. Figure 8 shows the growth of the amountof memory in TV’s.
1965 1979
2000 1990
1 kB
64 kB2 MB
Moore's law
Fro
m: C
OP
A tu
toria
l, R
ob
va
n O
mm
erin
g
Figure 8: Moore’s law
The amount of software in products, measured as lines of code, more or lessfollows Moore’s law. Unfortunately the software engineering discipline did notproceed at the same rate, which is reflected by a fault metric expressed as fault perthousand lines of code, varying between 1 for very rigid organized producers, to
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 4
10 or more for ad hoc products. Typical values for CE type products is 3 faults per1000 lines of code. See figure 9 for typical values as function of the year.
1000
100
10
man
year
s p
er
prod
uct
1990 1995 2000 2005
1000
10000
typi
cal a
mou
nt o
f er
rors
per
pro
duct
Figure 9: Problem: increasing SW size, decreasing reliability?
The increase of the amount of software causes many problems:
• Increase of development cost
• (Non) availability of skilled engineers
• Increase of development time, and hence time to market
• Decrease of product reliability
REUSE
time
$$
Prom
ise
Reality
Figure 10: The Holy Grail: Reuse
Reuse is often presented as the solution for all problems mentioned above.Experience learns that quite the opposite happens in many cases, see [2], thechallenge of executing a successful reuse program is often severely underestimated.
The most common root-cause of reuse failure is the mistake to see reuse as agoal rather than a means.
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 5
1.3 Example Digital Television
An example of a new product is a digital television, which is the merger of a settop box and a television. This television can directly connect to a digital cableinfrastructure and offer services provided by cable or content providers.
One way of realizing such a system is to declare the reuse of existing software,integrate all this software on a single hardware platform and to support this hardwareplatform factor out the “lower” software layers. Figure 11 shows the simplisticarchitecting to achieve this merger.
Set Top Box domain HW
Set Top Box Platform
MHP
Set Top Box
functions 3 rd party stack(s)
Computing HW
TV domain HW Computing
HW
TV domain platform
TV computing
Infra- structure
TV applications
Digital Video Platform SW
Set Top Box domain HW
TV domain HW Computing HW
analog TV Set top box
Digital Video Platform
Set Top Box domain HW
TV domain HW Computing HW
Digital Video Platform SW
Set Top Box Platform
MHP
Set Top Box
functions 3 rd party stack(s)
TV domain platform
TV computing
Infra- structure
TV applications
Digital TV UI
Digital TV
Merge
glue
Figure 11: Simplistic Architecting: Digital TV
Figure 12 shows the rationale behind the reuse of existing software packages.The cumulative effort of the software involved exceeds 500 manyears.
>100 Myr
>100 Myr
>200 Myr >100 Myr
Set Top Box domain HW
TV domain HW Computing HW
Digital Video Platform SW
Set Top Box Platform
MHP
Set Top Box
functions 3 rd party stack(s)
TV domain platform
TV computing
Infra- structure
TV applications
Digital TV UI
"Legacy" code > 500 Myr
glue
Figure 12: Available Code Assets
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 6
2 Architectural Refactoring
Combining existing software packages is mostly difficult due to “architecturalmismatches”. Different design approaches with respect to exception handling,resource management, control hierarchy, configuration management et cetera, whichprohibit straightforward merging. The solution is adding lots of code, in the formof wrappers, translators and so on, while this additional code adds complexity, itdoes not add any end-user value.
Performance and resource usage are most often far from optimal after a merger.Amazingly many people start worrying about duplication of functionality when
merging, while this is the least of a problem in practice. This concern is the causeof reuse initiatives, which address the wrong (non-existing) problem: duplication,while the serious architectural problems are not addressed.
tuner tuner MPEG MPEG
Duplication
Architectural mismatch : wrappers, translators, conflicting controls
Poor performance; additional resource usage
additional code and complexity,
no added value
UI UI
non problem Problems Architecture Reuse
Figure 13: Merge problems
The proposed solution to this set of problems is architectural refactoring.Architectural refactoring is an incremental approach, putting a lot of emphasis onfeedback. Two major criteria to get feedback on are:
• How well does the current architecture support today’s product needs?
• How well will the architecture evolve to follow the market dynamics?
In every increment to the market both concerns should be addresses, which trans-lates in clear business goals (product, functions, value proposition) and clear refac-toring goals fitting in a limited investment. The refactoring goals should be basedon a longer term architecture vision, see 14.
Examples of Refactoring goals can be seen in figure 15. These refactoringgoals should be sufficiently “SMART” to be used as feedback criterium.
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 7
Refactoring
within short term business goals
with limited but substantial refactoring goals
clear product clear value proposition
feedback on direction
limited investment based on long term architecture vision
Figure 14: Solution: Architectural Refactoring
Note: many refactoring projects spend lots of effort, while critical review after-wards does not show any improvement. Often loss of goal or focus is the basis forsuch a disaster.
+ Decrease Code Size
+ Decrease Resource Usage * power * memory * silicon area
+ Increase Performance * response time * throughput
Qua
lity
time
0%
10%
20%
Improvement investment as percentage of total budget
+ Increase quality * decrease fault density
Figure 15: Example of Refactoring Goals
Architectural refactoring looks at all architectural aspects, from functions andstructure to selection of mechanisms and technologies. Code refactoring, wellknown from extreme programming [1], plays a role at a much more microscopiclevel. See figure 16 which shows both ways of refactoring side by side. Some coderefactoring requires an update of the architecture. At the other hand architecturalchanges quite often have a significant software impact.
2.1 Prerequisites for effective architectural refactoring
Frequent feedbackUnderstanding of the problem as well as the solution is key to being effective.
Learning via feedback is a quick way of building up this understanding. Waterfallmethods all suffer from late feedback, see figure 17 for a visualization of theinfluence of feedback frequency on project elapsed time.
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 8
optimize() ...
add(a) ...
remove() ...
move() ...
accelarate() ...
add(a) ...
remove() ...
optimize() ...
move() ...
accelarate() ... add(a)
... remove()
...
Code Refactoring
old
new
old
new
new
Architectural Refactoring Function, Structure, Rationale
Mechanisms, Technologies
return( error )
free, alloc
raise( exception )
garbage collection
old
Figure 16: Architectural and Code refactoring
Awareness of dynamicsThe world is highly dynamic, the markets and applications change rapidly,
while the famous law of Moore shows the incredible speed of technological devel-opments. Unfortunately most people believe in stability and are biased towardsstabilizing architectures. Architectures and their implementations are sandwichedbetween the fast moving market at one side and technology improvements at theother side. Since both sides change quite rapidly, the architecture and its imple-mentation will have to change in response, see figure 18.
The evolution of a platform is illustrated in figure 19 by showing the change inthe Easyvision [4] platform in the period 1991-1996. It is clearly visible that everygeneration doubles the amount of code, while at the same time half of the existingcode base is touched by changes.
Long Term VisionIn order to set refactoring goals it is useful to have a long term vision on the
architecture. Such a long term vision may be quite ambitious. The ambition of thevision will be balanced by the pragmatics of short term business goals and limitedinvestments in improvement.
Figure 20 shows an example of a long term vision, where a framework isforeseen, which decouples 6 design and implementation concerns:
• applications
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 9
3 months
25 months
2 months
12 months
1 month
8 months
Start Start Start
Target Target Target
stepsize:
elapsed time
Small feedback cycles result in Faster Time to Market
Figure 17: Frequent feedback results in faster results and a shorter path to the result
• services
• personalization
• configuration
• computing infrastructure
• domain infrastructure
The actual implementation will not have such a level of decoupling for a long time,the penalty in effort, resource usage and many other aspects will be prohibitivefor a long time. Nevertheless the decoupling will become crucial if the variety ofproducts is really very large and dynamic.
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 10
Architecture
Platform
Dynamic Market
Fast changing Technology
How stable
is a platform
or an architecture?
Components
Figure 18: Myth: Platforms are Stable
1991
1992
1994
1991
1994
Last changed in:
Growth
Change
3rd
generation components are mature, active maintenance needed.
Growth and change continues, some "old" components become obsolete
1992
1996
Figure 19: Platform Evolution (Easyvision 1991-1996)
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 11
Computing Infrastructure
Domain Infrastructure
Services Applications
Config
urati
on
i.e. In
terna
tiona
lizati
on
perso
naliz
ation
i.e. tu
nes,
themes
Framework
Long Term Vision: Reference Architecture + Sample implementation of Framework and Components
Reference Architecture
Figure 20: Example Long Term Vision
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 12
3 Conclusion
Figure 21 shows how not to work towards the future:
• Don’t merge blindly
• Don’t a priori declare SW to be reusable
PDP DVR
DVD RW
Set top elec. Digital
Cable
Gate way ADSL TV2
re- ceiver
portable media- screen
trans mitter
game console
PDP
DVR
DVD RW
Set top
elec.
Gate way
re- ceiver
trans mitter
Digital TV
Opportunistic Legacy
Integration
Proclaimed reuse
Set Top Box domain HW TV domain HW Computing HW
Digital Video Platform SW
Set Top Box Platform
MHP
Set Top Box
functions
3 rd
party stack(s)
TV domain platform
TV computing
Infra- structure
TV applications
Digital TV UI
glue
Figure 21: Don’t do
While figure 22 illustrates architectural refactoring, applied on the example ofa digital television. The steps taken here are:
From TV to Hybrid TV. The conventional TV is refactored to use a more modernHW platform, while the lower layer is factored out. The set top box is physicallyintegrated in the television, while at software level both applications are pragmati-cally interfaced.
From Hybrid TV to Digital TV. More hardware is shared between the TV partand the set top box part of the system, with as refactoring goals: reduction ofresource usage and enabling a more harmonized user interface. The set top boxplatform is redesigned to make this possible.
From Digital TV to “All-in-one” TV. The TV computing infrastructure is simplified(reduce lines of count), while the next “legacy” application is merged in: storage.
4 Acknowledgements
Lex Heerink patiently listened to the presentation and provided valuable feedback.
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 13
TV domain HW Computin
g HW
TV domain platform
TV computing
Infra- structure
TV applications
TV Hybrid TV Digital TV
"All-in-one" combi TV
Set Top Box domain HW
Set Top Box Platform
M H P
Set Top Box
functions 3 rd party stack(s)
Computing HW TV domain HW
Computin g HW
TV domain platform
TV computing
Infra- structure
TV applications
Digital Video Platform SW
Set Top Box domain HW
M H P
Set Top Box
functions 3 rd party stack(s)
Computing HW TV domain HW
TV domain platform
TV computing
Infra- structure
TV applications
Digital Video Platform SW
Set Top Box Platform
Digital TV UI
Set Top Box domain HW
TV domain HW Computing HW
Digital Video Platform SW
M H P
Set Top Box functions
TV domain platform
TV computing
Infra- structure
Set Top Box Platform
3 rd party stack(s)
TV applications
storage domain HW
Storage applications
Digital TV UI
Storage srvices
PDP DVR
DVD RW
Set top
elec. Digital Cable
Gate way
ADSL TV2
re- ceiver
portable media- screen
trans mitter
game console
Figure 22: Conclusion: Refactoring the Architecture is a must
References
[1] Kent Beck. Extreme Programming Explained: Embrace Change. Addison-Wesley, Reading, MA, 2000.
[2] Gerrit Muller. Product families and generic aspects. http://www.gaudisite.nl/GenericDevelopmentsPaper.pdf, 1999.
[3] Gerrit Muller. The system architecture homepage. http://www.gaudisite.nl/index.html, 1999.
[4] Gerrit Muller. Case study: Medical imaging; from toolbox to product toplatform. http://www.gaudisite.nl/MedicalImagingPaper.pdf, 2000.
HistoryVersion: 1.3, date: June 13, 2002 changed by: Gerrit Muller
• minor changeVersion: 1.2, date: September 12, 2001 changed by: Gerrit Muller
Gerrit MullerFrom Legacy to State-of-the-art; Architectural RefactoringSeptember 9, 2018 version: 1.3
University of South-Eastern Norway-NISE
page: 14