Practical C++ Generative Programming

Practical Generative Programming

Schalk W. Cronjé

[email protected]

Even in this new millennium, many engineers will still build components that have very little reuse potential due to the inflexible way that they were constructed.

This leads to excessive time required to adapt a component for usage in another system.

Welcome to the world of

Generative Programming

Themes

• GP 101• Building a team• Building a single system• Technical footwork• Building multiple systems• Integration & maintenance

Definition

It is a software engineering paradigm where the aim is to automatically manufacture highly customised and optimised intermediate or end-products from elementary, reusable componentsby means of configuration knowledge.

Automatic programming

• Generative programming is not automatic programming.

• AP aims for highest level of automation• GP acknowledges that there are potentialy

different levels of automation possible in a complete system.

• AP usually involves some form of AI and high amounts of domain knowledge.

• GP provides practical leverage of state-of-the art software engineering practices.

Elements of Generative Programming

Problem space Solution space

ConfigurationKnowledge

•Illegal feature combinations

•Default settings & dependencies

•Construction rules•Optimisations

•Configured Components

•Domain-specific concepts

•Features

Benefits

• Economies of scope● Less time and effort to produce variety of products

• Software quality improvement ● Reuse of proven components

• Scalability● Can be applied to parts of a system or to whole

systems

• Optimisation at domain level● Maximal combinability● Minimal redundancy● Maximum reuse

Steps

• Domain scoping• Feature & concept modelling• Common architecture design and

implementation technology identification• Domain-specific notations• Specify configuration knowledge (metadata)• Implement generic components• Apply configuration knowledge using generators

There is no specific order to these steps !

Configuration Knowledge vs Metadata• Configuration knowledge is the term preferred

by Czarnecki & Eisenecker• Configuration knowledge can be considered the

holistic encapsulation of all knowledge related to building all variants

• Metadata is probably a more codified form of configuration knowledge.

• Some people find the term metadata easier to grasp and less confusing than configuration knowledge

• The rest of this presentation uses the term metadata

Introducing GP into theProcess

• Start small !● Use a pilot project or a small subset of an existing

system● Experiential learning is important – learn to learn.

• Take an iterative and incremental approach to the different GP steps.

• Don't worry too much about modelling up-front.

Building a Team

• Domain experts● Require the ability to codify configuration knowledge

into a reusable form.• Language experts

● Converting configuration knowledge in to programming language representation.

• Guiding light● One person whom understands GP well, ensures that

the development process stays on track and not descend into maintenance hell.

Strategies for C++

• Templates are the C++ way to generic programming

• Develop elementary components as generic components● Fully testable outside of the intended product

configuration

• Configure these components using generated traits / policy classes

• Aim for zero cyclomatic-complexity in the generated classes

Template Metaprogramming

• MPL is a key technology to build generic components● Best example is Boost C++ MPL

• MPL has been suggested as a domain-specific language● Metadata difficult to review to someone not familiar

with MPL• MPL should rather be used as implementation

strategy

Example #1: Configuration system

Name: NetworkPortDescription: Unrestricted port on which a

service can be startedType: uint16Minimum Value: 1024Maximum Value: 65535


template <typename CfgAttr>typename CfgAttr::value_typeget_config();

std::cout << “The network port we'll use is “ << get_config<NetworkPort>();

Example #1: A traits class

struct NetworkPort{ typedef uint16_t value_type; static const value_type const_min = 1024; static const value_type const_max = 65535;

// ... rest to follow};

Example #1: Alternative traits

Because other non-integral types cannot be initialised inline, it might be more practical to use the following alternative.

struct NetworkPort{ typedef uint16_t value_type; static value_type min_value() {return 1024;} static value_type max_value() {return 65535;}

// ... rest to follow};

Example #1: Basic generic function

std::string get_cfg_string( const char* name ); template <typename CfgAttr>typename CfgAttr::value_typeget_config(){ // Calls a basic configuration interface function std::string tmp=get_cfg_string( CfgAttr::name() );

// Converts to appropriate type, throws exception // on conversion failure return boost::lexical_cast<typename CfgAttr::value_type>(tmp);}

Introducing run-time safety

• In order to protect the system against external invalid data we need to add boundary checks.● Use min_values(), max_value() from traits● Add a default_value() to handle missing data

• Additional features could include:● Throwing an exception, instead of defaulting, when

data is missing.

Example #1: Extending the function

template <typename CfgAttr>typename CfgAttr::value_typeget_config(){ std::string tmp=get_cfg_string( CfgAttr::name() ); if(tmp.empty()) return CfgAttr::default_value(); else { typedef typename CfgAttr::value_type vtype; vtype ret= boost::lexical_cast<vtype>(tmp); return CfgAttr::bounded(ret); }}

Example #1: Updated traits

struct NetworkPort{ typedef uint16_t value_type; static value_type min_value() {return 1024;} static value_type max_value() {return 65535;} static value_type default_value {return 4321;}

static value_type& bounded(value_type& v_) { return v_=std::max(min_value(),std::min (v_,max_value())); }};

Capturing Configuration Knowledge

• Various methods have been used for codifying metadata● Text files● Graphical Tools● CASE Tools

• XML is a very convenient form for new projects ● Semi-human readable● Text – Unrestricted source-control● Easy to transform to other formats

• Includes non-code artefacts

● Custom editor can be created in Python or Java


<ConfigSystem> <Attr name=”NetworkPort” adt=”uint16”> <Description>Unrestricted port on which a

service can be started</Description> <Min>1024</Min> <Max>65535</Max> <Default>4321</Default> </Attr></ConfigSystem>

Prefer ADTs

• Use abstract data types (ADTs)• Use a XML lookup table to go from ADT to C++

type• Underlying C++ representation can be changed

without changing any of the metadata

Example #1: Simple Generator

<xsl:template match="Attr">struct <xsl:value-of select="@name"/>{ typedef <xsl:apply-template select="." mode="adt"/>

value_type; static const char* name() {return "<xsl:value-

of select="@name"/>";} static value_type min_value() {return <xsl:value-of

select="Min/text()"/>;} static value_type max_value() {return <xsl:value-of

select="Max/text()"/>;} static value_type default_value() {return

<xsl:value-of select="Default/text()"/>;}};</xsl:template>

ADT Lookup Table

<Types> <Type adt=”uint16” posix-type=”uint16_t” win32-

type=”WORD” embedded-type=”unsigned short” quoted=”no”/>

<Type adt=”string” posixtype=”std::string” win32-type=”std::string” embedded-type=”MyFixedString” quoted=”yes”/>

</Types>

Example #2

• Logging is an aspect of most systems that crosscuts the architecture.

• There might be many requirements in your system, on how logging and reporting is used.● Loggable entities● Levels of logging● User display issues● Localisation

• From a C++ point-of-view one important feature is how logging is generated at logging points

• Using a GP approach it is possible to introduce compile-time validation

Example #2: Legacy Logging

#define MINOR_FAILURE 1#define MAJOR_PROBLEM 2#define GENERAL_PANIC 3

void log_it( int id, const char* text );

// and then some smartie comes alonglog_it( MINOR_PROBLEM|GENERAL_PANIC, ”Voila!! An unsupported error”);

Example #2: Logging Metadata

<Logging> <Report id=”1” name=”MINOR_FAILURE”> <Text>The projector's bulb needs replacing</Text> </Report> <Report id=”2” name=”MAJOR_PROBLEM”> <Text>We're out of Belgium beer</Text> </Report> <Report id=”3” name=”GENERAL_PANIC”> <Text>Elvis has left the building</Text> </Report></Logging>

Example #2: Logging Function

template <typename Report>void log_it( Report const&, const char* text );

log_it( 3,”My code” ); // compile error

log_it( MAJOR_PROBLEM, “Out of German beer too” );

log_it( MINOR_FAILURE|MAJOR_PROBLEM, “No way” ); // Compile error

Example #2: Logging ID Class

// Define typeclass Reportable{ public: Reportable( unsigned id_ ); unsigned id() const;};

// then do either, initialising MINOR_FAILURE in .cppextern const Reportable MINOR_FAILURE;

// ornamespace { const Reportable MINOR_FAILURE =

Reportable(1); }

Preventing Code-bloat

• Only instantiate what is needed● For constant objects this is very easy using the MPL-

value idiom

• Due to ways some linkers work, concrete code might be included in a final link even if the code is not used, therefore only generate what is needed● Control the config elements available to a specific

system from metadata● Only generate the appropriate traits classes

• Cleanly separate common concrete class into a mixin class

The MPL-value idiom

template <int V>class A{ public: static const A<V> value; private: A();};

template <int V>static const A<V> A<V>::value;

Logging Reworked

template <unsigned id_>class Reportable{ public: unsigned id() const {return id_;} static const Reportable<id_> value;};const Reportable<id_> Reportable<id_>::value;

typedef Reportable<1> MINOR_FAILURE;

log_it( MINOR_PROBLEM::value,”Only instantiated when used”);

Adding logging actions

• A user might want to specify that some reports can have certain associated actions.

• For the logging example we might have ● GO_BUY● MAKE_ANNOUNCEMENT● CALL_SECURITY.

• As this is configuration knowledge we can add this to the metadata and then generate appropriate metacode.

Example #2: Logging Metadata

<Logging> <Report id=”1” name=”MINOR_FAILURE”> <Action>GO_BUY</Action> </Report> <Report id=”2” name=”MAJOR_PROBLEM”> <Action>GO_BUY</Action> <Action>MAKE_ANNOUNCEMENT</Action> </Report> <Report id=”3” name=”GENERAL_PANIC”> <Action>CALL_SECURITY</Action> <Action>MAKE_ANNOUNCEMENT</Action> </Report></Logging>

Using MPL as glue

template <unsigned id_,typename actions_list>class Reportable{ public: unsigned id() const {return id_;} static const Reportable<id_> value;

typedef actions_list valid_actions;};// Generated codetypedef Reportable<2, boost::mpl::vector<GO_BUY,MAKE_ANNOUNCEMENT> > MAJOR_PROBLEM;

Using SFINAE as validator

template <typename Report,typename Action>void log_it( const char* text, boost::enable_if< boost::mpl::contains< typename Report::valid_actions, Action >::type::value >*_= 0);

// Fails to compilelog_it<GENERAL_PROBLEM>( GO_BUY,”Bought Elvis beer”);

// OK,log_it<MAJOR_PROBLEM>( GO_BUY, ”Imported some Hoegaarden”);

Multiple Systems

• Examples until now have shown the GP steps for a configuration system and a logging system.

• The next step is to apply these to three systems:● System 1 uses XML files for configuration and sends

logs to syslog.● System2 uses INI files, and sends logs to NT Evlog● System 3 keeps configuration in a binary format (read-

only) and sends logs via SNMP.

Example Product Metadata

<Products> <System id=”1”> <Config type=”xml”/> <Logging type=”syslog”/> <Functionality> ... <Functionality> </System> <System id=”2”> <Config type=”ini”/> <Logging type=”ntevlog”/> <Functionality> ... <Functionality> </System></Products>

Building Multiple Systems

• Four generators can be applied to this product metadata.● Two of them we have already seen● These will generate configurations and logging aspects

• Another generator looks at logging and configurations and adds the appropriate subsystems.

• A fourth generator looks at the functionality and loads up all of the functional classes for the system● A creative exercise for the reader …

Testing

• Can tests be generated?● There have been various argument around this topic.

• Validate metadata independently

• Test data can be generated from metadata

• DO NOT generate unit tests to validate that the generated code is correct!● How can you verify that the generated tests is correct?

Maintenance

• Long-tem maintenance requires upfront investment in building quality generic components

• Metadata might have to refactored into smaller XML files or a tools should be developed to edit the metadata.

• Refactoring components into more generic components over time means that you will be able to configure and customise products even more.

Integration

• Many modern systems are multi-language / multi-platform

• These techniques extend easily into other programming languages / development environments

• The same configuration knowledge remains the driver

• Localisation data can be generated in various formats

• Parts of technical documents can also be generated.

Further Reading

• www.program-transformation.org

• www.generative-programming.org

• www.research.microsoft.com

http://www.generative-programming.org/

In this new millennium, engineers can build high-quality proven generic components with high reuse potential and adaptively configure them.

This leads to decreased time required to use a component in another domain and an increase in the variety of products that can be assembled.

Did anyone mention competitive edge?

Practical C++ Generative Programming

Technology

Transcript of Practical C++ Generative Programming