Generative Programming In The Large - Applied C++ meta-programming

Practical Generative Programming

Generative Programming in the Large
Applied meta-programming

Schalk W. [email protected]

Even in this new millennium, many engineers will still build components that have very little reuse potential due to the inflexible way that they were constructed.

This leads to excessive time required to adapt a component for usage in another system.

The world of modern

C++ Generative Programming

is a vastly untapped field

(we are just scraping the surface)

Definition

It is a software engineering paradigm where the aim is to automatically manufacture highly customised and optimised intermediate or end-products from elementary, reusable components by means of configuration knowledge.

Implies Code GenerationDomain-specific languageGeneric Techniques

Elements of Generative Programming

Problem space

Solution space

ConfigurationKnowledge

Illegal feature combinationsDefault settings & dependenciesConstruction rulesOptimisations

Configured Components

Domain-specific conceptsFeatures

Generic Components

This is a very high level conceptual definition. Most engineers will be more concerned with the techniques for applying configuration knowledge to get to the solution

Key Code-level Strategy

For effective implementation there is one basic principle that encompasses most GP strategies:

Dijkstra's Separation of Concerns

This principle accepts the fact that we cannot deal with many issues at once and that important issues should be addressed with purpose.

Key Code-level Strategies

Develop elementary components as generic componentsFully testable outside of the intended product configurationConfigure these components using generated artefacts appropriate to the programming languageAim for zero cyclomatic-complexity in the generated artefactsEliminate defects as early as possible

The zero cyclomatic-complexity aim is the ideal. The real purpose is to eliminate any need to have to debug inside the generated code. This also makes a strong argument for the use of generic programming as an implementation strategy, as all the debugging can be done on the generic components even before anything has been generated.

In the large ...

Generative C++ can lead to large number of configuration classes or meta-classes being generated.It is not uncommon to work with 50+ tag classes within a single type-listEffective integration of meta-programming, compile-time paradigms and run-time paradigms are required.Code-bloat must be avoided!

The zero cyclomatic-complexity aim is the ideal. The real purpose is to eliminate any need to have to debug inside the generated code. This also makes a strong argument for the use of generic programming as an implementation strategy, as all the debugging can be done on the generic components even before anything has been generated.

Conventions

For the examples assume that the following namespace aliases exist

using namespace AccuConf;namespace mpl = AccuConf::mpl;namespace bmpl = boost::mpl;

All code are implemented in namespace AccuConf.

Building a rule evaluator
(as an example of a generative library)

Requirement #1:
Solve complex boolean statements

if (cond_A && cond_B || !cond_C)

return outcome_A;

else if (cond_D)

return outcome_B;

else

return outcome_C;

Requirement #2:
Rule-types are defined in a DSL

condition_expr:input: IP-addressoperation: ==

condition_expr:input: IP-addressoperation: in_netmask

condition_expr:input: Email-addressoperation: regex_match

Requirement #3:
Short-circuit

// Jump to evaluate !cond_Cif (cond_A && !cond_A || !cond_C)

// Never evaluate cond_Dif( cond_B || !cond_B || cond_D)

// Don't evalute cond_A twiceif( cond_A || cond_A )if( cond_A && cond_A )

Requirement #4:
Compile-time & run-time rules

// Hard-code compile-timerule1 = cond_A && cond_B

// Loaded at run-timerule2 = load_rule( "ip in FEFE::/64" );

Requirement #5:
Early enforcment

Only allow attribute-operator combinations that have been defined in DSLOnly allow attributes to be passed that were defined in DSL

The conceptual evaluator
Evaluating rules

class Evaluator{public:

/**Evaluates the rules against a supplied set* of attributes* @param A A collected set of attributes that will* be used as input to the rules.* @return The outcome associated with the rule* that triggered*/Outcome const& resolve( Properties const& A) const;

};

It is totally possible to have different types depending on the platform. For instance it might be deemed to have WORD instead of uint16_t on a Win32 platform.

Types, however, do not have to be OS-specific, types can be varied in order to obtain optimisations in specific products.

The conceptual evaluator
Adding rules

class Evaluator{public:

Outcome const& resolve( Properties const& A) const;

/**Adds a rule to the rule set* @param O The outcome to return if this rule * triggers* @param N The order of this rule in the ruleset* @param R the rule to add*/void add_rule( Outcome const& O,Order const& N,Rule const& R);

};

It is totally possible to have different types depending on the platform. For instance it might be deemed to have WORD instead of uint16_t on a Win32 platform.

Types, however, do not have to be OS-specific, types can be varied in order to obtain optimisations in specific products.

Idiom #1: Generative Property Bag

What does the Properties class look like?

Properties class

This can be implemented as a heap-based or a stack-based approach.

Heap-based:

More memory-efficient, only allocate stored attributes. Suffers heap-access penalty. Potential for heap fragmentation.

Stack-based:

Pre-allocate space for all attributes, even if not used.

Heap-based Properties class

Use boost::any to provide generic storageUse boost::map to provide conditional storageIndex using meta-indexes

template class Properties{public:// To be implemented ...

private:typedef std::map< int,boost::any > container_type;container_type m_attrs;};

Generated Property Set

struct prop{struct ip_addr{typedef IPAddress value_type;typedef to-be-decided valid_matches;};

struct email_addr{typedef std::string value_type;typedef to-be-decided valid_matches;};

typedef boost::mpl::vector< ip_addr, email_addr > valid_attrs;};

Setting a Property

template class Properties{public:template void set( typename mpl::property_value::type const&);

private:typedef std::map< int,boost::any > container_type;};

Properties A;A.set< prop::ip_addr >( "192.168.1.1" );

Implementing Property::set

template template void Properties::set(typename mpl::property_value::type const& v_){// Ensure that P is valid !BOOST_MPL_ASSERT_MSG( (bmpl::contains< typename mpl::valid_properties< GeneratedPropSet >::type,P >::value),THIS_IS_NOT_A_VALID_PROPERTY,(P));

// Rest to follow ...}

Static Assertion provides relatively useful compile-time error if a invalid type is supplied


template template void Properties::set(typename mpl::property_value::type const& v_){// Ensure that P is valid !BOOST_MPL_ASSERT_MSG( bmpl::contains< typename mpl::valid_properties< GeneratedPropSet >::type,P >::value,THIS_IS_NOT_A_VALID_PROPERTY,(P));


boost::mpl::contains is a metapredicate that returns TRUEif P is in the list of valid properties


template template void Properties::set(typename mpl::property_value::type const& v_){// Ensure that P is valid !BOOST_MPL_ASSERT_MSG( (bmpl::contains< typename mpl::valid_properties< GeneratedPropSet >::type,P >::value),THIS_IS_NOT_A_VALID_PROPERTY,(P));


mpl::valid_properties is our ownmetaclass to extract the MPLsequence of valid property namesfrom GeneratedPropSet


template template void Properties::set(typename mpl::property_value::type const& v_){BOOST_MPL_ASSERT_MSG( ... );

// Find key for mapconst int pos = mpl::property_pos< GeneratedPropSet, P >::value;


mpl::property_pos is our own metaclass returning the relative position of a type in a typelist. We use this value as a keyinto the map. Due to the MPL_ASSERTwe can be ensured that this is avalid value.


template template void Properties::set(typename mpl::property_value::type const& v_){BOOST_MPL_ASSERT_MSG( ... );

// Find key for mapconst int pos = mpl::property_pos< GeneratedPropSet, P >::value;


Use of 'const int' allows for optimisationat compile-time


template template void Properties::set(typename mpl::property_value::type const& v_){BOOST_MPL_ASSERT_MSG( ... );const int pos = ... ;

// Assign the valuetypedef typename mpl::property_value::type value_type;

m_attrs[pos] = v_;

}

(alternative)

template template void Properties::set(V const& v_){BOOST_MPL_ASSERT_MSG( ... );const int pos = ... ;

// Assign the valuetypedef typename mpl::property_value::type value_type;

m_attrs[pos] = value_type(v_);

}

Getting a Property

template class Properties{public:template typename mpl::property_value::type const& get() const;

private:typedef std::map< int,boost::any > container_type;};

Properties A;A.set< prop::ip_addr >( "192.168.1.1" );std::cout ( );

Implementing Property::get

template template typename mpl::property_value::type const&Properties::get() const{BOOST_MPL_ASSERT_MSG( ... );

// Find key for maptypename container_type::const_iterator itr = m_attrs.find(mpl::property_pos< GeneratedPropSet, P >::value);


Reusing mpl::property_pos we obtainan iterator to the appropiate property.Note the use of MPL_ASSERT to ensure a valid index value



... itr = m_attrs.find( ... );

if( m_attrs.end() == itr )throw range_error("Property not set");


Throw an exception if the property is not set. Use Property::is_set predicate if a no_throw check is needed.(is_set implementation left as an exercise)



... itr = m_attrs.find( ... );

if( m_attrs.end() == itr ) ... ;

typedef typename mpl::property_value::typevalue_type;

return *boost::any_cast(&(itr->second));}

We can assume that the type stored is correct. Need to use the & to invoke anon-copy versionof any_cast.

Idiom #2: GGS

Generic-Generated Separation: Access to generated typelists from generic code should go through metaclasses.

MPL metaclasses

Abstract access to type information through metaclassesThis allows for specialisation when neededImproves long-term maintenance

Obtaining the value_type

template struct property_value{typedef typename Property::value_type type;};

Sequence of Valid Properties

template struct valid_properties{typedef typename Properties::valid_properties type;};

Property Position

template struct valid_properties :bmpl::find< typename valid_properties< Properties>::type,Property>::type::pos{// Prevent invalid PropertyBOOST_MPL_ASSERT_MSG( bmpl::contains< typename valid_properties< Properties>::type,Property >::value,PROPERTY_NOT_FOUND_IN_SEQUENCE,(Property,Properties));

};

Idiom #3: Visit Each
Apply an operation to each generated member

Using Boost for_each

bmpl::for_each( FUNCTOR() );

struct FUNCTOR{template void operator()(P) const;};

For each type in sequence calls the functor


bmpl::for_each( FUNCTOR() );

struct FUNCTOR{template void operator()(P) const;};

Functor must be ableto accommodate eachtype, otherwise compile-error will occur


struct print_ip{Properties const& p;

print_ip(Properties const& p_): p(p_) {}

template void operator()(P) const{}

void operator()(ip_addr) const{std::cout

Rule Class

template class Rule{public:

// C-tors to follow ...

Rule operator || (Rule const&) const;Rule operator && (Rule const&) const;Rule operator !() const;

bool operator()(Properties const&) const;

private:

boost::function< bool(Properties const&) > m_rule;

};

Rule Class

template class Rule{public:

template Rule( Operand const& op_ );

};

Idiom #5: Hekaton Typelist
Working around limitations in existing typelist libraries

Limitations in Boost MPL sequences

Maximum size of template parameter list is implementation-dependantTypically 20GP might require very long lists100 types are not uncommonA special type-list might be needed to convert a very long generated list into a standard Boost MPL sequence.

Hekaton-typelist

template struct typelist{typedef bmpl::vector _list0;typedef bmpl::vector _list1;typedef bmpl::vector _list2;// etc.

// Join liststypedef bmpl::joint_view _join0;typedef bmpl::joint_view _join1;//etc.// until the last join which becomestypedef to-be-defined type;};

Hekaton-typelist (detail)

namespace detail{template struct typelist{

// detail to follow ...

};} // namespace detail


namespace detail{

#define P_LIST(z,n,t) BOOST_PP_COMMA_IF(n) \typename V##n = t

template < BOOST_PP_REPEAT(10,P_LIST,bmpl::vector) >struct typelist{

// detail to follow ...

};#undef P_LIST

} // namespace detail


namespace detail{

#define P_LIST(z,n,t) BOOST_PP_COMMA_IF(n) \typename V##n = t template < BOOST_PP_REPEAT(10,P_LIST,bmpl::vector) >struct typelist{

typedef bmpl::joint_view _join0;typedef bmpl::joint_view _join1;

//etc. until the last join which becomestypedef bmpl::joint_view type;

};#undef P_LIST} // namespace detail

Hekaton-typelist (final)

#define P_LIST1(z,n,t) BOOST_PP_COMMA_IF(n) \typename T##n = t#define P_LIST2(z,n,t) BOOST_PP_COMMA_IF(n) t##n#define P_LIST3(z,n,t) BOOST_PP_COMMA_IF(n) \t < BOOST_PP_REPEAT_FROM_TO( \BOOST_PP_MUL(n,20), \BOOST_PP_ADD(BOOST_PP_MUL(n,20),20), \P_LIST2,T ) \>template < BOOST_PP_REPEAT( 100,P_LIST1,bmpl::na ) >struct typelist : detail::typelist< BOOST_PP_REPEAT( 5,P_LIST3, bmpl::vector ) > ::type{};#undef P_LIST1#undef P_LIST2#undef P_LIST3

Idiom #6: Discriminant Union from Typelist
boost::make_variant_over

Discriminant Union

typedef mpl::valid_properties::typeexample_typelist;

typedef boost::make_variant_over::type example_variant;

X

Discriminant Union

struct extract_value_type{template : mpl::property_valuestruct apply {};};

typedef boost::transform_view::type example_typelist

typedef boost::make_variant_over::type example_variant;

Implementation Specifics

Compiler Limitations

Few compilers can handle the deep level of template instantiation very wellgcc 4.x is fastVC 8 (VS2005) is adequategcc 3.x is slow and has big memory footprintIt is not always possible to use the puristic template solutionUse the Boost Preprocessor LibraryGenerate code of higher levels of cyclomatic-complexity

Further Reading

Touching the void

Pushing C++ skills far beyond the knowledge of many C++ practitionersPolitics of introducing a new technologyTeaches very good software skills outside the C++ domainAm I in the wrong language?(Yeah, thanks Kevlin!)Shows requirements for future languagesPowerful syntaxSimplicity of expression

Klik om die titelteksformaat te redigeer

Klik om die skemateksformaat te redigeerTweede skemavlakDerde skemavlakVierde skemavlakVyfde skemavlakSesde skemavlakSewende skemavlakAgste skemavlakNegende skemavlak

ACCU 2007

Generative Programming In The Large - Applied C++ meta-programming

Documents

Transcript of Generative Programming In The Large - Applied C++ meta-programming