7/24/08Rob Armstrong Creating an HPC application framework from a mere HPC application automatically...

7/24/08Rob Armstrong

Creating an HPC application framework from a mere HPC application automatically

Rob ArmstrongSNL


The problem is that application developers see themselves at the center of the universe...

Explains the “Cactus” phenomenon: domain-specific, special purpose framework that has

a home-grown audience. framework is adapted from a successful application in

that domain. has a “big event loop” architecture that is less flexible.

aside: why is this? has difficulty expanding outside its original domain.

CCA has grown the ability to do this. Onramp->Bocca->Ccaffeine->CCASpec->Babel

... and not framework developers as it rightly should be...


Autogen'ing an HPC framework from an existing monolithic application Identify components and

methods with Onramp. The hard work is still there:

how is this to be decomposed into components?

what is the execution graph for these components?

Does it make sense? Iterate

/*CCA->COMP(Test)->BEGINCCA->PORT(Hello)->BEGIN*/

int Hello1(int x, char *arg){

int i;i=2+3;return i;

}

//CCA->ARG(2,out)int Hello2(int x, char *arg){

int i;i=2+3;return i;

}


Re-create the application using existing tools... Use Appgen to create a main() that marshals

the components. Componentized framework does exactly what

the original did... but now is a plug-able framework presumably more amenable to collaboration is full-up CCA compliant, has a generally expandable

architecture. Resulting framework is, and can remain, an

automated derivative of the original A framework is generated as part of the build process

for the original application.


Workflow for “automatic” framework generation...

CmdLineBuilderViewForHuman::~CmdLineBuilderViewForHuman() { out = NULL; bm = NULL;}

/** Implements ComponentChangedListener. Signal a change in the Component's status. */void CmdLineBuilderViewForHuman::componentChanged(ComponentChangedEvent* evt) { if(out != 0) { fprintf(out, "revalidate \"%s\"\n", evt->getComponentInstance()); }}

void CmdLineBuilderViewForHuman::setOutputStream(FILE *out_) { out = out_;}

void CmdLineBuilderViewForHuman::setBuilderModel(BuilderModel *bm_) { bm = bm_; bm->addComponentChangedListener(this);}

void CmdLineBuilderViewForHuman::displayPallet() {

if(bm == 0) { pn("!no pallet available"); return; }

std::vector< std::string > pal = bm->getPallet(); pn("Components available:"); for(int i = 0;i < (int)pal.size();i++) { pn(pal[i]); }

}

void CmdLineBuilderViewForHuman::displayInstantiatedComponents() {

if (bm == 0) { pn("!No instantiated builder."); return; } ::std::map< ::std::string, ComponentInfo_shared > arena = bm->getArena(); boolean shown = false;

::std::map< ::std::string, ComponentInfo_shared >::iterator pos; for ( pos = arena.begin(); pos != arena.end(); pos++) { ComponentInfo_shared ci = pos->second; if (ci == 0) { pn("!corrupted arena!!"); assert(ci!=0); exit(1); } p(ci->getInstanceName()); p(" of type "); pn(ci->getClassName()); shown = true; }

if (!shown) { pn("!no instantiated components available"); }

}

void CmdLineBuilderViewForHuman::displayComponentInfo(const char *instanceName){

::std::map< ::std::string, ComponentInfo_shared > arena = bm->getArena(); ::std::map< ::std::string, ComponentInfo_shared >::iterator pos; pos = arena.find(instanceName); if (pos == arena.end()) { pn("!displayComponentInfo component instance not found"); return; } ComponentInfo_shared ci = arena[instanceName]; if (ci == 0) { pn("!corrupted arena!!"); assert(ci!=0); exit(1); }

pn("------------------------------------"); p("Instance name: "); pn(ci->getInstanceName()); p("Class name: "); pn(ci->getClassName());

pn("------------------------------------"); p("UsesPorts registered for "); pn(ci->getInstanceName()); pn(""); ::std::vector< UserPortData > pi = ci->getUsesPortRegister(); size_t i; if (pi.size() != 0) { char tmp[40]; for(i = 0; i < pi.size(); i++) { sprintf(tmp,"%d. Instance Name: ",int(i)); p(tmp); p(pi[i].getPortName()); p(" Class Name: "); pn(pi[i].getPortType()); } } else { p("!No UsesPorts in "); pn(ci->getInstanceName()); }

pn("------------------------------------"); p("ProvidesPorts registered for "); pn(ci->getInstanceName()); pn(""); const ::std::vector< ProviderPortData > ppi = ci->getProvidesPorts(); if (ppi.size() != 0) { for(i = 0; i < ppi.size(); i++) { p("Instance Name: "); p(ppi[i].getPortName()); p(" Class Name: "); pn(ppi[i].getPortType()); } } else { p("!No ProvidesPorts in "); pn(ci->getInstanceName()); } pn("------------------------------------");}

void CmdLineBuilderViewForHuman::pullDownComponent(const char *className, const char *instanceName) { p(instanceName); p(" of type "); p(className); pn(" \nsuccessfully instantiated");}

void CmdLineBuilderViewForHuman::connect(const char *fromInstance, const char *providesInstance, const char *toInstance, const char *usesInstance) { p(toInstance); p("))))"); p(usesInstance); p("---->"); p(providesInstance); p("(((("); pn(fromInstance); pn("connection made successfully");}

void CmdLineBuilderViewForHuman::disconnect( const char *fromInstance, const char *providesInstance, const char *toInstance, const char *usesInstance) { p(toInstance); p("))))"); p(usesInstance); p("-\\ \\-"); p(providesInstance); p("(((("); pn(fromInstance); pn("connection broken successfully");}

Component1

Component2

providesport

usesport


Needful things ... SWIG-like typemaps

Arg marshaling in Py

a = dict()a[“asdf”]=”;lkjj”a_component.a_method(a)

IOR in

Arg marshaling in C

IOR out

Arg marshaling in Java

/* in java AComponent */static void a_method(Hashtable a) {String value = (String)a.get(“asdf”)

Onramp only supports argument types that are convertible to Babel, many needed types do not exist:

Types that are not supported in all languages: e.g. hashtables.

User-defined types that need to be preserved across languages.

Nothing new here, for this purpose Babel is no different than SWIG: most of SWIG is typemaps (measured in lines of code).


Toolkit Update

Masha has made progress with Scalapack Standards for extra-bocca components:

https://www.cca-forum.org/wiki/tiki-index.php?page=Component+Builders+Guide+Sans+Bocca

(Just mouse down from the “Toolkit” menu) BoF Friday Another Telecon in a week. Possible “intensive” efforts for a small number

of toolkit components at a time you visit us we visit you



7/24/08Rob Armstrong Creating an HPC application framework from a mere HPC application automatically...

Documents

Transcript of 7/24/08Rob Armstrong Creating an HPC application framework from a mere HPC application automatically...