Post on 28-Jun-2020
Achieving LanguageInteroperability with
Gary Kumfert with
Bill Bosl, Tammy Dahlgren, Tom Epperly, Scott Kohn, &
Steve Smith
GKK 2CASC
Babel’s Scope & CCA� Babel provides language
interoperability, not components.
� We collaborate with CCA to add parallel distributed support
� We also provide tools (Quorum & Alexandria) to facilitate component development and
deployment
Compilers &Linkers
Operating System
CCA Compliant FrameworksComponentSemantics
GKK 3CASC
Release Announcement
� C, C++, F77, Python(client)
GKK 4CASC
babel-0.5.0.tar.gz
� Babel code generator� written in Java
� Babel runtime library� written in ANSI C
� Docs (minimal)� papers, talks, javadoc html� babel101 tutorial
GKK 5CASC
Hand Coded Language Interoperability
JNINativeSWIG
Platform Dependent
C
C++
f77
f90
Python
Java
GKK 6CASC
Babel Enabled Language Interoperability
� Truly Object Oriented� Reference Counting
� Exception Handling� RMI (future)
C
C++
f77
f90
Python
Java
GKK 7CASC
Babel Enabled Language Interoperability
What’s In This Release:What’s In This Release:
C
C++
f77
f90
Python
Java
GKK 8CASC
Babel Has Two Types of Customers
� Have a code� Want to increase
their user base� Will learn SIDL� Want Babel
general and powerful
� Have a problem� Want to solve their
problem� May never see SIDL � Want software that’s
easy & trustworthy
Developers Users
GKK 9CASC
Babel’s Design Priorities
� Performance� Developer/User dichotomy� What’s natural for each language?
� Could expose C array structs in C++� C++ style would be SIDL::array<T>template <>array<item_t> : public array_mixin
< array_t, item_t, item_cxx_wrapper_t> { }
template <>array<item_t> : public array_mixin
< array_t, item_t, item_cxx_wrapper_t> { }
GKK 10CASC
SIDL (Scientific Interface Definition Language)
� Builds on Industry IDL technology
� CORBA� COM
� Designed for Scientific Apps
� complex types� dynamic
multidimensional arrays
version Hello 1.0;package Hello { class World {string getMsg();
}}
GKK 11CASC
version MySolverLib 0.1.0;import ESI; package MySolverLib { interface MatrixGenerator { ... } class OptionDatabase {void getOption( in string name,
out string val);}class Vector implements-all ESI.Vector {void setOptions( in OptionDatabase db );
}class Bizarre implements MatrixGenerator {
... void setData( in array<dcomplex,2> a );
}}
GKK 12CASC
Many forms of language interoperable interfaces
SIDLScientific Interface
Definition Language
Human Comp
IORInternal ObjectRepresentation
mpiCo
XMLeXtensible Markup
Languageb Compatible
atible
We
ler Compatible
GKK 13CASC
XML enables......automated creationvia higher-level tools...Type Descriptions
on Shared Repositories
...Browsingfor Types
...automated search &
discovery by advanced
builders
GKK 14CASC
Language Interoperability: How Babel Makes it Work
Impls
� Application: user’s LOP(Language Of Preference)
� Implementation: developers LOP (Can be wrappers to legacy code)
SIDLSIDL
If developer used Babel,they also have a SIDL file.
Stubs
Skels
IORs
Application
GKK 15CASC
Language Interoperability: How Babel Makes it Work
Application
Stubs
Skels
IORs
Impls
� Application: user’s LOP(Language Of Preference)
� Implementation: developers LOP (Can be wrappers to legacy code)
� Client Side Stubs: user’s LOP to C
� Internal Object Representation (IOR): Always in C
� Server Side Skeletons: translates IOR (in C) to developer’s LOP
GKK 16CASC
How Much Language Interoperability Have We Achieved?
� Basic Types� bool� char� int� long� float � double� fcomplex� dcomplex� string� opaque
� Basic Types� bool� char� int� long� float � double� fcomplex� dcomplex� string� opaque
� Extended Types� Objects� enumerations� arrays of basic types� arrays of objects� (arrays are
multidimensional, no arrays of arrays)
� Extended Types� Objects� enumerations� arrays of basic types� arrays of objects� (arrays are
multidimensional, no arrays of arrays)
� Modes� in� out� inout� return
value
� Modes� in� out� inout� return
value
� OO Method Dispatch� regular� final� static� interfaces� classes
� OO Method Dispatch� regular� final� static� interfaces� classes
� Exception Handling� Exception Handling
� For All Combinations of Languages� C C� C++ C++� F77 F77� Python
� For All Combinations of Languages� C C� C++ C++� F77 F77� Python
� 1431 test cases (and counting)
� 1431 test cases (and counting)
GKK 17CASC
0
200
400
600
800
1000
1200
1400
1600
Jan-00
Feb-00
Mar-00
Apr-00
May-00Jun-00Jul-0
0Aug-00Sep
-00Oct-
00Nov-00Dec-00Jan
-01Feb
-01Mar-
01Apr-0
1May-01Jun-01Jul-0
1
passxfailfail
Test Historysparc-sun-solaris2.7-gcc#
Test
Cas
es
Date
babel 0.5.x
babel 0.4.x
C++ start
Parser start
ANSI C start
Python start
F77 start
IOR
Rew
rite
GKK 18CASC
What we foresee, based on experience with our tests...
Language Interoperability
Developer Concerns:Configuration, Packaging,& Deployment
User Concerns:InstallationTrust
GKK 19CASC
GKK 4CASC
Hand CodedLanguage Interoperability
JNINativeSWIG
Platform Dependent
C
C++
f77
f90
Python
JavaGKK 4CASC
Hand CodedLanguage Interoperability
JNINativeSWIG
Platform Dependent
C
C++
f77
f90
Python
Java
These aren’t new problems...But they are on a larger scale
Developer Concerns:Configuration, Packaging,& Deployment
User Concerns:InstallationTrust
Babel enabledLanguage Interoperability
� Object Oriented� Reference Counting
� Exception Handling� RMI (future)
C
C++
f77
f90
Python
JavaGKK 5CASC
Babel enabledLanguage Interoperability
� Object Oriented� Reference Counting
� Exception Handling� RMI (future)
C
C++
f77
f90
Python
Java
GKK 5CASC
GKK 20CASC
For Example...� Consider the following statements...
� Java is more portable than C� C is more portable than C++ � C++ is more portable than F77
� “portable” in a different sense� Java doesn’t need to be recompiled (like C)� C is mature, isn’t as hard to parse (as C++)� C++ can bind to C or Java easier than F77
� BUT F77 is also more portable than C because of header/library issues
Babel’s
regression tests
have the worst of
all worlds!!!
GKK 21CASC
For Example...� To support Python and Java
� All libraries must be shared (*.so) not statically linked (*.a)
� C++ shared libraries are problematic � Exception support is platform/compiler
dependent� Linking issues when interoperating with
other languages� Can create valid shared library with
uncatchable exceptions
GKK 22CASC
Babel’s Configuration/Build� GNU Make � Autoconf
configuration� Automake
build Makefiles� Libtool
shared libraries� CUTE
custom testing
� python’s own build system
� java’s built-in (broken) make
� helper scripts� fixes to autoconf,
automake &libtool
� lots of hacks
GKK 23CASC
0
200
400
600
800
1000
1200
1400
1600
Jan-00
Feb-00Mar-
00Apr-0
0May-00Jun-00Jul-0
0Aug-00Sep-00Oct-0
0Nov-00Dec-00Jan
-01Feb-01Mar-
01Apr-0
1May-01Jun-01Jul-0
1
passxfailfail
Test Historysparc-sun-solaris2.7-gcc#
Test
Cas
es
Date
babel 0.5.x
babel 0.4.x
IOR
Rew
rite
Configure/Build/Repository faults,not software failures!!!
GKK 24CASC
Problems affect the User too...� How does a user get and install
“language interoperable” software?� Binary: if supplied by developer� Source:
Assume “configure; make install”?� How to link into application?
� If any C++ code, must use C++ linker� Which C++ to use?
C++ has no std binary interface
GKK 25CASC
Crux of the problem
� We’re building 21st century technology...
� ... using 30 year old tools.
MakeBourne shell
Autoconf (M4)Automake (perl)libtool (perl/sh)
JavaPythonF77C++C
Autoheaderaclocal
GKK 26CASC
Solution
� Integrated config, build, package, test and management system.� no make inside!
can have action create many filesunderstands directories
� uses real database� program all aspects in one language� MUST BE OPEN SOURCE
GKK 27CASC
In the mean time...
� Babel works on other platforms, just not automated config/testing� Java code generator (precompiled)� ANSI C runtime library (no problem)
� Babel’s tests are (necessarily) pathological worst-case examples
� We didn’t create these problems, we just exercise them aggressively
GKK 28CASC
Future Babel: Will Provide More Build Help
� “babel.make” � currently lists code generated� may add additional flags, macros, etc.
� configure� currently used for regression tests� may generate artifacts useful for
developershelper scriptswarnings
GKK 29CASC
Farther Future Babel:Will Do Distributed Computing
Internet
Marshaler
Line Protocol
Line Protocol
UnmarshalerStubs
IORs
Skels
IORs
Application
Impls
GKK 30CASC
Closing Remarks
� Babel Beta 0.5 is released� Babel enables language
interoperability � connect C, C++, F77, and Python � provide a uniform object-model,
even in non-OO languages. � Deploying & Installing Language
Interoperable Code in General� is still very hard� has broken every tool we use
GKK 31CASC
The Endbabel-announce@llnl.gov
babel-users@llnl.gov
http://www.llnl.gov/CASC/componentscomponents@llnl.gov
Bill Bosl, Tammy Dahlgren, Tom Epperly, Scott Kohn,
Gary Kumfert, & Steve Smith
GKK 32CASC
A.3. Can HPC and Component Technology REALISTICALLY be integrated?
� Yes.� But HPC Components have huge
(and unique) hurdles:� Diverse Architectures� Diverse OS’s� Integration of SPMD and Dist. Comp.� Archaic Pkg/Devel/Config/Build tools� Non-CS trained (or interested) users
GKK 33CASC
B.3. Can the HPC community really afford yet another compiler such asBabel?
� Is language interoperability important?
� How important?
GKK 34CASC
B.3. How is the Java subset of C++ inadequate as an HPC IDL?
� What is a “Java subset of C++” ?� How does one
� use it to bind to other languages? � get a common inheritance model?� get a common exception model?
GKK 35CASC
B.5. What is the role of traditional (parallel)tools in component technology?
� Hopefully, they’re replaced by modern parallel tools.
GKK 36CASC
B.9. What will be the configuration issuesfor components...to be portable andhigh-performance?
� Lots.� Lack of Configuration, Packaging,
& Deployment tools � is the #1 Achilles heel for
components� is the #1 day-to-day pain in Babel
development� #1 cause for failure in regression
tests
GKK 37CASC
C.1. Will anyone actually make the effort tocomponentize their applicationssoftware?
� Yes.� But it will be messy.
� Efforts to Babelize at LLNL:� hypre - want OOP in ANSI C &
automatic F77 bindings� ALPS - want scripting interface for
laser plasma physics � SAMRAI - framework used in ALPS
GKK 38CASC
D.4. Should components be viewed as mostly a library/runtime developertechnology?
� No. � I used components in this
PowerPoint Presentation� Users can use components without
knowing they’re using them.� This is harder to achieve in UNIX than
other platforms
GKK 39CASC
23 Jul 2001UCRL-PRES-144649
Work performed under the auspices of the U. S. Department of Energy by the University of California, Lawrence Livermore National Laboratory under Contract W-7405-Eng-48