Software Integration 04052006 - The EU Provenance … · (NASTRAN) Wrapper Aero elastic Wrapper ......
-
Upload
dangkhuong -
Category
Documents
-
view
221 -
download
0
Transcript of Software Integration 04052006 - The EU Provenance … · (NASTRAN) Wrapper Aero elastic Wrapper ......
Folie 1 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Numerical simulation software integration with TENTSoftware Integration and Workflow ManagementAndreas Schreiber, DLR Simulation and Software TechnologyJoint ESTEC-DLR Workshop, ESA/ESTEC, Noordwijk, May 3-5, 2006
Folie 2 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
IntroductionWorkflows
Today, many problems require complex numerical simulations. Examples:
Re-entry simulation of space vehiclesAero dynamical and aeroelastical analysis of flight maneuvers
Performing such simulations is software technologically complexInvocation of many different codes in correct orderUsage of high performance computersTransfer of data between the different codesCollaboration with colleagues
Complex simulations are Workflows of many codes:
SimulationPostprocessing/
VisualisationPreprocessing
Data Data
Folie 3 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
What is TENT?Answer for Users
Essential features:Easy setup and configuration of simulation workflows with integrated applicationsUsage of distributed computing resourcesOnline steering and visualizationProject-based data managementwith support for cooperative workingMultidisciplinary coupledsimulations
TENT is an integration and simulation environment for engineering applications.
Folie 4 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
What is TENT?Answer for Computer Scientists
Essential characteristics:Component based
CORBA Peer-to-peer modelExtensible Java-GUIDistributed computing (Grids)Flexible integration of existing tools
Large development libraryData management
Open solution (WebDAV & XML)
Development based only on accepted standardsCORBA, LDAP, FTP, HTTP, WebDAV, XML, ...
Development based only on accepted standardsCORBA, LDAP, FTP, HTTP, WebDAV, XML, ...
TENT is an open and extensible framework for tool integration and workflow management.
Folie 5 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
CORBA
GUI
Plug-In . . . Plug-In
System components
Data Server
Factories
Nam
e Server
Co
up
ling
Scriptin
g
Applications
Wrapper
CFD
Wrapper
CSM
Wrapper
Visu
ali-satio
nWrapper
Filter
System Architecture
Folie 6 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Architecture and Deployment
Fact
ory
Visuali-zation
Nam
ing
Ser
vice
Factory
Pre-processor Simulation
Co
mp
ute
Ser
ver
Front-end Machine
CORBA
Key:
Java
other
Factory
Post-processor Adaption
Co
mp
ute
Ser
ver
Dat
a Se
rver
Workflow Visualization (GUI)
Workflow
Control
Folie 7 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Application Application browserbrowser
Property editorProperty editorand plugand plug--insins
WorkflowWorkflowDesignDesign
Help, Logging, GUI plugHelp, Logging, GUI plug--insins
ControlControl
Folie 8 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
PlugPlug--In: Panel In: Panel
PlugPlug--In: ToolbarIn: Toolbar
Meta dataMeta data
Project browserProject browser
Folie 9 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Example GUI Plug-In2D-Plot
Folie 11 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
CORBA
Factory
GUI
FactoryFactory
TENT-Factory: (CORBA-)Server for starting programs
Grid
Cluster (LSF, NQS, ...)
Workstation / PC
Distributed Computing
Divisions-/Companies
Local Server/Workstation
Worldwide
Folie 12 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Integration of ApplicationsWrapper
StartControl, Properties
CORBA-Interface
Control
Oth
er T
ENT
com
po
nen
t
Oth
er T
ENT
com
po
nen
t Application
Wrapper (application specific)
CORBA
Folie 13 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Integration of ApplicationsWrapper Library (SDK)
Wrapper development in Java or PythonSome features
Generation of input filesTemplating (replacement of parameters)
Parsing of result filesExtraction of valuesGeneral filter mechanism
ExecutionRemote executionApplication control via pipes, TCP/IP sockets, CORBA, …
Wrapper code can be generated…
Folie 14 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Integration of ApplicationsAutomatic Generation of Wrapper Code
To reduce the effort and minimize coding errorsTENT specific Eclipse plug-ins for:
Generating new wrapper codeExtending existing code
Folie 15 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Control flow(method invocation, properties)
Control flowEvents and method invocations through CORBA
Communication
CORBA-Interface CORBA-Interface
Data flow(FTP, GASS, ...)
Data flowSmall data set as parameter through CORBAMassive data through GASS, FTP, GridFTP, Socket, ...
CORBA
Folie 16 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Data ManagementProject and Workflow Structure
Free hierarchical structure of projects and data:
Cooperative workingUser management and access controlData exchange via data serverSimilar to PDM systems
Check-Out, Check-In
Components
Simulations
Projects
Configuration & data files
Folie 17 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
TENT Data server
Data ManagementXML & WebDAV
ConceptStorage of project data in XML format on a serverAccess to data using the standard protocol WebDAV
TENT
Other applications
InternetFile Server
DatabaseHTTP/WebDAV
Web
DA
V
HTTP/WebDAV
WebDAV: Web Distributed Authoring & VersioningExtension of the HTTP protocol (locking, meta data, versioning, search, and access control)
Folie 18 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Data ManagementData Provenance
Provenance is information in addition to the standard logging which helps answer questions about origins of dataTypical such questions are
Given some data item, what was the simulation case?Given some parameter, in what simulation(s) has it been used?What data has been recorded in a simulation with a specific parameter?What simulations have been run using a given model (aircraft design)?Given two/more simulations with the same setup, what is the result and the difference in provenance?
TENT has integrated a provenance storing and querying service(developed in the EU GRID PROVENANCE project)
Folie 19 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
ScriptingIntegration, Control and Automation
Applications do not have everything, so scripting is needed.Usage of scripting in TENT
Integration of applicationsWorkflow control (loop constructs, conditional constructs, coupling control)Test scripts for quality assurance automationDebuggingCommand journaling
TechniqueEmbedded Python interpreterScript interpreter in the GUIScript console in the GUIScripting block in workflows (control and code integration)
Folie 20 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Application Systems
Virtualaircraft
Space reentry vehicles
AutomobiledesignFlight maneuvers
Turbine engines
Folie 21 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Example Application 1 Flight Maneuver Simulation
Interactive simulation environment for the simulation of a freely flying, fully configured, elastic warplaneProject SikMa: X-31
Folie 22 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Flight Maneuver SimulationModel
Simplified model for wind tunnel experimentsSimulation for validation of experiments
Folie 23 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Flight Maneuver SimulationResults
Coupled simulation: Aerodynamic – flight mechanics – aeroelasticityComputing times: ~ 7 days on large computing cluster (> 64 procs.)Important: Online monitoring and restart capability
Folie 24 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Coupling Control
Structure solver(NASTRAN)
WrapperAero elastic
Wrapperflight
mechanics
CFD solver(TAU code)
Wrapperfluid
dynamics
CO
RB
A
Files
Files
TENT GUI
Flight mechanics(SIMULA)
Files
Aero elastic(MATLAB)
Files
HPCmachine
Visualization
Flight Maneuver SimulationIntegration in TENT
Filetransfer
CORBA
Data
IPC
Host/Filesystem
Folie 25 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Filetransfer
Programmstart
TENT-Daten/Monitoring
Host/Filesystem
Netzwerk
SIMULASIMULAServer
TAUCodeTAUCode
Desktop machine
GUI VisualizationVisualization
WrapperCFD
WrapperCFD
Cluster
CouplingControl
CouplingControl
WrapperFlight mech.
WrapperFlight mech.
Data server
Batch
-SystemB
atch-System
Flight Maneuver Simulation - Deployment
Folie 27 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Flight Maneuver Simulation Online Monitoring of Results
Folie 28 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Example Application 2Space Re-Entry Simulation
Simulation of thermal heavily loaded parts of the X-38 space re-entry vehicleFluid-Structure-Thermal coupling
Folie 29 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Space Re-Entry Simulation Model
Simplified model for wind tunnel experimentsSimulation for validation of experimentsFocus on flaps and gap
Folie 30 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Space Re-Entry Simulation Results
Coupled simulationDLR-TAU code for aerodynamicsANSYS and MSC/NASTRAN for heat transport and deformation in the structure
Folie 31 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Coupling with TENTUse of coupling library MpCCI
MpCCI/MPIMpCCI/MPI
TENT Coupling
Configuration
Resource selectionand start
Structuresolver
Co-ProcessStructure
WrapperCo-Process Structure
Couplingcontrol
Wrappercouplingcontrol
CFDsolver
Co-ProcessFlow
WrapperCo-Process
Flow
CO
RB
A
Mp
CC
IM
pC
CI
Files
Files
TENT GUI
Visualization
Filetransfer
CORBA
MpCCI
IPC
Host/Filesystem
Folie 33 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
New TechnologiesTechnology Upgrade for TENT
TENTGUI with Plug-Ins (proprietary development)Single role for all usersSingle communication protocol (CORBA)Authentication with user name & passwordFile-based data managementModularity in sub systems
RCE (Project SESIS)GUI Plug-Ins Framework (Eclipse)Multiple user rolesVarious protocols (Web Services, Java RMI, CORBA, …)Authentifizierung with certificates (single-sign-on)Data management with complex data structure (and files)Everything is modular
Project SESIS
TENT used modern technologies of the past 10 yearsTechnology upgrade for the next 10+ years in project SESIS
Folie 34 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Project SESISGoals
Design in 7 Days in a virtual organization
Early ship design for creation of offersReduction of design timesDistributed design und simulation
Shipyards organizing ship building processIntegration of supplier in design process
Quelle: Flensburger Schiffbau-Gesellschaft mbH & Co. KG
Folie 35 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Project SESISSystem Design
System designed with a Plug-In architecture based on the OSGi platform
RCE (Reconfigurable Computing Environment)Advantages
Consistent modularizationGood extensibility und scalabilitySpecificity by configurationUse of many existing Plug-Ins (see Eclipse!)
Folie 36 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
HardwareGrid Middleware
Project SESISBase-System Layer
Usage of databases and Grid middleware solutionsSESIS supports
CommunicationUpdatesPrivilege managementServices discovery
SESIS
Installation of identical base system on every machine
Specificity by loading additional plug-ins
Installation of identical base system on every machine
Specificity by loading additional plug-ins
Folie 37 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006
Usage ScenarioVirtual Organization
RCESESIS
RCESESIS
DBRCESESIS