High-Level Workflowsfor Social Science

Ken Turner (and Larry Tan)Computing Science and Mathematicswww.cs.stir.ac.uk/~kjt/research/cress.html

24th November 2010

Workflows in Social Science low-level (micro) flows are sequences of

steps using some statistical package, e.g.:

• retrieve datasets D1 and D2• recode variable V1• cross-tabulate V1 and V2

high-level (macro) flows combine the capabilities of service partners, e.g.: data retrieval data cleaning data fusion data analysis

High-Level Workflows in DAMES

an approach has been developed for high-level workflows in social science: the services are external, being packages

that conform to web/grid computing standards

the workflow logic is defined graphically this is automatically analysed, and

translated into BPEL (Business Process Execution Logic)

the supporting tools are: CRESS: workflow definition and translation ActiveBPEL: workflow orchestration

CRESS Communication Representation

Employing Systematic Specification: graphical workflow notation application/language/platform-independent automated analysis and implementation mature - under development for 13 years

supported by other packages: CHIVE: graphical workflow editor MUSTARD: workflow validator CLOVE: workflow verifier MINT: performance analyser

CRESS Methodology

WorkflowDiagramWorkflowDiagram

PreciseSpecification

PreciseSpecification

automaticspecification

RigorousAnalysisRigorousAnalysis

validation/verification

Implementation Code

Implementation Code

automatic compilation

PerformanceAnalysis

PerformanceAnalysis

scenarioevaluation

designcorrections

CRESS Example the following example illustrates mapping

one occupation to two different schemes only an outline is given, omitting details the cooperating services are:

lookup: performs parallel mapping (workflow) allocator: finds an available job mapper then

does the mapping (workflow) factory: manages mapper resources (partner) mapper: performs a mapping for some scheme

(partner)

Parallel Job Translation

3 Invokeallocator.job.translate

mapping1 code1

3 Invokeallocator.job.translate

mapping1 code1

4 Invokeallocator.job.translate

mapping2 code2

4 Invokeallocator.job.translate

mapping2 code2

1 Receivelookup.job.translate

schemes

1 Receivelookup.job.translate

schemes

2 Fork2 Fork

6 Replylookup.job.translate

codes

6 Replylookup.job.translate

codes

5 Join5 Join

Job Mapper Allocation1 Receive

allocator.job.translatemapping

1 Receiveallocator.job.translate

mapping

2 Invokefactory.job.allocator

scheme mapper

2 Invokefactory.job.allocator

scheme mapper

4 Replyallocator.job.translate

mapping

4 Replyallocator.job.translate

mapping

3 Invokemapper.job.translate

job mapping

3 Invokemapper.job.translate

job mapping

Summary low-level workflows define the sequence

of basic steps in an analysis package high-level workflows invoke external

service partners and combine their results

CRESS allows high-level workflows to be defined, analysed and executed