Slide 1

PGP Project Viktor Yarmolenko Lewis Mackenzie Paul Cockshott Ewan Borland Slide 2 Background Motivation Proposed Solution Status Questions Contents Slide 3 Background Setup Slide 4 Background Algorithm View from 3 cameras Slide 5 Background Algorithm X warp Y warp Correlation Slide 6 Background Algorithm Reconstructed model viewed from a Virtual perspective position. Slide 7 Background Highly parallelizable problem the following can be done independently: 8 Matching jobs per group N process groups per sequence ~1min on Matching ~1min on Building (2GHz) The problem Slide 8 Motivation Dynamic data graphs S M T S M 1 1 Slide 9 Motivation S M T S M B 1 1 1 Dynamic data graphs Slide 10 Motivation S M T S M B M M 1 1 1 2 2 Dynamic data graphs Slide 11 Motivation S T S B M M 2 2 1 Dynamic data graphs Slide 12 Motivation The search for silver bullet Allows dynamic creation of processes (arbitrary code) Allows creation of communications channels between processes. Allows channels to be dynamically reconfigurable. 1.To develop a highly parallel system for 3D computer vision algorithms, 2.Which also can be a general framework for distributed processing. Our Aim: Primary Requirements: Slide 13 Motivation The search for silver bullet Use of GRID as a transport layer sounds good The requirements could not be readily met by existing GRID protocols No current API (MPI, RMI, PVM, DSM) can readily meet the requirements We need a conceptually new parallel architecture Slide 14 Solution The key concepts taken from Milners calculus are the ability to dynamically create processes and communication channels and to transmit communication channels along other channels. What is J ? J is a Java interface loosely modelled on the primitives of the -calculus (Milner) to be used as a substratum for GRID based parallel computing. "The Polyadic -calculus, a tutorial, Milner, R., (1991) A calculus of mobile processes, Milner, R., Parrow, J., Walker, D., Information and Computation, 100:1-77, (1992) (Milner) Slide 15 JPieTask implements Runnable JPieFunnel extends OutputStream JPieTap extends InputStream JPiePipe contains connected JPieTap and JPieFunnel J primitives Solution 1000km Slide 16 VM 21 1 J example Solution Slide 17 ILS model Initiator Locator Servent Initiator which Starts job Locator currently at EPCC Runs web services Maintains MySQL database of servents Servent runs JPie daemon Register With locator Query locator Reply with Available servents Any servent can also become an initiator during the course of a computation and spawn more tasks Slide 18 ILS model Initiator Locator Servent Initiator Locator keeps track of available memory and performance of servents Servent Register performance and memory Queries locator For machines of Above performance Level X Reply with list of suitalble servents Java Grande Benchmark used To rate Servent performance Starts job Slide 19 Status Need Your Thoughts Performed network tests using the data demanding part of the algorithm. Data transfer is not a bottleneck in this problem. Completed the multi processor implementation of J , using sockets. Currently installing resource locator using Web services to find free cpus. Would like to implement J transport using a GRID layer. Acknowledgements: NeSC Sponsors of the project EPCC Discussions Slide 20 Examples Conformed sequence Slide 21 Examples Conformed mesh Slide 22 Examples Landmark grid Slide 23 Examples Textured Slide 24 Examples Textured + mesh Slide 25 Stream down the stream in pictures VM 21 1 Slide 26 Preliminary Tests Exp Total CPUs Local CPUs Remote CPUs Time (min) Comments 0440215Four local PCs 112 07512 local PCs (4 Dual + 4 Single, in the same room) 2160 4816 remote PCs (16xCPU IBM machine 60 miles away) 32812162916 remote + 12 local 44312312116 remote + 12 local + 15 other PCs from the department A part of algorithm was used Total data transfer is over 5GB Processed 12 sec of 3D video That is 3600 images (12 25 4 3) The bandwidth at its bottleneck 100Mbits Virtually theoretical speedup Can be improved by using J Client 1 Thread 1 Host 1 HDD Thread 2 Thread 3 Thread n Client 4 Thread 1 HDD Thread 2 Thread 3 Thread n Host 2 Host 3 Host 4 Host 5 Host 6 Host 7 Host N