NEWERAHPCwww.newerahpc.com

Varun Mittalvarunmittal@newerahpc.com

Udit Kaushaluditkaushal@newerahpc.com

What is Grid Computing?

Grid computing solution is a program which can make your work easy by utilizing the power of

other computers.

Grid computing programs are categorized under two sub categories : pvm and mpi.

PVM(parallel virtual machines) : PVM is used in places where purely mathematical calculations are required and the algorithm can be divided into small pieces and combined later.MPI(message passing interface) : MPI is another category where some network instructions are embedded into the program, it is an efficient method but requires an external file sharing server or a similar service and both the server side and client side source code requires altogether reprogramming to insert mpi specific code into it.

The biggest problem!!!!

Most of the pvm or mpi libraries are designed to perform some specific kind of functions. Like some are designed to perform mathematical calculations, some to encode or decode videos.But very few are customizable enough to perform different kind of jobs like running a crawler for a search engine etc.

NeweraHPC Search Engine

We are using this library primarily for running a search engine's crawler. An address is chosen which has not been indexed yet and send to the client. The client processes the web address and passes through our algorithm and returns the result to the server.

NeweraHPC Search Engine

Another major application of this library is a new database system that we will be managing on heterogeneous systems. When the client submits the processed data it goes to a uniform buffer where the grid collects information and stores data on some portion of the grid. Thus uniformly dividing data.

SERVER

HPC Network Library

Uniform DiskAccess model

TCP/IP Protocol Layer

Newera Network Library

Protocol Analyzer

Pool of custom functions

General buffer/Data Dispatcher

General Overview

The grid server operates on port 80. It has an inbuilt http server and a grid server.When a client makes any request it analyzes the protocol and accordingly forwards request to the desired function.Grid specific function write data to the client on top of tcp/ip layer and convert back the result into the desired format or variable type.

How to use the library

The first step is to integrate your startup function(s) to the grid.In src/grid/grid_custom_functions. Mark your custom function(s) with empty function pointer. For eg: int(*func1)(comm_instruction_set *) = &my_function1;Comm_instruction_set in a standard way of representing a task in newera library.

Comm_instruction_set

struct comm_instruction_set{ int function; int data_type; int argument_count; void *values; int (*func)(comm_instruction_set *); int sockfd; int status; char *result_char; pthread_t c_thread; FILE *data_file_primary; FILE *required_exec;};

Comm_instruction_set

int function; alias for custom client function int data_type; alias for custom or pre defineddata type

int argument_count; number of arguments to be passedvoid *values; arguments in any data type

FILE *data_file_primary; any file required by the client FILE *required_exec; any executable/dynamic library for client

Rest all is done by the libraryint (*func)(comm_instruction_set *);##done by the libint sockfd;int status;char *result_char;

pthread_t c_thread;

For server side example look into test.cpp in the root directory of the program.

For the test program

We will be using arctan algorithm for calculating pi value.

THANK YOU