Deploying Tata Steel’s R&D Algorithms at Corus

(M. SHOAIB JAMEEL - 200411054)

Under the Guidance of

Prof. (Dr.) M. K. Ghose Mr. Fredi B. Zarolia Head of Department Senior Manager, I.T. Department of Computer Science I.T.S. and Engineering, Tata Steel Limited Sikkim Manipal Institute of Technology Jamshedpur

Introduction

• Project was undertaken in Three Phases

• Phase 1: Apache HTTP Server Configuration.

• Phase 2: Linux Adapter• Phase 3: CPU Load Balancing in Linux

Cluster

Problem Statement

• Proprietary algorithms have been developed over the years at both Tata Steel and Corus

• These programs were developed by scientists and researchers in the companies and as such are not built as per software development standards

The programs were developed using the following:• Languages : Delphi at Corus, Fortran and C at Tata

Steel• Platform: MS Windows/DOS and Linux• Compilers : GNU Fortran or C and Visual Studio

based

ASP.NET Web Services Infrastructure – Algorithms exposed as Web ServicesASP.NET Web Services Infrastructure – Algorithms exposed as Web Services

Windows Backend Server

(With Windows based Algorithms)

Linux Backend Server

(With Linux based Algorithms)

Linux Backend Server

(With Linux based Algorithms)

DOS Backend Server

(With Dos based Algorithms)

DOS Backend Server

(With Dos based Algorithms)

Windows Adapter Client Linux Adapter Client DOS Adapter Client

Meta Repository

Meta Repository

Algorithm Invoker ServiceAlgorithm Invoker Service

Windows AdapterWindows Adapter Linux AdapterLinux Adapter DOS AdapterDOS Adapter

Proposed Solution Architecture - Microsoft

Proof of Concept

1. User fills up a form and submits

2. The values are passed onto the Algorithm

3. The algorithm processes and returns the result

Challenges

1. Should map to the solution architecture – Generic

2. The consumer, the web service host and algorithm are on separate systems

3. Consumer and web service host run on Windows 2003 .The algorithm runs on a Linux Cluster @ R&D.

4. The output is a graph plot – a JPEG image

5. Communication among systems to be on TCP/IP stack (cannot use Disk or File system i/o)

GNU/Linux System

Microsoft Windows System

Consumer/User

Web Service

Modeling Algorithms of R&D Dept. Tata Steel

1.Invokes Web Service

2.Form Loads

Linux Adapter (C Program)

3. Enters values &Clicks “SUBMIT”

4. Connects to the Adapter via sockets and passes values along with the file name

5. Passes values to the modeling algorithm

6. Algorithm creates an image

7.Image encoded to Base64 format by the adapter

8. Encoding passed to Web service via sockets

9. Decoding, Image Displayed

Adapter Algorithm

• Listens to a port.

• Receives values of the form 1#2#4#5#...%32893823829223892

• Segregate numbers and unique identifier.

• Count values.

• Invoke Modeling Algorithm.

• Return to the web service.

Output – User Input

Base64 Encoding sent to Web Service

Image Output – base64 Decode

CPU Load Balancing in Linux Cluster

• Design Description:

• One Master Server

• One Load Balancer

• Client Machines – includes Master and Load Balancer.

Solution Architecture

The Master Server

The Load Balancer

Major Accomplishment – Worth Mentioning

• Solving the Readers-Writes Problem

Conclusion

• Adapter Currently serving requests from Corus

• Load Balancer Functioning at R&D’s Computing Cluster

Achievements

• Source codes of the adapter and CPU Load Balancing architecture to be made Open Source under GPL.

• Invitation to build and compile their custom Linux Kernel.

• Invitation to study the ClickThrough Data.• Design and problem solving appreciated by all

including Microsoft.• The project will also feature in the Research and

Development Department’s Annual Report. • Presentation at Knowledge Management Seminar

(K.M.) at Tata Steel Limited

Future Work

• Language Support – Dutch, Chinese, French.

• Security

References

• Definitions from Wikipedia and W3.org

• Books –• Coreman,T.H., Leiserson, C.E., Rivset, R.L., Stein, C., Introduction

to Algorithms, MIT Press, Mc-Graw Hill Book Company, Second Edition.

• Mall, Rajib., Fundamentals of Software Engineering, Prentice Hall India Private Limited, Second Edition ISBN-81-203-2445-5

• Stallings, W., Operating Systems: Internals and Design Principles, 4/E, Pearson Publishers

• Newcomer, Eric; Lomow, Greg (2005). Understanding SOA with Web Services. Addison Wesley. ISBN 0-321-18086-0

• Sinha, P.K., Distributed Operating Systems: Concepts and Design, Wiley-IEEE Press 1996

Acknowledgements

• Project Guide at Tata Steel – Mr. Fredi B. Zarolia

• Project Guide at SMIT- Prof. M.K. Ghose

• Microsoft Corporation

• Teachers at School

• Teachers of SMIT

Thank You

Questions

Ideas

Criticism

Suggestions