INFOMR Project

Dafne van Kuppevelt ● Vikram Doshi ● Seçkin Savaşçı

Development Review

Heat Kernel Signature

Before Development

• C# develop faster use .NET framework’s abilities our experience on C# and other similar languages(java)

• x86 architecture• Visual Studio IDE• Google Code for project hosting

Easy to start Complete package for project

management Support git, svn & mercurial http://code.google.com/p/infomr-group-x/

Initial Parsing

Getting Laplacian Matrices form Off Files

• L = D – A

Storage Problem

3 -1 0 -1 -1

-1 3 -1 0 -1

0 -1 3 -1 -1

-1 0 -1 3 -1

-1 -1 -1 -1 4

1

0 1

1 0 1

1 1 1 1

450002 * 32 = 7.54 GB 45000*44999/2*1= 120 MB

~64X Compression Computational Overhead

Initial Parsing

Storage Problem( continued)

Human Readable Matrix File Serialization to Files

Able to use in different language implementations

Language Specific

Storage Inefficient(string conversion,whitespaces, line feeds)

Storage Efficient

Own parsing method Deserialize and use!

Time Problem

Human Readable - Integer Serialization – Bitwise

~2 hours ~2 mins

Larger files cannot be parsed All can be parsed

Initial Parsing

Getting Eigen Values & Vectors from Laplacian Matrices

• We tried to implement our own eigen decomposer FAILED

• We started to search for a suitable library Our needs were:

- Structure for storing sparse symmetric matrices - Eigen decomposer method that is specialized for sparse symmetric matrices partially FAILED

We switched to trial & error for finding a good library, our goal was now speed and goodmemory usage.

Initial Parsing

library Spec Status Time Memory

Math.NET C# x86 Seq LAPACK xGPL 20+ mins Average

DotNumerics C# x86 Seq LAPACK port xGPL 2 mins Average

CenterSpace C# x86 Seq LAPACK $1295 30+ mins Good

Eigen C++ x86 Seq UNIQUE xGPL 2 hours Good

Experiment results with 2000 vertex model :

DotNumerics Structure for storing Symmetric matrices Eigen Decomposer for Symmetric matrices

~40 largest models are out of project scope due to time and memory problems

Initial Parsing

Time and Memory Problems

• Largest model will take more than 250 hours to parse• For the largest model we must have ~16 GB memory space because DotNumerics use Double precision for storing values

Overview of running on current architecture (x86): 2GB process dedicated memory /largeaddressaware 3 GB We cannot make injections due to LAPACK calls

It is impossible to reach beyond 3 GB address space on x86, theoretically 4 GB

Curious Cat?

Initial Parsing

PrototypeFrom To

Arch x86 x64

Virtual address space 3 GB 8 TB

Virtual Memory 4 GB 20 GB

Language C# C++

Library DotNumerics Armadillo(xGPL, tweaked for x64)

Base library LAPACK Intel MKL Parallel ( $129)

Other Library - Boost

Parallel , Used CPUs No, 1 Yes, 2

Optimization for Intel CPU No Yes

Other Memory Improvements - Memory Mapped File

Result : Eigen Decomposition time for 2000 vertex model? Guess?

Initial Parsing

Model Vertex Count Time Memory

2000 10 seconds Superb

Largest one (45000) 2 days ( Due to my 3GB main memory and memory mapped files, Disk I/O times become significant ; average 10 times slower for the best access)

Very bad because of memory mapping

Prototype Results

We didn’t change to develop on this prototype, because :

• Parsing the largest one is still infeasible• Not all of us have x64 architecture