The MU3C Computer Cluster The MU3C Computer Cluster ArchitectureArchitecture

Mike PoublonMike PoublonHope CollegeHope College

What is a Computer Cluster?What is a Computer Cluster?

A computer cluster is a group of A computer cluster is a group of networked computers that work together networked computers that work together as a single high performance computeras a single high performance computerA computer cluster can speed up compute A computer cluster can speed up compute times through parallel executiontimes through parallel execution

Computer clusters provide a high level of Computer clusters provide a high level of performance at a low cost when compared performance at a low cost when compared to traditional supercomputersto traditional supercomputers

The MU3C Computer ClusterThe MU3C Computer Cluster

KVM switches

KVM switches

network switches

(rear)computenodes

compute nodes

raid arrays keyboard

& displayhead nodes compute

nodesUPS

(bottom)

Computer Cluster UsesComputer Cluster Uses

Clusters can be used on any computational task, Clusters can be used on any computational task, but perform their best when working on a task but perform their best when working on a task that can be broken down into smaller, parallel that can be broken down into smaller, parallel subtaskssubtasks

Examples:Examples:–– Molecular ModelingMolecular Modeling

–– Weather PredictionWeather Prediction

–– Genetics Research Genetics Research

–– Physics Simulations Physics Simulations

–– 3D Scene Rendering (Movies like 3D Scene Rendering (Movies like ShrekShrek, Bug, Bug’’s Life, etc)s Life, etc)

Why Use a Computer Cluster?Why Use a Computer Cluster?

Computations can take large amounts of time on Computations can take large amounts of time on a single computer due to:a single computer due to:–– High accuracy modeling High accuracy modeling –– Highly dimensional calculationsHighly dimensional calculations

Examples:Examples:–– 4 atom high accuracy PES takes 4 atom high accuracy PES takes

1 month compute time; 5 atoms 1 month compute time; 5 atoms takes 1 yeartakes 1 year

–– 1 nanosecond simulation of QM 1 nanosecond simulation of QM molecular dynamics takes 10 yearsmolecular dynamics takes 10 years

Computer Cluster AdvantagesComputer Cluster Advantages

Cluster computers:Cluster computers:–– are affordable when compared are affordable when compared

to their supercomputer cousinsto their supercomputer cousins–– run standard software (Linux)run standard software (Linux)–– can be expanded over timecan be expanded over time–– are fault tolerant in many casesare fault tolerant in many cases

MU3C Computer Cluster LayoutMU3C Computer Cluster LayoutLocal Network or

InternetHead NodeHead Node

Hope Hope College

Storage NodeStorage Node

Jobs issued by the head node (PBS)

Parallel Node Communication (MPI)

College

CarletonCarleton

Command Command Network

DataDataNetworkNetwork Network

Compute NodeCompute Node

GustavusGustavus--Adolphus

Macalester

File sharing traffic (NFS)

AdolphusMacalester

MU3C HardwareMU3C Hardware

2 x 2.6 GHz 2 x 2.6 GHz Opteron Opteron processorsprocessors8 GB ram8 GB ram250GB hard 250GB hard drivedrive

2 x 2.6 GHz 2 x 2.6 GHz Opteron Opteron processorsprocessors8 GB ram8 GB ram2 x 250GB 2 x 250GB hard drive hard drive (raid 1)(raid 1)2 x 5 TB raid 2 x 5 TB raid arrayarray

2 x 2.6 GHz 2 x 2.6 GHz Opteron Opteron processorsprocessors8 GB ram8 GB ram2 x 250GB 2 x 250GB hard drive hard drive (raid 1)(raid 1)

Compute NodeCompute NodeStorage NodeStorage NodeHead NodeHead Node

MU3C Hardware TotalsMU3C Hardware Totals

112 2.6 GHz Processors112 2.6 GHz Processors448 GB ram448 GB ram14 TB of distributed (on node) storage14 TB of distributed (on node) storage10 TB of central file storage (on storage 10 TB of central file storage (on storage node)node)

Services ProvidedServices ProvidedNTP:NTP: (Network Time Protocol) Synchronizes (Network Time Protocol) Synchronizes time inside the clustertime inside the clusterSamba:Samba: Allows Mac and Windows computers to Allows Mac and Windows computers to access files across the networkaccess files across the networkWWW:WWW: Standard web server gives users easy Standard web server gives users easy access to documentationaccess to documentationSSH/SCP:SSH/SCP: Users can log in to the cluster from Users can log in to the cluster from their desktop computers to submit jobstheir desktop computers to submit jobsPBS:PBS: Manages job schedules to efficiently use Manages job schedules to efficiently use cluster resourcescluster resourcesMPI:MPI: Allows nodes to communicate in a parallel Allows nodes to communicate in a parallel fashionfashion

Chemistry ProgramsChemistry ProgramsAces2Aces2Amber 8Amber 8Gaussian03Gaussian03GamessGamess (parallel)(parallel)MolproMolpro 2002 (parallel)2002 (parallel)MopacMopacNWChemNWChemQchemQchemTinkerTinker

What Can The Cluster Do For Me?What Can The Cluster Do For Me?

Initial results show that computations Initial results show that computations benefit greatly from parallelismbenefit greatly from parallelism

Molpro 2002 CCSD(T) Energy Calculation of HFCO (in sec.)

5318531810034629894

72897289124956110683

1082610826187186813862

21017210173590163725031

aug-cc-pvqzcc-pvqzaug-cc-pvtzcc-pvtz6-31gProcessorsProcessors

Aggregate PerformanceAggregate Performance

Performance measured using the High Performance measured using the High Performance Performance LinpackLinpack BenchmarkBenchmarkwww.Top500.orgwww.Top500.org–– 11stst Place Place –– 280600 280600 GflopsGflops (131072 Processors)(131072 Processors)–– 100100thth Place Place –– 3980 3980 GflopsGflops (2560 Processors)(2560 Processors)–– 500500thth Place Place -- 1645.7 1645.7 GflopsGflops (460 Processors)(460 Processors)–– MU3C MU3C –– 343.5 343.5 GflopsGflops (106 Processors)(106 Processors)

Top 100 Over The YearsTop 100 Over The Years

72722828CurrentCurrent(280600 (280600 GflopsGflops))

373763635 years ago5 years ago(4938 (4938 GflopsGflops))

1515858510 years ago10 years ago(170.4 (170.4 GflopsGflops))

Computer Computer ClusterCluster

Super Super ComputerComputer

Date and Max Date and Max GflopsGflops

SummarySummary

Computer clusters allow computation of Computer clusters allow computation of more complex problemsmore complex problemsComputer clusters enable researchers to Computer clusters enable researchers to achieve results in much less timeachieve results in much less timeComputer clusters allow sharing of Computer clusters allow sharing of expensive resources among multiple expensive resources among multiple researchersresearchers

AcknowledgmentsAcknowledgments

Dr. Polik Dr. Polik –– Research MentorResearch MentorDepartment of Chemistry, Hope CollegeDepartment of Chemistry, Hope CollegeResearch Corporation and Howard Research Corporation and Howard Hughes Medical InstituteHughes Medical InstituteNSFNSF