High Performance Computing

G Burton – ICG – Oct12 – v1.1

1

Agenda

• Commodity Clusters

• Compute Pool

• Interconnects and Networks

• Shared Storage

• Login Layer and Middleware

HPC alternatives

HPC Facts• IBM Sequoia - Number 1 in top

500 with 1.572 million Cores.• 20 petaFLOPS ( Floating Point

Operations / Second ). Sciama 10 teraFLOPS.

• China has Number 5 plus 62 other systems in top 500, now ahead of Germany, UK, Japan and France.

• 75% of Top 500 use Intel processors.

Demystifying the Techno Babble

Demystifying the Techno Babble (2)

Commodity Clusters

7

Commodity Clusters• Made from commodity (off-the-shelf)

components (read PC’s).

• Consequently (relatively) cheap.

• Usually Linux based

• High availability storage (no single point of failure)

• Generic compute pool (cloned servers that can easily be replaced).

Cluster Concept

10

Compute Pool - Just a bunch of PC’s

In the “good-ol-days” things were simple ……….

11

In the “good-ol-days” things were simple ……….

12

… these days much more packed into the same space … but basically the

same!

13

These are the building blocks of HPC similar to Sciama

Total ICG Compute Pool > 1000 Cores

14

Coke or Pepsi – Chalk & Cheese

• Only 2 remaining commodity CPU makers are Intel and AMD.

• Latest AMD “Bulldozer” architecture competing with Intel “Sandy Bridge” Architecture.

• Both architectures are multi core ( Intel 48 cores)• Architectures use same memory / video cards /

hard drives etc• CPU speed constraints down to on-chip

transmission delays and heat dissipation ( 22nm ).

Intel-AMD – Bangs per Buck

Graphical Processing Units (GPU’s actually GPGPU’s – General purpose)

CPU’s still in charge

Special programminglanguage. CUDA and OpenCL

Three players:-IntelAMDNvidia

Cpu – multiple coresGpu – 100’s of cores

Interconnects and Networks

18

Interconnects and Networks

Moving away from the Processor towards the Internet you get slowerand slower due to Increased Latency and Reduced Bandwidth

Processor Interconnects

For processor running at 3.2GHz – QPI Bus would be 25GBytes / Second ( Kindle version of “War & Peace” is 2GBytes)

Peripheral Component Interconnect Express (PCIe)

PCI bus is interconnect to the outside world

External Networks

Interconnects are Parallel – Bytes / secondNetworks are serial – bits / second(shown here in B/s for comparison – eg. DAS is 6Gb/s)

Sciama Network

Sciama Traffic

Connecting to Sciama

26

Shared Storage

Raw Disks are Dumb

Remember: PATA, IDE (Advanced Technology Attachment)

Intelligence is in the File System

HPC’s Require many disks

Use High Capacity Arrays

HPC’s require large chucks of storage

Many RAID options 1-6 / 10 /50

Directly Attached Storage (DAS)

Directly Attached Storage (DAS)

Of limited use as cannot be shared.

Network Attached Storage (NAS)

NAS or Network Appliance

Network BW is often the bottleneck

NAS - Lustre File System

Lustre is an example of a distributed file system. There are many more.

Sometimes called a “Cluster” file system

NAS – Lustre

Often used with an Infiniband fabric.

Storage Area Networks

Storage Area Network using iSCSI

Fibre Channel High Availability SAN

Sciama NAS Storage

Sciama Storage Hardware

Storage is expensive.

250gbp / Tbytes

No Backup

Highly Available Hardware

Two paths to most components

Additional Sciama Storage/mnt/astro(1-5)

10 GbE

Login Layer and Middleware

48

Why Login Servers• Login servers will provide the gateway to the cluster.• Users can remotely login into the servers using “ssh” or

a Remote Desktop Client.• A desktop client gives a full working desktop in the

environment (can full screen)

Some users are at remote locations ..

50

Use of Remote Login Client

51

Executable and Jobscript setup in Login Layer

Jobs submitted to the queues

> qsub –q Queue1 run_script.sh

Scheduler prioritises and deems a job ready to run.

Job passed to resource manager

Resource manager (Torque) checks for available resources.

Job either runs in the compute pool or returns to queue