Japan’s 10 Peta FLOPS Supercomputer Development Project ... · Japan’s 10 Peta FLOPS...

Ryutaro Himeno

Japans 10 Peta FLOPS Supercomputer Development Project and its energy saving designs

1) Group Director of R&D group, NGS R&D Center

2) Deputy Program Director for Computational Science

3) Director of Advance Center for Computer &

Communication

2

Contents

1. Self introduction2. Japans next generation supercomputer R&D project:

1. Before the project started2. Introduction and current status3. Energy issues

3. Introduction of focused application area

3

Who am I?

1985 1987

1988

1990

1992

1993

1995

Courtesy of Nissan

5

Hardware development at RIKEN

Special purpose computersIn 1989, GRAPE developed by U. TokyoMDM:MD-GRAPE2 develop by RIKENMDM was integrated in RSCC, 2004 MD-GRAPE3 developed in 20042nd fastest record on MD-GRAPE3: Gordon Bell Prize, 2006.MD-GRAPE-3 will be integrated in RSCC soon.

ASCI Red

ASCI BlueMountain

ASCIBlue Pacific ASCI White

ASCI Q

ASC Purple

BlueGene/L

100Tera

1Tera

10Tera

Theoretical Peak Performance

[year]

1Peta

installed

Planed

*ASCI: Accelerated StrategicComputing Initiative

'96 '98 '00 '02 '04 '06 '08 '10

10Peta

100Peta

RSCC(Riken)

GRAPE-DR(U.Tokyo/RIKEN)

CP-PACS

HPCSNLCF

41TeraFLOPS

367TeraFLOPS

Japan US

PACS-CS(Tsukuba)

ASC Red Storm

BlueGene/Q

Earth Simulator

MD-GRAPE3(RIKEN)

Tsubame(TITech)

1PetaFLOPS

MD-GRAPE2

MD-GRAPE3

6

512Nodes (1024CPUs)

Cluster

128Nodes (256CPUs) Cluster4

Intel Xeon 3.06GHz2GB/Node,Myrinet/IB,1.56TFLOPS4

Intel Xeon 3.06GHz4GB/Node, IB6.26TFLOPS

NEC SX-7/32282.5GFLOPS256GB

Tape Storage200 TB

POWER4 Server: 7Tape Drive: 4 (16)Disk Cache: 2TB

HDD20 TB

Itanium2 Server: 2

Vector Parallel

UserUser User

User

Front endServer

Myrinet

InfiniBand

InfiniBand

Current RSCC System installed in 2004 at Riken Advance Center for Computing & Communication

MD-GRAPE364TFLOPS

7

Basic concept of RSCC

Each computer architecture has each advantage and disadvantageThere is no single architecture which fits wide variety of applicationsRSCC is composed of 3 subsystem, Linux cluster with Xeon, Vector Parallel computer and MD-GRAPE3

We made a proposal of the NGSP based on We made a proposal of the NGSP based on this concept to MEXTthis concept to MEXT

8

Our ProposalOur Proposal

Needs of Multi-scale Multi-

physic simulation

Integration of multiple

architecture

Tightly-coupled heterogeneous computer

Needs of multiple

computation components

SwitchPresentPresent

Faster interconnect

Vector Node

Scalar Node

MD Node

Slower connectionFaster interconnect

Faster interconnect

Vector Node

Scalar Node

FPGA Node

MD Node

Faster interconnect

Proposing architecture

9

Policy and Outline of A Next Generation Supercomputer Project

Purpose of policy:development, installation and application of an advanced high performance supercomputer system, as one of Japans Key Technologies of National Importance

Total Budget: about 115 billion Yen ( 1.15 billion US dollars )

Period of Project: FY2006 FY2012

9

10

Formation of the 10 Peta R&D Project

Evaluation Scheme

EvaluationCommittees

Industries

Industrial Forumfor Promotion

of Supercomputing

R&D Scheme

Advisory

Board

Computer Companies

RIKEN: Project HQNext-Generation

Supercomputer R&D Center (Ryoji Noyori)

NII: Grid Middleware and Infrastructure

IMS: Nano Science Simulation

Riken Wako Institute: Life Science Simulation

(Note) NII: National Institute of Informatics, IMS: Institute for Molecular Science

Project Leader: Tadashi Watanabe

MEXT: Policy & FundingOffice for Supercomputer Development Planning

Project Committee

(MEXT and CSTP)

Visiting Researchers from Univ. & National Labs.

R&D G.: Ryutaro Himeno

11

Goals of the Next Generation Supercomputer Project

1. Development and installation of the most advanced

high performance supercomputer system

2. Development and wide use of application software to

utilize the supercomputer to the maximum extent

3. Provision of flexible computing environment by sharing

the next generation supercomputer through connection

with other supercomputers located at universities and

research institutes

4. Establishment of Advanced Computational Science

and Technology Center (tentative name)

11

12

1P

100G

10T

100P

Sustained Performance (FLOPS)

2000 2010 2015 2020

The Next GenerationSupercomputer Project

10T

1990

PersonalEntertainment

PC, Home ServerWorkstation,

Game Machine, Digital TV

EnterpriseCompany,Laboratory

National Infrastructure

Institute,University



InstituteUniversity






1P

100G

100PNationalLeadership

The Next Generation Supercomputer

NationalLeadership

(Earth Simulator)

NationalLeadership CP-PACS,

NWT




Government Investment

Earth Simulator Project

2025

EnterpriseCompany, Laboratory






NationalLeadership

NationalLeadership

Next-next Generation

Project

Next-next-next

GenerationProject

Expansion of Highest Computer for Global Usage

13

FY2008 FY2009 FY2010 FY2011

Computerbuilding

Researchbuilding

FY2007FY2006 FY2012

Shared file system

Processing unit

Front-end unit(total system software)

Next-GenerationIntegrated NanoscienceSimulation

Next-GenerationIntegratedLife Simulation VerificationVerificationDevelopment, production, and evaluationDevelopment, production, and evaluation

Tuning and improvementTuning and improvement

VerificationVerification

Production, installation, and adjustmentProduction, installation, and adjustment

Production, installation, and adjustment

Production, installation, and adjustment

ConstructionConstructionDesignDesign

ConstructionConstructionDesignDesign

Prototype andevaluationDetailed designDetailed design

Conceptualdesign

Detailed designDetailed designBasicdesignBasicdesign

Development, production, and evaluationDevelopment, production, and evaluation

Production and evaluation

System

Bu

ildings

Detailed designDetailed designBasicdesignBasicdesign

Schedule of Project

Application

s

present

14

Optimization of Computer Design

Technology LimitSpin off to the consumer electronics

Other Project Watch

Requirement from Grand Challenges

Nano Sci. Life Sci. Others(Engineering etc)

Technology Survey

LSI process

Power savvingSOI

Low-kHigh-k

Light transmit.tech.

SoftwareOS, compiler///

FMO

RISM

MO

Popular numerical scheme

Linear Solver

FVM

FEM

Requirement from Computer Centers

0

200

400

600

800

1000

1200

1400

0 500 1000 1500 2000 2500 3000 3500

kW

)

Power, Space

Reliability, operability

Cost (development, manufacturing, maintenance

Computer center in Japan

Optimum system

Fastest in the world

Operation & Utilization

Essential Element technologies

Real space density function

method

Fast transform

Monte Carlo

Searching algorism

Life Science6

Engineering.4

Env./ Desasterprev. 3

Nano Science6)

Physics, Astro.2

21 Selected Target applications

15

Target applications and BM suitThe application software committee started selecting target applications for making a benchmark suit in January, 2006.

21 codes were selected in March and NextBMT suit was developed.

16

NextBMT and Peta-scale BMT(1/2)

Coarse-Grained Molecular Dynamics CalculationOcta

First-Principles Molecular Dynamics Simulation within the Plane-Wave Pseudopotential formalismPHASE

Analysis of Electron States of Proteins in Solution with the 3D-RISM/FMO MethodRISM/3D-RISM

Ab-initio Molecular Dynamics Calculation in Real SpaceRSDFT

Massively-parallel multipurpose software for molecular dynamics calculationModylas

FMO Molecular Orbital CalculationGAMESS/FMO

Nono(6)

Ab-initio Molecular Dynamics Calculation in Large-Scale Protein SystemsProteinDF

Docking simulation of protein and drugsievgene/myPresto

Simulation for blood-flow analysisMC-Bflow

Validation of statistical significance for development of individualized medicineMLTest

Inference of Genetic Networks from Experimental Data of Gene ExpressionGNISC

Prediction of Protein StructureSimFold

Life (6)

DescriptionProgram nameFields

Peta-scale BMT

17

NextBMT and Peta-scale BMT(2/2)

Unsteady Flow Analysis based on Large Eddy Simulation (LES)FrontFlow/Blue

Structural Calculation by Finite Element Method (FEM)FrontSTR

Computation of compressible fluid in aircraft and spacecraft analysisLANS

Computation of Unsteady Cavitation Flow by the Finite Difference MethodCavitation

Engineering(4)

Super-resolution Ocean General Circulation ModelCOCO

Simulation of Seismic-Wave Propagation and Strong Ground MotionsSeism3D

Nonhydrostatic ICosahedral Atmospheric Model (NICAM) for Global-Cloud Resolving Simulations

NICAM

Geophysics (3)

Super large-scale gravitational many-body simulation for finding the celestial origin

NINJA/ASURA

Study of elementary particle and nuclear physics based on Lattice QCD simulation

LatticeQCDPhysics

Astronomy(2)

Peta-scale BMT

18

System Configuration

The Next-Generation Supercomputer is designed as hybrid general-purpose supercomputer that provides the optimum computing environment for a wide range of simulations.

Calculations will be performed in processing units that are suitable for the particular simulation.Parallel processing in a hybrid configuration of scalar and vector units will make larger and more complex simulations possible.

19

Vector unit is suitable for continuum physics simulationScalar unit is suitable for Data Base application, particle-based simulationHybrid system has both strong points for users to choose better units for different

applications

Scalar UnitData base, homology searchSimilarity with PC

high speed low power CPU

New strong network for enormous parallelism

Vector UnitVector processor based on the Earth Simulatorstrong for continuum physics simulation

New generation low-power vector processor with optical network

Hybrid Supercomputer System composed of Scalar and Vector Units

Front end computer(Integrated OS)

Scalar Unit Vector Unit

Nono device

Local fileGlobal file

Control unitProcessing unit

Local file

Disk unit

Solar cell

Plazma Physics

Climate change CFD

- -e- e-I3-

I -

Characteristics

Homology search

20

For Energy Saving,

Cogeneration system: gas turbine power generator & steam generator using natural gas

Generates electric power and steamSteam is used in cooling system which makes child waterChild water is used in two ways

Water-air heat exchangerDirectly provided to computer

Clock frequency is decided from energy saving point of view

21

Cogeneration system: 2 points of Saving Energy

Total energy efficiency of cogeneration system is better than ordinary commercial electric power plant

Using natural gas as fuel, CO2 will be reduced at the same time

Direct water cooled computer system can save pumping loss of air blowing for cooling

Full water cooled system has difficulty in maintenanceWe will use partial water cooling system

22

Photographs of the facilites

2009/02/06

2009/04/25

2009/03/04

23

Major Applications of Next Generation Supercomputer

Targeted as grand challenges

24

Next-Generation Energy

Solar energy fixation

Fuel alcoholFuel cells

Electric energystorage

Electrons and

moleculesElectrons

Domain

Electron theory of solids

Quantum chemistry

Doping of fullerene and carbon nanotubesMolecular

dynamics

Condensed matters

Integrated system

5nmSelf- organized

magnetic nanodots

Semi-macroscopic

Molecular assembly

Next-Generation Nano Biomolecules

Next-Generation information Function Materials

One-dimensional crystal of silicon

Polio virus

Orbiton(orbital waves)

Ferromagnetic half-metals

off on

lightlightOptical switch

Liposome

Nafion

Water

Nafion

Water15nm

Mesoscale structure of naflon membrane

Self-assembly Capsulation

Nafion membrane

Medicines, New drug, and DDS

RMSD 4.8 (all C)

Protein folding

Nonlinear optical DeviceNano quantum devices

Spin electronicsUltra high-density storage devices

Integrated electronic devices

Water molecules inside lisozyme

cavity

Electronic conduction in

integrated systems

VirusesAnticancer drugsProtein control

Nano processes for DDC

light

27 nm

46 nm

27 nm

46 nm

To create next-generation nano-materials(new semiconductor materials, etc.) by integrating

theories (such as quantum chemistry, statistical dynamics and solid electron theory) and - simulation techniques in the fields of new-generation information functions/materials, nano-biomaterials, and energy

Nano-Science Simulation Software

25

To provide new tools for breakthroughs against various problems in life science by means of petaflops-class simulation technology, leading to comprehensive understanding of biological phenomena and the development of new drugs/medical devices and diagnostic/therapeutic methods

Whole bodyCardiovascular system

Cells OrgansTissues

Micro MacroMeso

Microscopic approach

MD/first principle/quantum chemistry simulations Continuous entity simulations

Size

26

Total Simulation of Living Matter

GoalUnderstand the life phenomena: reproducing life program on computerDevelop new medicine and medical equipment

Approachcombination of analytical approach and Data-driven approach on Peta-scale computer Data-Based

approach to analysisAnalytical

approach

Molecule scale

Cell scale

Organ/Whole body scale

Basic principal (Equations)

Experimental data

New methodology

Understand the life phenomena: reproducing life program on computerDevelop new medicine and medical equipment

Fusion of Data based approach and Anaytical approach

Peta scale simulation

27

Cyber Science Infrastructure Plan

Project VO

Virtual research environment for various fields

Next Generation Supercomputer

s ( Virtual Organizations

Infrastructural middlewareGRIDInfrastructure for certification, etc.)

Industrial project VO

University/inter-university research institutes VO

Operated by National Institute of Informatics (NII)

(Note) VO :Virtual Organization 27

28

CACST: Center for Advanced Computational Science and Technology (tentative name)

Computer science and Computational scienceComputer science and Computational scienceBoth researchers will gather and expect to develop new Both researchers will gather and expect to develop new research fields and methodologiesresearch fields and methodologiesCurrently, we are designing the center and operation Currently, we are designing the center and operation policy of the supercomputerpolicy of the supercomputer

The users will be chosen by a new committee The users will be chosen by a new committee independent from RIKEN to pick up valuable subjectsindependent from RIKEN to pick up valuable subjects

Kobe Port-Island

29

COE

Nono

Astro

Nuclear

Life

Geo

Eng.

Physics

Meteology

CACSTIndustrial use

Computer Science

Tentative Plan of the computer center

Hub of the Computer Science and Computational Science in Japan and Asia.

Cross point of various sciences and technologies using HPCOperation of 10-Peta FLOPS SupercomputerUser support and spreading the usage in various fieldDevelopment of the next-next supercomputerResearch on computational science Education

30

Summary

From energy saving point of view, gas cogeneration system is adoptedFor saving pumping loss of air handling, water cooling system is partially adoptedWe are focusing on application development in Nanoscience and Life scienceCurrently all development of hardware and software are on schedule

Japan’s 10 Peta FLOPS Supercomputer Development Project ... · Japan’s 10 Peta FLOPS...

Documents

Transcript of Japan’s 10 Peta FLOPS Supercomputer Development Project ... · Japan’s 10 Peta FLOPS...