I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

TransPACHPIIS Performance Review

Michael A. McRobbie

Vice President for Information Technology and CIOIndiana University

October 25, 2000

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Overview

• HPIIS, TransPAC and APAN• The TransPAC Cooperative Agreement• Progress toward objectives• Science enabled by TransPAC

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Goals of the HPIIS program

• US NSF Program for High Performance International Internet Services (HPIIS), solicitation NSF97-106.

• “The goal is to provide the basis for the development of next generation applications supporting international collaborations and enabling interaction with global information and experimental resources.”

• HPIIS connections link international high-performance production networks for international scientific, research and educational collaborations. Additionally, HPIIS networks enable global testbeds for new networking technologies.– e.g. multicast, IPv6, QoS and web cache

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Historical Overview

• April 1997 HPIIS solicitation• August 1997 IU’s TransPAC response• February 1998 TransPAC Proposal revised• August 1998 Cooperative Agreement

signed• August 1998 TransPAC operational

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Asia-Pacific Advanced Network (APAN)

• 14 member-nations cooperating to build a high performance research and education network across the AP region

• 5 primary members– Australia, Japan, Korea, Singapore, USA

• 2 associate members– China, Malaysia

• 7 affiliates, liaison and other members– Hong Kong, Indonesia, Thailand, Philippines,

Canada, Europe (Dante), Consultative Group on International Agricultural Research (CGIAR)

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

TransPAC: Connecting APAN to the US

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

STAR TAP Logical Map

TransPAC (OC-3)

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

TransPAC Objectives

• Provide leading edge connectivity between vBNS and APAN

• High throughput production network for research and education

• Testbed for new protocols, network services and applications

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

TransPAC Consortium

• Indiana University (lead US organization)• APAN (APAN-JP lead organization)

• National Science Foundation HPIIS program• Japan Science and Technology Agency (JST)

• Kokusai Denshin Denwa, Co. Ltd (KDD)• AT&T• Korea Telecom

• STAR TAP• NCSA and NPACI

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

TransPAC Cooperative Agreement

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Cooperative Agreement

• TransPAC operates with funding from – The NSF– the Japan Science and Technology

Agency – additional contributions from Indiana

University• Operation is controlled by NSF CA ANI-

9730201

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Terms of the Cooperative Agreement

• Primarily an agreement for services• 5 year term, renewable annually• Specific deliverables based on HPIIS

program objectives• General notion of cooperating with

other HPIIS funded projects• Review at the beginning of the third

year– Changes in R&E networks– Impact on science

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Cooperative Agreement Deliverables

1. Provide and manage a direct, high-speed network between the vBNS and APAN

2. Allow access only to HPIIS approved institutions3. Monitor the performance and use of the

TransPAC connection4. Cooperate with NLANR to develop and test new

Internet protocols5. Provide publicly-accessible information about

TransPAC and the projects it enables6. (a) In concert with NLANR provide consultative

user services and (b) front-end tools for direct access to differentiated network services

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

1. Provide and manage a direct, high-speed network between the vBNS and

APAN• Network infrastructure

– Chicago-Tokyo link provided through annually renewable contract with AT&T and KDD

– Peering at STAR TAP and the APAN Tokyo Exchange Point

– Infrastructure jointly funded by the NSF, JST• Network engineering

– Provided by IU, STAR TAP, and KDD staff– Funded by NSF, IU and KDD (in Tokyo)

• Network operation and information centers– US side: Indiana University (Global NOC)– APAN side: APAN/KDD Tokyo XP

• User support– US side: NSF and IU funded (IU and NLANR DAST staff)– APAN side: provided by the APAN community

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

155

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Highlights

• Network and support staff jointly funded by NSF, JST and IU

• TransPAC was up and on-line before the cooperative agreement was signed

• Partnership with JST, AT&T and KDD have led to continuous improvements in service levels without additional NSF funding

• Synergies within the HPIIS program and with Abilene/Internet2 have led to the Global NOC, run by Indiana University

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

TransPAC “No-Cost” Bandwidth Improvements

0

50

100

150

200

250

30008

/01/

97

11/0

1/97

02/0

1/98

05/0

1/98

08/0

1/98

11/0

1/98

02/0

1/99

05/0

1/99

08/0

1/99

11/0

1/99

02/0

1/00

05/0

1/00

08/0

1/00

11/0

1/00

02/0

1/01

05/0

1/01

Ban

dwid

th (

Mbp

s)

Initial 35Mbps

73 Mbps JSTupgrade (5/99)

AT&T consolidationupgrade (11/99)

OC-3 “southern route” (Q2/2001)

155 Mbps upgrade (10/00)

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Leveraging the HPIIS TransPAC CA

• “First” circuit– NSF: $2M annually for 5 years (US half circuit)– JST: $3M annually (est.) for 5 years (JP half circuit)

• “Second” circuit– JST: $2M annually through IU for 5 years based on

MOU (US half circuit)– JST: $3M annually (est.) for 5 years (JP half circuit)

• IU: $300K annually for 5 years• TOTAL: $51.5M over 5 years for $10M NSF investment

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

TransPAC Peering (STAR TAP)

• vBNS• Abilene• ESNet• NREN/NISN• EuroLink• MirNet

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

2. Allow access only to HPIIS approved institutions

• Policy based routing restricts TransPAC traffic to HPIIS authorized institutions only

• AP traffic segregation done at Tokyo XP

• At STAR TAP only authorized networks can peer with TransPAC

• Traffic segregation is effective

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

HPIIS Networks Acceptable Use Policy

• HPIIS networks are provided for non-commercial advanced research and education applications.

• Limited extensions are granted to the research arms of commercial organizations that participate in collaborative projects with non-commercial research and education institutions.

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

• General traffic monitoring done at STAR TAP and Tokyo XP with MRTG and OC3Mon with graphs available on the TransPAC web site

• BGP session monitoring at STAR TAP• Sysmon router exception logs available on-line• “Top talkers” display (from MirNet MADAS, soon)• Looking Glass servers available at both ends of

the network• Test workstations available in Chicago and Tokyo

for special monitoring applications

3. Monitor the performance and use of the TransPAC

connection

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

4. Cooperate with NLANR to develop and test new Internet

protocols• TransPAC participates in MBone, 6Bone and web

cache projects• TransPAC supports

– native IPv6 (application example: telemicroscopy)

– native multicast (many events and meetings are broadcast annually)

– APAN web cache• Production diffserv QoS in the works

– Several experiments and demonstrations in the US this year will help define production parameters

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

5. Provide publicly-accessible information about TransPAC and the

projects it enables

• Information avaiable through the TransPAC web site (www.transpac.org):– Monthly and annual reports– Utilization, current and historical record– Descriptions of major projects and applications– Network status– AUP and application procedures– Problem reporting and tracking– User support contact information– Application development and tuning

information– Engineering documents

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

6. In concert with NLANR provide:

• Consultative user services– DAST provides application tuning workshops for US TransPAC

users and consulting to resolve performance problems. – TransPAC engineering staff actively engaged in NLANR activities

including participation in 1999 and 2000 NLANR Joint Techs meetings

– NLANR operates TransPAC’s OC3mon at the STAR TAP and makes traffic data available in the DataCube (moat.nlanr.net/Datacube)

• Front-end tools for direct access to differentiated network services– Currently not a big issue due to bandwidth expansions (QoS by

over-provisioning)– TransPAC is a focal point for coordination between I2 QBone

effort and APAN QoS Working Group

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Enabling Science

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

HPIIS and TransPAC Expected Outcomes

• Better infrastructure for international scientific collaborations

• Qualitative and quantitative changes in research methodologies

• New modes of communication between individuals and groups

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Unexpected outcomes

• Global standardization in data handling, aggregation and distribution, e.g.– HEP: GriPhyN– Genomics: BioMirror and AP-BioNet– Astronomy: Sloane DSS

• Much lower barriers to the use of “world class” instruments (e.g. Osaka electron microscope)

• Greater interest in forming international collaborations

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Earth ObservationData and Information Access Link (DIAL)

DIAL is a web-based distributed system to search, access and visualize satellite remote sensing data for Global Change research. In collaboration with NASDA and other institutions, NASA has DIAL servers set up to distribute satellite remote sensing data. NASA and NASDA also collaborate on the Tropical Rainfall Measurement Mission (TRMM); 3D data is transferred from NASA to NASDA using TransPAC/APAN, processed and visualized for the web.

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

High Energy PhysicsJapan-US-Australia Collaboration in the Silicon Vertex Detector Project for the BELLE High Energy Physics

Experiment at KEK

The BELLE detector is the state-of-the-art detector to investigate CP violating phenomena with unprecedented precision at the KEK B meson factory. The CP (C=Charge conjugation, P=Parity) violation is a key to explain why the universe is dominated by the matter, not by the anti-matter. The primary goal of the BELLE detector is to identify the origin of the CP violation. The BELLE collaboration consists of more than 40 institutions from Japan, Korea, China, Taiwan, India, Russia, USA, Australia, and Europe.

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Scientific InstrumentsTrans-Pacific Telemicroscopy

Scientists at the Osaka University Research Center for Ultra High Voltage Electron Microscopy (UHVEM) and University of California San Diego National Center for Microscopy and Imaging Research (NCMIR) successfully use international advanced research networks to couple the world's largest and most powerful (3 million volt) transmission electron microscope at UHVEM to a remote-use computer pavilion set up at NCMIR.

http://www.npaci.edu/online/v3.10/telemicroscopy.html

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Scientific InstrumentsJapan-US Collaboration in the Sloan Digital Sky

Survey

Sloan Digital Sky Survey (SDSS) is a project to carry out imaging and spectroscopic surveys of half the northern sky using a dedicated, wide-field, 2.5-m telescope. The imaging survey with a large mosaic CCD camera will produce digital photometric maps of the sky in five color bands. These maps will be used to extract the position and various photometric parameters of about 100 million galaxies and close to the same number of stars. The SDSS is a collaborative project between the US and Japan involving seven US institutions and the Japan Promotion group (JPG). The JPG will produce merged pixel maps from flat-fielded data.

http://www.sdss.org/

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

DNA data has accumulated more rapidly than compute power so researchers must often exclude potentially infor-mative data to make statistical analysis practical. Utilizing the computationally intensive maximum-likelihood method of phylogenetic inference in a globally distributed collection of computational nodes, Indiana University, National University of Singapore and ACSys CRC in Australia have analyzed the DNA of cytoplasmic coat proteins, micro-sporidia, and cyanobacteria.

Distributed Computing and Data ResourcesMaximum Likelihood Analysis Of Phylogenetic

Data

http://www.indiana.edu/~rac/hpc/cp.html

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Bio-Mirror is a world bioinformatic public service for high-speed access to up-to-date DNA & protein biological sequence databanks. Mirror sites have been established in Australia, China, Japan, Korea, Singapore, Thailand, and the US. New data are propagated to all mirror sites as soon as they are made available, creating a uniform, universal and reliable data base for biological and medical research.

Global Distribution of Research DataBio-Mirror: Sequence & Bioinformatic data

http://www.bio-mirror.net/

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Advanced CollaborationSpace Physics and Aeronomy Research

Collaboratory

The Space Physics and Aeronomy Research Collaboratory (SPARC) is an NSF-sponsored community resource for the upper atmospheric and space sciences; operating 24 hours a day for scientific collaboration and access to real-time and archival data.http://sparc-1.si.umich.edu/sparc/central

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Conclusions

• HPIIS and TransPAC are excellent investments for US science that allow access to global facilities and expertise

• The HPIIS program has set new standards for infrastructure in network-enabled international scientific collaboration

• For many new projects this is critical infrastructure

• TransPAC has been able to deliver 4-5 times the capabilities expected for the NSF’s original investment

• Synergies between HPIIS projects built on the global network crossroads, the STAR TAP, have led to the establishment of an integrated global NOC at IU