4 October 2001Smith/Germain/Loiacono1 Earth Observing System: Measuring the Benefit of Internet2 for...
-
Upload
francis-mitchell -
Category
Documents
-
view
225 -
download
0
Transcript of 4 October 2001Smith/Germain/Loiacono1 Earth Observing System: Measuring the Benefit of Internet2 for...
4 October 2001 Smith/Germain/Loiacono 1
Earth Observing System: Measuring the Benefit of Internet2
for Science Data Distribution
Jeff SmithJoe LoiaconoAndy Germain
NASA EOS - GSFC
4 October 2001 Smith/Germain/Loiacono 2
Outline
• EOS Overview– EOS Program– EOS Data and Information System
• Network Measurements– Data Collection
• Abilene Traffic Loads• Abilene Utilization• Server Statistics
– Active Testing
• Future Plans
4 October 2001 Smith/Germain/Loiacono 3
The Earth Observing System (EOS) is the centerpiece of NASA's Earth Science Enterprise (ESE). It consists of a science component and a data system supporting a coordinated series of polar-orbiting and low inclination satellites for long-term global observations of the land surface, biosphere, solid Earth, atmosphere, and oceans. By enabling improved understanding of the Earth as an integrated system, the EOS program has benefits for us all. The EOS Project Science Office (EOSPSO) is committed to helping bring program information and resources to program scientists and the general public alike.
EOS Overview
4 October 2001 Smith/Germain/Loiacono 4
•Terra Mission - launched December 1999The Terra (formerly AM-1) satellite is the flagship of NASA's ESE (Earth Science Enterprise). Terra is the first EOS platform and provides global data on the state of the atmosphere, land, and oceans, as well as their interactions with solar radiation and with one another.
•Aqua Mission - rescheduled: early 2002Aqua will collect climate-related data. Its on-board sensors will measure clouds, precipitation, atmospheric temperature/moisture content, terrestrial snow, sea ice, and sea surface temperature during its six-year mission. Flying in low-earth orbit, Aqua will pass over the equator at the same time each afternoon, allowing the comprehensive measurements needed to assess long-term change.
•Aura ICEsat Mission - scheduled: July, 2003The EOS - Chemistry satellite hosts a suite of scientific instruments designed to make the most comprehensive measurements ever undertaken of atmospheric trace gases. Its objective is to study the chemistry and dynamics of the Earth's atmosphere with emphasis on the upper troposphere and lower stratosphere (0-30km). The mission will measure ozone, aerosols, and several key atmospheric constituents that play an important role in atmospheric chemistry, air quality, and climate.
Current and Upcoming EOS Missions
4 October 2001 Smith/Germain/Loiacono 5
EOS Missions
4 October 2001 Smith/Germain/Loiacono 6
EOS Platforms, Instruments, and Science Objectives
4 October 2001 Smith/Germain/Loiacono 7
The EOS Data and Information System (EOSDIS) is designed to operate the satellites and instruments, capture the satellite data, generate useful Earth science data products, and make the products available to users in the U.S. and throughout the world. The two major parts of EOSDIS are Mission System and Science System. The Mission System commands and controls EOS spacecraft and instruments, monitors their health and safety, and performs mission planning and scheduling, initial data capture, and initial data processing. The Science System generates higher level data products suitable for analysis by NASA principal investigators and the general scientific community.
EOSDIS - EOS Data Infrastructure
4 October 2001 Smith/Germain/Loiacono 8
NASA EOS Satellitesand Instruments
Other NASA Satelliteswith EOS Instruments
ESA Satellites withEOS Instruments
NASDA Satellites withEOS Instruments
EOSDIS
Commuications
NSIDC
ASF
EDC
GSFC
JPL
LaRC
ORNL
External
Communications
ValueAdded
Providers
Investigators
InstrumentSupport
Terminals
ScienceComputingFacilities
ExternalDataUsers
ScienceComputingFacilities
DistributedActive Archive
CentersEMSnet
ESSnet
EDOS
EBnet
EOSDIS Enterprise
4 October 2001 Smith/Germain/Loiacono 9
Mid-AtlanticCrossroads
EOSDIS Network Architecture
vBNS+
Chi
NISN
4 October 2001 Smith/Germain/Loiacono 10
Data Product Levels• Raw Telemetry - as transmitted from spacecraft
– Instruments multiplexed, CCSDS frames• Level 0: Instrument telemetry
– Time stamped, Reed-Solomon decoded, fill data, etc.• Level 1: Instrument Measurements
– Calibrated, georeferenced• Level 2: Scientific Parameter Retrieval
– Inferred geophysical properties– Corresponding to individual instrument measurements
• Level 3: Gridded– Interpolated / extrapolated to useful space/time coordinates
• Level 4: Integrated– E.g., Multiple instruments, movie loops
NET DAAC QA SCF
EOS 9-1-01
LaRC
4 October 2001 Smith/Germain/Loiacono 12
Data Collection Overview
• Data Collected– Traffic loads on key interfaces– Traffic volumes overall– Traffic volumes to sites– Traffic per host (sender & receivers - not included)– Traffic characteristics (ports, types - not included)
• Purposes– Historical data and trends vs. requirements– Planning– Troubleshooting– Security– Who’s doing what to whom on the EOS network
4 October 2001 Smith/Germain/Loiacono 13
GSFC DAAC Router interface to Abilene
- Science data product transfers from GSFC to Principal investigator sites located at Abilene member universities continues to grow.
- Note: Traffic loading on other key interfaces is not included in presentation
Abilene Traffic Loads
4 October 2001 Smith/Germain/Loiacono 14
EROS Data CenterSioux Falls, SD
NSIDCColorado Springs, CO
LARCLangley, VA
GSFCGreenbelt, MD
NetFlow Collector: cflowd
ARTS reports and Custom Scripts
EOS Performance Web-site
- Cisco NetFlow data is sent from EOS routers at above locations to ‘cflowd’ collector at GSFC every few seconds throughout the day. ARTS and custom scripts are runautomatically every night producing web-based reports. Data is compressed andarchived every week. * ‘cflowd’ and ARTS software is maintained by CAIDA.
Utilization Data Collection Process
4 October 2001 Smith/Germain/Loiacono 15
- EOS use of Abilene for science data product transfers from GSFC to Abilene partner institutions is significant
Daily Quantities of EOS Science Data to Abilene Partners
Abilene Science Data Utilization
4 October 2001 Smith/Germain/Loiacono 16
Top EOS Partner Institutions using Abilene
- Custom scripts that interpret Cisco’s NetFlow data keep track of major EOS Partners using Abilene to efficiently retrieve data from GSFC
Top Users of Traffic to Abilene
4 October 2001 Smith/Germain/Loiacono 17
EOS Science data products delivered to U. Miami over Abilene
- University researchers use Abilene for efficient retrieval of large science data sets. U Miami is largest single EOS partner institution using Abilene to retrieve science data products from GSFC, often extracting over 50 Gigabytes daily
Utilization for Single Abilene Site
4 October 2001 Smith/Germain/Loiacono 18
Abilene Statistics for router: GSFC
This Abilene Flow Distribution Report was generated at 09/04/2001 06:15:00 for Router: gsfc
Total Bytes to Abilene : 173,127,413,068 Pct.: 99Total Bytes to NSI Router: 670,153,399 Pct.: 0
Organization Name Network Science Bytes Pct. Test Bytes PI Total
University of Miami 129.171.0.0 122,163,427,632 96 2,098,222,247 124,261,649,879University of Wisconsin 144.92.0.0 2,543,093,993 2 187,598,416 2,730,692,409Oregon State University 128.193.0.0 2,157,704,001 2 1,779,053,008 3,936,757,009NCAR 128.117.0.0 131,818,168 0 4,936,226,700 5,068,044,868ECS (2) 192.107.195.0 5,145,462 0 5,145,462University of Alaska 137.229.0.0 3,497,758 0 3,497,758NASDA 133.56.0.0 2,447,226 0 2,447,226University of Maryland (2) 128.8.0.0 1,783,506 0 6,000 1,789,506ECS 198.118.203.0 1,667,795 0 5,880 1,673,675University of Arizona 128.196.0.0 1,627,629 0 4,208,195,612 4,209,823,241JPL 137.78.0.0 1,121,335 0 2,604,571,172 2,605,692,507UC Santa Barbara (2) 128.111.0.0 1,071,281 0 787,782,556 788,853,837University of Montana 150.131.0.0 1,070,530 0 525,665,836 526,736,366University College (SCF) 128.40.25.0 938,880 0 357,299,280 358,238,160Ohio State University 128.146.0.0 805,896 0 620,255,536 621,061,432Boston University 128.197.75.0 746,384 0 7,031,575,652 7,032,322,036ECS (3) 192.102.216.0 391,139 0 391,139University of Alabama (UAH) 198.116.0.0 215,336 0 215,336UMBC 130.85.0.0 115,441 0 115,441University of Colorado 128.138.0.0 4,123 0 4,123UC Berkeley 128.32.0.0 398 0 398University of Hawaii 128.171.0.0 286 0 286University of Delaware 128.175.0.0 72 0 72
Total PI data flows (see above) 127,018,694,271 90 25,136,457,895 152,155,152,166
Non-PI data flows (various) 13,363,794,346 9 7,608,466,556 20,972,260,902
Total Abilene bytes 140,382,488,617 100 32,744,924,451 173,127,413,068
PI Bytes as pct. of total (Abilene and NSI) 127,018,694,271 90
Abilene Daily Report
4 October 2001 Smith/Germain/Loiacono 19
Customized ARTS Network Report
• Network Statistics for router: GSFC
• router: 192.43.240.53• ifIndex: User Network (Fddi1/0/0 198.xxx.yyy.65)• period: 09/25/2001 19:58:57 - 09/26/2001 19:58:41 EDT• Src Network Pkts Bytes Destination• ------------------ -------------- -------------- -----------------------------------------------------------• 198.xxx.yyy.64/27 36601161 54863498565 198.116.57.0/24 -- NSI Netblock• 198.xxx.yyy.64/27 33690523 50462935052 129.171.96.0/19 -- University of Miami (NET-MIAMI)• 198.xxx.yyy.64/27 21579829 31970765873 128.149.0.0/16 -- NASA Headquarters (NET-JPL-NET)• 198.xxx.yyy.64/27 9995871 14983779816 129.171.97.106/32 -- miracle.rsmas.miami.edu -- University of Miami (NET-MIAMI)• 198.xxx.yyy.64/27 8920265 12661046304 192.67.108.0/24 -- National Aeronautics and Space Administration (NETBLK-NASA-
CN• 198.xxx.yyy.64/27 3921594 5846242247 128.183.0.0/16 -- NASA Goddard Space Flight Center (NET-GSFC)• 198.xxx.yyy.64/27 2467372 3690544592 129.116.0.0/16 -- University of Texas (NET-CHPCHYPERHOSE)• 198.xxx.yyy.64/27 2428576 3630351822 132.239.0.0/16 -- University of California, San Diego (NET-UCSD-SUBNETS)• 198.xxx.yyy.64/27 2345597 3500985134 128.197.0.0/16 -- Boston University (NET-BU-NET)• 198.xxx.yyy.64/27 2219596 3307136570 137.78.0.0/16 -- Jet Propulsion Laboratory (NET-JPL-NET2)• 198.xxx.yyy.64/27 2201514 3290519498 128.117.0.0/16 -- National Center for Atmospheric Research (NET-UCAR)• 198.xxx.yyy.64/27 1851848 2766130262 128.196.0.0/16 -- University of Arizona (NET-UNIV-ARIZ)• 198.xxx.yyy.64/27 1376872 2065165027 128.111.100.207/32 -- taichi.icess.ucsb.edu -- University of California,
Santa )• 198.xxx.yyy.64/27 1349216 2004394808 144.92.0.0/16 -- MACC (NET-UWMSN-NET)• 198.xxx.yyy.64/27 1071927 1607363478 138.194.0.0/16 -- CSIRO IT Services (NET-CSIRO-CLAYTON)• 198.xxx.yyy.64/27 963529 1445049266 128.49.0.0/16 -- Naval Ocean Systems Center (NET-NOSC-ETHER)• 198.xxx.yyy.64/27 949036 1423408199 128.110.0.0/16 -- University of Utah (NET-UTAH-NET)• 198.xxx.yyy.64/27 934161 1388544068 128.193.0.0/16 -- Oregon State University (NET-ORST)• 198.xxx.yyy.64/27 876516 1303285172 128.111.0.0/16 -- University of California, Santa Barbara (NET-UCSB)• 198.xxx.yyy.64/27 617633 876607883 192.67.108.14/32 -- matrix4.nsstc.uah.edu -- National Aeronautics and Space• 198.xxx.yyy.64/27 659790 730631285 192.107.190.0/24 -- NASA Ames Research Center (NET-EOS-DAAC-NET2)• 198.xxx.yyy.64/27 474613 700318390 198.118.203.0/24 -- NSI Netblock• 198.xxx.yyy.64/27 312384 468120358 128.138.0.0/16 -- University of Colorado (NET-COLORADO)• 198.xxx.yyy.64/27 258856 376589356 128.40.0.0/16 -- University College London (NET-UCL-NET-A)• 198.xxx.yyy.64/27 198689 297880544 216.17.192.50/32 -- 216.17.192.50 -- Front Range Internet, Inc. (NETBLK-NET-
RII-• 198.xxx.yyy.64/27 199406 288131072 128.146.0.0/16 -- Ohio State University (NET-OHIO-STATE)
4 October 2001 Smith/Germain/Loiacono 20
Sample from Custom Flow Report
• Detailed Flow Data for External Destinations
• Source Port Bytes Destination• -------------------------------------------------------------------------------------------------• 198.xxx.yyy.147 389D 2930 memberdir.netscape.com -- Netscape Communications Corp. (NETBLK-• 198.xxx.yyy.148 80D 91401 (web requests out) • 198.xxx.yyy.149 443D 6012 (secure web in) • 198.xxx.yyy.149 80D 848446 (web requests out) • 198.xxx.yyy.151 443D 67601 (secure web in) • 198.xxx.yyy.151 80D 608132 (web requests out) • 198.xxx.yyy.153 21D 2382 (ftp control out) • 198.xxx.yyy.153 20D 137368 (ftp sent out) • 198.xxx.yyy.153 80D 3984705 (web requests out) • 198.xxx.yyy.153 137D 1872 128.161.3.12 -- NASA Ames Research Center (NET-NSN-NET)• 198.xxx.yyy.153 137D 1872 128.161.7.33 -- NASA Ames Research Center (NET-NSN-NET)• 198.xxx.yyy.153 0D 720 polaris.umuc.edu -- University of Maryland University College• 198.xxx.yyy.153 2048D 22500 polaris.umuc.edu -- University of Maryland University College• 198.xxx.yyy.153 22D 352 polaris.umuc.edu -- University of Maryland University College• 198.xxx.yyy.153 2048D 912 www.cisco.com -- Cisco Systems, Inc. (NET-SWLAB1)• 198.xxx.yyy.153 2048D 1794 u1.abs.net -- ABSnet Internet Services, Inc. (NETBLK-ABS-NET-• 198.xxx.yyy.153 110D 1931 mail1.abs.net -- ABSnet Internet Services, Inc. (NETBLK-ABS-NET-• 198.xxx.yyy.153 1259D 13364 idyle.com -- Cable & Wireless USA (NETBLK-CW-10BLK) CW-10BLK• 198.xxx.yyy.153 1975D 4066 aim1.radiate.com -- Abovenet Communications, Inc. (NETBLK-• 198.xxx.yyy.153 137D 234 so-4-1-0.mp2.Washington1.Level3.net -- Level 3 Communications, • 198.xxx.yyy.153 2048D 5148 p2.my.vip.snv.yahoo.com -- GeoCities (NETBLK-NETBLK-GEO-1)• 198.xxx.yyy.153 2786D 252444 ns1.solarwinds.net -- Southwestern Bell Internet Services• 198.xxx.yyy.153 137D 468 so-2-0-0.mp2.SanJose1.Level3.net -- Level 3 Communications, Inc. • 198.xxx.yyy.153 137D 468 gigabitethernet9-0.ipcolo1.SanJose1.Level3.net -- Level 3• 198.xxx.yyy.154 80D 182348 (web requests out) • 198.xxx.yyy.156 23D 176 polaris.umuc.edu -- University of Maryland University College• 198.xxx.yyy.156 110D 9151 eos.east.hitc.com -- Performance Systems International (NET-
4 October 2001 Smith/Germain/Loiacono 21
Server Statistics• http://m0mss01.ecs.nasa.gov/smc/vrdash/
4 October 2001 Smith/Germain/Loiacono 22
Active Testing Overview• End-to-end user level test
– “Active” testing– little or no visibility into network internals
• Purposes– EOS: assess whether networks as implemented meet
EOS requirements– CEOS: assess whether networks can support intended
applications– Resolve user complaints:
• Network problems -- or somewhere else??
– Determine bottlenecks -- seek routing alternatives– Provide a basis for allocation of additional resources
4 October 2001 Smith/Germain/Loiacono 23
Active Testing - Process
• Test script runs hourly to each site:• Traceroute (1 way)
– Number of hops -- route stability --> Hops Chart
• Pings – 100 pings prior to thruput test or 100/300 during– Round Trip Time --> RTT Chart– Packet Loss --> Packet Loss Chart
• TCP Throughput – TCPwatch, Iperf, or FTP --> Thruput Chart– requires cooperating client at destination– TCPwatch / Iperf keeps send buffer full for 30 Seconds– Netstat -> packets retransmitted (if pings blocked)
4 October 2001 Smith/Germain/Loiacono 24
EOS DAAC
NASA Nodes
SCFsQA Other
Key:
ORST
UCSB
ArizLANL
Wisc
Miami
SUNY-SB
BU
GSFC
LaRC
EDC
MSFC, NSSTC
NCAR
Mont
JPL
Toronto
Colo St.
Niagara
ASF
Chicago
Other Nodes
SLAC
NSIDC
NMEX
CCRS
UVA
UMD
GPNNGDC,NOAA
USF
RSS
US Active Test Sites
Texas
UCSD
Wash
Mich
NOAA
Ohio
4 October 2001 Smith/Germain/Loiacono 25
On line results
• Updated Daily– http://corn.eos.nasa.gov/networks– "Network Health Statistics"
• GSFC Sites• EOS Sites• EMSNet Sites• CEOS Sites• Other sites
– Charts: All sources to specified destination• Thruput, RTT, Packet Loss, Hop Count
– Weekly: 24 measurements per day from each source– 60 day: Daily Average– Yearly (RSN)
4 October 2001 Smith/Germain/Loiacono 26
Sample Chart: BU Thruput (current week)
4 October 2001 Smith/Germain/Loiacono 27
Sample Chart: BU RTT, Loss (current week)
4 October 2001 Smith/Germain/Loiacono 28
Sample Chart: BU Thruput (2 months)
4 October 2001 Smith/Germain/Loiacono 29
Sample Chart: UCSD Thruput (current week)
4 October 2001 Smith/Germain/Loiacono 30
Sample Chart: UCSD Thruput (2 months)
0.1
1
10
100
Mb
ps
Minimum
Maximum
Median
Thruput Range: GSFC to Abilene Sites
AZ
, T
ucs
on
(U
of
AZ
)
CA
, U
CS
B
CA
, U
CS
D -
SIO
CO
, C
olo
Sta
te U
niv
CO
, N
CA
R -
Bo
uld
er
FL
, U
niv
. o
f M
iam
i
MA
, B
ost
on
Un
iv
MD
, U
MD
-C
oll
ege
Par
k
MT
, U
niv
of
Mo
nta
na
NY
, S
UN
Y S
ton
y B
roo
k
OH
, O
hio
Sta
te U
niv
OR
, O
reg
on
Sta
te U
niv
.
TX
, U
Tex
as-
Au
stin
VA
, U
VA
4 October 2001 Smith/Germain/Loiacono 32
POCs
• Jeff [email protected]
• Joe Loiacono [email protected]
• Andy [email protected]