Tara Villanueva Andrew Vanderslice Tony Kratz ZAVELLE BOOKSTORE.
Cyberinfrastructure for Environmental Observationsceoa.ucsd.edu/docs/Smarr-Nov05Symposium.pdfUsed by...
Transcript of Cyberinfrastructure for Environmental Observationsceoa.ucsd.edu/docs/Smarr-Nov05Symposium.pdfUsed by...
"Cyberinfrastructure forEnvironmental Observations"
Invited Talk to Symposium on
Science and Technology in GEOSS: The Role of Universities
Hosted by Calit2@UCSD
for the SIO Center for Earth Observations and Applications
La Jolla, CA
November 21, 2005
Dr. Larry Smarr
Director, California Institute for Telecommunications andInformation Technology
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
Components of a Future Global System for Earth Observation(Sensor Web)
Vision for Creating an Integrated InteractiveInformation System for Earth Exploration
Major Obstacle: Trying to Do Global Earth SciencesOn a Shared Internet Designed for Email and FTP
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,0002
00
1
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
20
11
20
12
20
13
20
14
Calendar Year
Cu
mu
lati
ve T
era
Byte
s
Other EOS
HIRDLS
MLS
TES
OMI
AMSR-E
AIRS-is
GMAO
MOPITT
ASTER
MISR
V0 Holdings
MODIS-T
MODIS-A
Other EOS =
• ACRIMSAT
• Meteor 3M
• Midori II
• ICESat
• SORCE
file name: archive holdings_122204.xls
tab: all instr bar
Terra EOM
Dec 2005
Aqua EOM
May 2008
Aura EOM
Jul 2010
NOTE: Data remains in the archive pending transition to LTA
Source: Glenn Iona, EOSDIS Element Evolution Technical Working Group January 6-7, 2005
AddingSeveralTBs per
Day
Challenge: Average Throughput of NASA Data Productsto End User is < 50 Mbps
TestedOctober 2005
http://ensight.eos.nasa.gov/Missions/icesat/index.shtml
Internet2 Backbone is 10,000 Mbps!Throughput is < 0.5% to End User
fc *=
Dedicated Optical Channels MakesHigh Performance Cyberinfrastructure Possible
(WDM)
Source: Steve Wallach, Chiaro Networks
“Lambdas”Parallel Lambdas are Driving Optical Networking
The Way Parallel Processors Drove 1990s Computing
10 Gbps per User ~ 200x Shared Internet Throughput
San Francisco Pittsburgh
Cleveland
National Lambda Rail (NLR) and TeraGrid ProvidesCyberinfrastructure Backbone for U.S. Researchers
San Diego
Los Angeles
Portland
Seattle
Pensacola
Baton Rouge
HoustonSan Antonio
Las Cruces /El Paso
Phoenix
New York City
Washington, DC
Raleigh
Jacksonville
Dallas
Tulsa
Atlanta
Kansas City
Denver
Ogden/Salt Lake City
Boise
Albuquerque
UC-TeraGridUIC/NW-Starlight
Chicago
International Collaborators
NLR 4 x 10Gb Lambdas Initially Capable of 40 x 10Gb wavelengths at Buildout
NSF’s TeraGrid Has 4 x 10Gb Lambda Backbone
Links TwoDozen State
and RegionalOptical
Networks
DOE, NSF,& NASA
Using NLR
September 26-30, 2005Calit2 @ University of California, San Diego
California Institute for Telecommunications and Information Technology
Global ConnectionsBetween University Research Centers at 10Gbps
iGrid 2005T H E G L O B A L L A M B D A I N T E G R A T E D F A C I L I T Y
Maxine Brown, Tom DeFanti, Co-Chairs
www.igrid2005.org
21 Countries Driving 50 Demonstrations1 or 10Gbps to Calit2@UCSD Building
Sept 2005--A Number of Projects are SensorNets
Prototyping Cabled Ocean Observatories EnablingHigh Definition Video Exploration of Deep Sea Vents
Source John Delaney & Deborah Kelley, UWash
Canadian-U.S. Collaboration
A Near Future Metagenomics Fiber Optic Cable Observatory
Source John Delaney, UWash
Calit2 is Partnering with the new SIOCenter for Earth Observations and Applications
• Viewing and Analyzing Earth Satellite Data Sets
• Earth Topography
• Project Atmospheric Brown Clouds
• Climate Modeling
• Ocean Observatories
• Coastal Zone Data Assimilation
• Marine Microbial Ecology Metagenomics
• Earth Sciences Collaboratory
ROADnet and HiSeasNet are Prototypes of theFuture of In Situ Earth Observing Systems
http://roadnet.ucsd.edu
ROADNet Architecture: SensorNets, Storage Research Broker, Web Services, Work Flow
KeplerWeb ServicesSRBAntelope
Frank Vernon, SIO; Tony Fountain, Ilkay Altintas, SDSC
12
14
16
18
20
22
22-Aug 23-Aug 24-Aug 25-Aug 26-Aug 27-Aug 28-Aug
Date
Wate
r T
em
pera
ture
(°C
)
0
2
4
6
8
10
12
14
16
18
20
Pre
cip
itati
on
(mm
per
5 m
inu
te i
nte
rval)
Surface
0.5 meters
1 meter
1.5 meters
2 meters
2.5 meters
3 meters
Precipitation
Typhoon
Yuan Yang Lake, Taiwan – August 2004
Part of a growing global lake observatory network -
http://lakemetabolism.org
Access can be
difficult during the
most interesting times
Photo by Peter Arzberger, October 2004
Used
by
NS
F D
irecto
r Feb
2005
Source: Tim Kratz
Remote Observation of Episodic Eventsin Water-Based Ecological Systems
Supported by Moore Foundation
LOOKING:(Laboratory for the Ocean Observatory
Knowledge Integration Grid)
Adding Web & Grid Services to Optical Channelsto Provide Real Time Control of Ocean Observatories
• Goal:
– Prototype Cyberinfrastructure forNSF’s Ocean Research InteractiveObservatory Networks (ORION)
• LOOKING NSF ITR with PIs:
– John Orcutt & Larry Smarr - UCSD
– John Delaney & Ed Lazowska –UW
– Mark Abbott – OSU
• Collaborators at:
– MBARI, WHOI, NCSA, UIC, CalPoly,UVic, CANARIE, Microsoft, NEPTUNE-Canarie
LOOKING is Driven By NEPTUNE CI Requirements
http://lookingtosea.ucsd.edu/
Making Management of Gigabit Flows Routine
Pilot Project ComponentsPilot Project Components
LOOKING Builds on the Multi-InstitutionalSCCOOS Program, OptIPuter, and CENIC-XD
• SCCOOS is Integrating:
– Moorings
– Ships
– Autonomous Vehicles
– Satellite Remote Sensing
– Drifters
– Long Range HF Radar
– Near-ShoreWaves/Currents (CDIP)
– COAMPS Wind Model
– Nested ROMS Models
– Data Assimilation andModeling
– Data Systems
www.sccoos.org/
www.cocmp.org
Yellow—Initial LOOKING OptIPuter Backbone Over CENIC-XD
The OptIPuter Project –Linking Global Scale Science Resources to User’s Linux Clusters
• NSF Large Information Technology Research Proposal– Calit2 (UCSD, UCI) and UIC Lead Campuses—Larry Smarr PI
– Partnering Campuses: USC, SDSU, NW, TA&M, UvA, SARA, NASA
• Industrial Partners– IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent
• $13.5 Million Over Five Years—Entering 4th Year
• Creating a LambdaGrid “Web” for Gigabyte Data Objects
NIH Biomedical Informatics NSF EarthScopeand ORION
Research Network
NSF is Launchinga New Cyberinfrastructure Initiative
www.ctwatch.org
“Research is being stalled by ‘information overload,’ Mr. Bement said, becausedata from digital instruments are piling up far faster than researchers can study.In particular, he said, campus networks need to be improved. High-speed datalines crossing the nation are the equivalent of six-lane superhighways, he said.But networks at colleges and universities are not so capable. “Those massiveconduits are reduced to two-lane roads at most college and universitycampuses,” he said. Improving cyberinfrastructure, he said, “will transform thecapabilities of campus-based scientists.”
-- Arden Bement, the director of the National Science Foundation
UCSD is Prototyping Campus-Scale On-Rampsto the National LambdaRail, TeraGrid, and GLIF
SIO Ocean SupercomputerIBM Storage Cluster
2 Ten GbpsCampus Lambda
Raceway
StreamingMicroscope
Source: Phil Papadopoulos, SDSC, Calit2
UCSD Campus LambdaStore Architecture
Global LambdaGrid
Flat File
Server
Farm
W E B PORTAL
Traditional
User
Response
Request
Dedicated
Compute Farm
(100s of CPUs)
TeraGrid: Cyberinfrastructure Backplane(scheduled activities, e.g. all by all comparison)
(10000s of CPUs)
Web(other service)
Local
Cluster
LocalEnvironment
Direct
Access
Lambda
Cnxns
OptIPuter Cluster Cloud
Data-
Base
Farm
10 GigE
Fabric
Calit2/SDSC Direct Access Core ArchitectureSupporting Massive Instrumental Datasets
Source: Phil Papadopoulos, SDSC, Calit2
EmergingServices Oriented
ArchitectureEnabling Use of
LambdaGrids + W
eb
Se
rvic
es
Tiled Walls for Interactive Exploration of Large Earth SciencesData Sets With Integration of Streaming High Resolution Video
Calit2@UCI Apple Tiled Display WallDriven by 25 Dual-Processor G5s
50 Apple 30” Cinema Displays200 Million Pixels of Viewing Real Estate!
Source: Falko Kuester, Calit2@UCINSF Infrastructure Grant
Data—One Foot Resolution USGS Images of La Jolla, CA
HDTV
Digital Cameras Digital Cinema
Combining Telepresence with Remote InteractiveAnalysis of Earth Sciences Data Over NLR
HDTV Over Lambda
OptIPuter Visualized
Data
SIO/UCSD
NASA Goddard
http://www.calit2.net/articles/article.php?id=660August 8, 2005