Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.
-
Upload
marsha-kennedy -
Category
Documents
-
view
218 -
download
2
Transcript of Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.
Integrated Data Cycle Systems
Harvey E. Rhody
Chester F. Carlson Center for Imaging Science
The Problem
An expensive facility with many customers
The customers have different needs
Expert knowledge is needed to operate the facility
Operate the facility at low cost with good service
The Need
Increased Productivity
Fast Response
Continuous Improvement
Required Functions
Job Specification
Job Planning
Job Execution
Delivery of Results
Archive
Kinds of Knowledge
• Technical knowledge
• Systems knowledge
• Project management knowledge
• Business knowledge
All are necessary for a successful high-value project
Astronomical Observatories
Complex and Expensive System
Remote Location
Shared Resource
Leading-edge Technology
Long System Life
Spirex Telescope with Abu Camera in Antarctica
Winterover on arrival at the pole….
Six Months Later….
Abu Pipeline
Astronomer
ScienceCenter
AnalysisCenter
DataArchive
ObservationCenter
NOAO/RIT
SouthPole
White Sands/RIT
RIT
IDL, C,...XMLFITS, TIFF, ...
Pipeline
Database
ImageData
CAMLDocuments
Algorithms
CAMLController
Java, ORB
OperatorInterface
NetworkInterface
Large Magellanic Cloud
30 Doradus nebula
Star formation
NGC 6334
Abu/Spirex 1998
Productivity Challenge
960 observing hours per year
160 observing flights per year
Variety of instruments
Large community of observers
Conclusion: The traditional “PI” approach is unworkable. A new system for efficiency and relentless observation is needed.
The DCS Mission
The DCS shall provide the infrastructure to support the operation of facility instruments on SOFIA using best current knowledge and tools, and supporting continuous improvement in an efficient, extensible, and modular architecture.
SOFIA Facility Instruments
Instrument Institution Characteristics
AIRES NASA-Ames Echelle Spectrometer17 to 210 micronsR=10,000
HAWC U. ChicagoRITNASA-Goddard
Far Infrared BolometerCamera40-300 microns
FORCAST Cornell Mid IR Camera5-40 microns
FLITECAM UCLA Near IR Test Camera1-5 microns
SOFIA Team Players and Strengths
IPAC
IRAS, ISO, 2Mass, WIRE, SIRTF
Archiving, Observation planning
ARC
Smart scheduling activities for instruments and telescopes for HST, Mars Rover, Deep Space 1
GSFC
HST and NGST development
HAWC and SAFIRE instrument architecture
Object Oriented development
RIT
South Pole Data Pipelining
HAWC development
UCLA
Keck
FLITECAM
SOFIA Data CycleD ata
M is s io nP lan
O b s erver'sP ro p o s al &Exp erim ent
S im ulatio n
Q uic kLo o k
A nalyzedD ata
A rc hive
O b s erver'sP lan
PR
DataCo l l e c t i o n
P l an
P ub lic atio n
A nalys isP lan
Science Center Activities
Community Development and Support
Maintain Science Data Archives
General Investigator Support
Maintain Center Information Infrastructure System Documentation , Observatory Performance Information, Observation Constraints
Calibration Information, Atmospheric Models
Maintain FI Tools Facility Software Management, Proposal Development Tools, Proposal Evaluation Tools
Observation Planning Tools , Observing Tools, Flight Planning Tools
Data Reduction Tools -- Pipeline analysis modules
Coordinate with Observatory Operations
User Needs
• Archival Research • Get Information on Observatory Capabilities • Get Information on Instrument Capabilities • Build and submit a Proposal • Build an Observation Plan • Build a Data Reduction Pipeline • Access Observation Results • Interpret Experimental Results • Publish
User Access Requirements
User access to all user functions shall be supported via the Web
1.Access to archived science products 2.Access to supported external archives 3.Access to observatory capabilities information 4.Access to instrument capabilities information 5.Proposal development, submission and tracking 6.Observation timing estimation 7.Atmospheric modeling resources 8.Access to observation data 9.Access to flight and housekeeping data 10.Access to flight logs 11.Data processing 12.Published document archiving
System Requirements
User Access
Administrator Access
Data Storage and Retrieval
Data Processing
Planning and Scheduling
Documentation Support
Data Reduction Resources
FSI & MCS
Interfaces
Task Library
DCS Data Storage
User Interaction
Layer
prefs, context
requests, results
run an experiment
raw and processed
data
AOTs, exp info, raw data, obs logs..
various searches
run a pipelineSSMOC
Internet
DCS Data Storage
DCS DataStorageInterface
DB LocatorAnd Selector
User DB
Accounts Groups
Privileges Contexts
Preferences
Experiment DB
Templates Planned Accepted
Completed Reviews
Comments
Data DB
Raw Instr DataOBS Logs
Processed Data
Instr DB
GSFC’s IMLFSI Chain ListsHistory of Instr
Use
Provides common DCS interface to all storage,
translates DCS data access to standard DB
interface methods Convenient switch for criteria such as: load
balancing, fault tolerance
The RIT Astronomy Pipeline Team
Harvey Rhody, Scott Lawrence, Erika Tolar, Stephen Schaeffer, James DeFelice, Adith Chandrasekhar, J. Fernando Naveda, Bob Krzaczek (standing) and Ian Gatley. Not shown: Jim Chauvin, Brian Perry, Joel Kastner.