THEMIS/GBO Review 1 Banff, March 29, 2006
Ground Based Observatories (GBO) System Design Overview
Operational StatusLessons Learned
S. E. Harris
University of California - Berkeley
THEMIS/GBO Review 2 Banff, March 29, 2006
GBO Components
AC Power
Ground Magnetometer (GMAG)
Telesat Dish
Iridium
GPS
All Sky Imager (ASI)
Observatory Support Equipment (OSE)
inside the “Hut” enclosure
Internet
Variations possible at some sites:
Existing magnetometer
“Hut” enclosure not needed
Existing Internet connection
THEMIS/GBO Review 3 Banff, March 29, 2006
Complete Installation at McGrath
THEMIS/GBO Review 4 Banff, March 29, 2006
All Sky Imager (ASI)
Main Components• Camera, Starlight Xpress, MX716
– Designed for amateur astronomy– CCD Format: Interline transfer, 376 x 290
pixels (binned), 8mm diag.– Cooled with TEC for reduced dark current
• All Sky Lens– Telecentric design by Keo Consultants– Full hemisphere viewing, full resolution
obtained 256 x 256 image• Environmental Enclosure with Sun Shade
THEMIS/GBO Review 5 Banff, March 29, 2006
ASI Unit Test Summary
Acceptance Tests:• FOV (>170°)
• Exposure Time (duration at least 1 sec)
• Spectral Response – Produce detectable response to source radiance < 10kR
– Goal < 1kR– Passband 400 – 750nm– 1 sec exposure duration
• Spatial Resolution (>250 pixel image diameter)– Verify point source response
• Cadence (better than 10s)– Goal < 3s
• Record dark and bias images (room temperature)
Pre-Shipment:• Focus adjustment and verification
• Alignment verification (top of image relative to alignment datum)
• Heater control functional test
• Sun Shade solenoid functional test (#iterations: 360)
• Internal mechanical inspection checklist (thm_gbo_116)
• Final functional test and System Burn-in (duration: several days)
THEMIS/GBO Review 6 Banff, March 29, 2006
ASI Responsivity Testing
Data report: thm_gbo_118
Recorded on all Imagers:
• On Axis spectral response
• Off Axis spectral response
• Dark vs exposure time
• Resolution
0 500 1000 15002500
2510
2520
2530
2540
2550
2560
Desired Exposure Time (msec)
Dar
k In
tens
ity
Cam 7 Dark Intensity vs. Desired Exposure Time
400 450 500 550 600 650 700 750 8000
0.01
0.02
0.03
0.04
0.05
0.06
Wavelength (nm)
Res
pon
se (
Co
unts
/Ra
yle
igh
/sec
)
Cam 7 On-axis Response vs. Wavelength 450-700 nm
-80 -70 -60 -50 -40 -30 -20 -10 00.015
0.02
0.025
0.03
0.035
0.04
0.045
0.05
0.055
Angle in Degrees
Res
pon
se (
Co
unts
/Ra
yle
igh
/sec
)
Cam 7 Response vs. Angle for 500 nm, 600 nm Wavelengths
500 nm
600 nm
THEMIS/GBO Review 7 Banff, March 29, 2006
ASI Resolution Testing
On-Axis 40º Off-Axis 80º Off-Axis
THEMIS/GBO Review 8 Banff, March 29, 2006
ASI Enclosure Design
Design: Allison Park Group, Inc.
Heaters
S.S. Enclosure
Acrylic Dome
THEMIS/GBO Review 9 Banff, March 29, 2006
Sun Shade
ASI must survive non-operating exposure to daytime sun• CCD is Sony ICX249AL
– Plastic “microlens” on each pixel, transmission reduction w/ exposure to UV
• All Sky Lens uses Peleng Fisheye f/3.5– A/R coating exhibited discoloration after one summer in ATHA
Solenoid drive actuates to close shade• Falls open when solenoid power is removed
THEMIS/GBO Review 10 Banff, March 29, 2006
ASI Enclosure Performance
ASI installed at Ft. Yukon, AK
Dome:• Generally stays clear of snow without
intervention• Some ice buildup on flange
Heating:• Maintains 60° T using about 50W avg.
heat power• Total heating power available is 240W
(60W in dome)
18:00 00:00 06:00 12:00 18:00 00:00 06:00-50
-40
-30
-20
-10
0
10
20Tue Jan 24 16:00:13 PST 2006
Time of day
ext tempasi tempasi duty cycle
THEMIS/GBO Review 11 Banff, March 29, 2006
Ground Magnetometer
GPS Receiver (included at all GBO sites) Antenna and Electronics Integrated into one package NTP compatible (1msec time accuracy)
Fluxgate Magnetometer (sensor shown below) ±72KnT dynamic range @ 0.01nT Resolution (~23 bits) Offset DAC system for 256 possible ranges per axis 2 Vectors per second data rate Low Power < 4W USB interface for data retrieval and firmware upload
THEMIS/GBO Review 12 Banff, March 29, 2006
GMAG Installation
Fluxgate Magnetometer Installed at Athabasca
THEMIS/GBO Review 13 Banff, March 29, 2006
OSE Layout (1)
Rack Mount Shipping Case (12U)
System Computer
GMAG Interface Electronics
APS Serial Switch
Swappable Hard Drive (2 ea)
Iridium modem
UPS
28”
THEMIS/GBO Review 14 Banff, March 29, 2006
OSE Layout (2)
Power Control Unit
CR10X Datalogger
UPS
ASI Power Supply
CR10X Battery
THEMIS/GBO Review 15 Banff, March 29, 2006
Power Control Unit (PCU)
PCU Features• CR10X Datalogger is the environment controller for GBO
– Extended temperature range (-55º to +85ºC)– Always operating and remotely accessible (via Iridium) – Low power consumption (battery can operate it for months)– Simple programming and data logging capability
• Provides room temperature environment inside OSE and ASI – Enables use of standard commercial computers, hard drives, etc.– Implements graceful shutdown of system in either event of:
– Loss of Power– Loss of Temperature (either too high, or too low)
• CR10X is the analog & digital I/O peripheral for System Computer• Provides “watchdog” function for System Computer (PC)
THEMIS/GBO Review 16 Banff, March 29, 2006
Power Control Unit Interfaces
SSRJ7
OSE Heater
CR10X
5A
SSRJ8
UPS
6.3A
SSRJ13
OSE Cooler
6.3A
SSRJ14
System Pwr
SSRJ15
Iridium/APS
SSRJ12
ASI Power
1A, SB
From AC Main
SSRJ3
ASI Heater
5A
Ext_temperature
OSE_temperature
ASI_temperature
Int_temperature
Vline (AC Main)
Sys_imon (AC line current)
Asi_vmon
Stl_vmon
Sol_imon (Sun Shade)
OSE_humidity
UPS Battery Charge Time
CB1
CR10X Power Supplyw/ Battery Backup
Charger
SystemComputer
UPS
APS SerialSwitch
IridiumModem
PowerControl
Unit
Serial
THEMIS/GBO Review 17 Banff, March 29, 2006
OSE Heating / Cooling Devices
Small Space Heaters 175W, 120VAC, 2 ea
Solid State Air Conditioner
163W Capacity
120VAC Power
THEMIS/GBO Review 18 Banff, March 29, 2006
“Hut” Enclosure
“A Box within a Box”
Internal Rack Mount Case for Equipment
(Doubles as Shipping Case)
External Insulated Environmental Enclosure
THEMIS/GBO Review 19 Banff, March 29, 2006
Hut Performance
No heating in Hut required down to –40ºC
GBO#10, Ft. Smith
Cooling required above –5ºC
GBO#10, Ft. Smith
00:00 06:00 12:00 18:00 00:00 06:00 12:00-10
-5
0
5
10
15
20
25
30
35Wed Mar 22 18:00:10 PST 2006
Time of day
External temp
OSE temp
A/C duty cycle
00:00 06:00 12:00 18:00 00:00 06:00 12:00-50
-40
-30
-20
-10
0
10
20
30
Time of day
Tue Feb 14 18:00:10 PST 2006
External temp
OSE temp
Heater duty cycle
THEMIS/GBO Review 20 Banff, March 29, 2006
Instrument Data Flow
System Computer
ASI
GMAG Telesat Modem
Hot Swap Hard
Drive(s)
USB
Serial I/O
10/100 Base T
GPS
Internet
THEMIS/GBO Review 21 Banff, March 29, 2006
Status of Operations
Number of GBOs Deployed: 11• Installation and logistics can be difficult• Monitoring and maintaining is harder
Some operational hazards that we’ve survived so far:• Cold
– e.g. door to the shack stuck open• Hot
– e.g. stuck thermostat in the shack• Power
– generally it’s been reliable, but outages are a fact of life• Inaccessibility
– Ekati and Inuvik• RF Interference (More on this later)• Internet connection outages
– data back up is working fine• Software issues
– system processes can hang up (USB interfaces, NTP daemon)– periodically we need to reboot the system
• Non-responsive custodian
THEMIS/GBO Review 22 Banff, March 29, 2006
System Uptime
Initial analysis, 8 of 10 stations analyzed• Based on availability of CR10X records logged on site• At each station, number of records is averaging 97.5%
– Of 288 records maximum per day
• Indicates the systems are up and running normally most of the time
THEMIS/GBO Review 23 Banff, March 29, 2006
Learning and Improving
System issues are addressed in Problem Reports• PFRs describe problem
and corrective action• We track PFRs for each
station – Leaves a paper trail of
changes for each site– A copy also is attached to
site log on ‘wiki’
THEMIS/GBO Review 24 Banff, March 29, 2006
Lessons Learned - Prototype
Ice buildup on ASI dome• Added dome heaters
Insulation outgassing in ASI• Changed material to foil-
faced air pillow
THEMIS/GBO Review 25 Banff, March 29, 2006
Lessons Learned (1)
Issue PFR Description / Resolution
ASI cable length A deployment issue first increased to 60’ settled on 100’
-001-003-006-008-021
Changed ASI supply to regulated 12V (60’)Increased sun shade solenoid drive to 24VACChanged ASI 12V supply to meet in-rush currentModify ASI power interface wiring (100’)Increased sun shade solenoid drive to 28VAC
ASI Power Diagram
Condor 12V,3.4A Linear
Starlight PowerBrick
J3 P3 P1 J1
Starlight MX716
400mACAM_P12V
TEC_CCS
CAM_RTN
100 ft cable, 9 pr
All Sky Imager
12.5 V
asi_vmon stl_vmon
P12Switched
Outlet
THEMIS/GBO Review 26 Banff, March 29, 2006
Lessons Learned (2)
Issue PFR Description / Resolution
ASI USB InterfaceSusceptible to EMI, causes lock upDesign is “always on”
-005-018
Add’l power line filtering on Starlight Xpress boardDesign remote-control USB Disconnect deviceEnables an easy power reset optionInuvik is skipping exposures to elim. RF interference
Modified USB Board
USB Disconnect Device Prototype
THEMIS/GBO Review 27 Banff, March 29, 2006
Lessons Learned (3)
Issue PFR Description / Resolution
System monitoring “more is better”
-002-010
Add AC line current monitorAdd monitor for PC +5VAdd monitor on ASI power supplies
ASI Power Supply Overheating
-015 Modify power brick, add heat sink and fan
Overheated PC Boardcourtesy of Mikko Syrjasuo
Modified Starlight Brick
THEMIS/GBO Review 28 Banff, March 29, 2006
Lessons Learned (4)
Issue PFR Description / Resolution
Component failures -019-020
Ekati – USB hard drive failed, PC would not bootAtha – PC power supply failed, blew internal fuse
Obsolescence -022 USB Extender device now unavailable, changing all to make them interchangeable
PC USB DisconnectUSB Extender
Base Unit
Control fromPCU
J4 P4 P2 J2
Starlight MX716
100 ft cable, CAT5
All Sky Imager
USB ExtenderRemote
Starlight USB I/F
USB
ASI Data Interface Diagram
THEMIS/GBO Review 29 Banff, March 29, 2006
Open PFRs
Issue PFR Description / Resolution
Iridium modem configLearning curve
-016 APS serial switch auto-configures modem on resetPower reset modem and APS every dayLearn all failure modes Recently discovered a situation where modem and APS switch can get into hung state – investigation is ongoing
GMAG restart problem
-007 Problem associated with GMAG USB interface, does not restart correctly if system reboots when it’s coldVery infrequent occurrence, can be handled by operator intervention
Top Related