Data Processing and Display Challenges for Solar Dynamics

Observatory Using the Heliophysics Event Knowledgebase

Ralph Seguin Ankur SomaniLockheed Martin Solar & Astrophysics Laboratory

SDO GENERATES AS MUCH DATA IN A SINGLE DAY AS OUR PREVIOUS MISSION,

TRACE, PRODUCED IN FIVE YEARS.

• The Heliophysics Events Knowledgebase (HEK):• Helps researchers find data sets relevant for their topics of

interest.• Serves as an open forum where solar/heliospheric features

and events can be reported and annotated.• Facilitates discovery of statistical trends / relationships

between different classes of features and events.• Lightens the load on SDO data system in terms of providing

access to large subsets of data (too large to transmit over the Internet).

HOW?• Data mining

• Automated identification of interesting events in data• Event Detection System (EDS) to control and manage mining modules

• Data markup • Recording and annotating events for later data recall, extracting sample images and movies

• Capturing mission metadata associated with data, e.g. Planning logs, instrument settings...

• Automated (via EDS) and manual entries using EVACS/Panorama software suite

• Guided data searches• Web and Java clients

LMSAL DATA MINING HARDWARE

• 400+TB Apple Xsan•5TB SAS for database•100TB Cache•300TB user/archive•10Gb link to Stanford•10Gb link to LM science network• Attached Apple and SGI servers

• SGI Compute servers•22 node SGI XE Cluster for EDS•SGI UV100 w/72 core, 750GB memory

• HiPerspace Datawall:•Quad HD display (3840x2160 pixels) •9-panel 30” display (7680x4800 pixels)•Seven-node CGLX cluster

• MacPro science workstations

HEK SYSTEM

EVENT CLASSESEvent Class Description

Active Region Solar Active Region

Coronal Mass Ejection Ejection of material from the solar corona

Coronal Dimming A large-scale reduction in EUV emission

Coronal Jet A jet-like object observed in the low corona

Coronal Wave EIT or Morton waves spanning a large fraction of the solar disk

Emerging Flux Regions of new magnetic flux in the solar photosphere

Filament Solar Filament or Prominence

Filament Eruption A sudden launching of a filament into the corona

Filament Activation A sudden change in a filament without launching

Flare Solar Flare

Loop Magnetic loops typically traced out using coronal imagery

Oscillation A region with oscillating coronal field lines

Sigmoid S-shaped regions seen in soft X rays; indicator for flares

Spray Surge Sudden or sustained intrusion of chromospheric material well into the corona

Sunspot Sunspots on the solar disk

Plage Bright areas associated with active regions

Polarity Inversion Line Lines where field reverses in photosphere; Indicator for flares

Other Something that could not be classified – good candidate for further research

Nothing Reported Used to indicate that the particular data were examined, but had nothing noteworthy to the observer.

EDS DATA MINING

EDS MONITORING

ANNOTATING NEW DATA

EVENT VALIDATION AND CONTROL SYSTEM & PANORAMA

• EVACS/Panorama provides

• Search & display for events and features

• Interactive movie browsing

• Annotation/Markup tool

• Java/C for Mac OSX and Linux

GUIDED DATA REQUESTS

• Researchers identify regions, intervals, wavelengths from event descriptions

• Automatic extraction of datasets from archive

• Extracted regions shared with other researchers

HEK: FEATURES AND EVENTS; VALIDATION AND COORDINATION

• Data Mining efforts:

• Collaboration with SAO-led Feature Finding Team on development of modules for the LMSAL Event Detection System

• Validation by inter-comparison of methods and by comparison with results from visual inspection system, and by usage in scientific investigations.

• Supporting, and coordinating with, e.g., the ISSI workshop series on AIA data mining.

• Access to feature/events/coverage:

• Internet: iSolSearch: functional in FireFox and Mozilla (not yet in Explorer), being tested and expanded

• IDL: SolarSoft: functional (Mozilla, FireFox), being tested and expanded

• Interface compatibility: web URL can be copied into SSWIDL routine, and vice versa.

For details: http://lmsal.com/sungate For results:http://sdowww.lmsal.com

Ralph Seguinseguin@lmsal.comAnkur Somaniankur@lmsal.com