WP-8, ZIB
WP-8: Data Handling And Visualization
Review Meeting Report
Felix Hupfeld, Andrei Hutanu, Andre Merzky, Thorsten Schütt, Brygg Ullmer
Zuse-Institute-Berlin
WP-8, ZIB
Overview
Formal Matters
Visualization
Tangible Devices
Remote File Access
Replica Management / File Movement
Metadata Service
WP-8, ZIB
Formal Matters
Deliverables: most right on time, some delivered earlier
Milestone 8.2 (M24): Dynamic data access techniques are designed and implemented
Replica catalog (Location independant file access)Remote File Access (Pitfalls and GridFTP)Visualization service
Plan for the rest of this year:Further integration with other WPsImplement additional features (user management)
WP-8, ZIB
Formal Matters (II)
B. Ullmer, A. Hutanu, W. Benger, H.-C. Hege. (2003). Emerging Tangible Interfaces for Facilitating Collaborative Immersive Visualization. Workshop on Collaborative Virtual Reality and Visualization, October 2003.
H.-C. Hege, A. Hutanu, R. Kähler, A. Merzky, T. Radke, E. Seidel, B. Ullmer. (2003). Progressive Retrieval and Hierarchical Visualization of Large Remote Data. Workshop on Adaptive Grid Middleware, September 2003.T. Schütt, A. Merzky, A. Hutanu, F. Schintke, Remote Partial File Access Using Compact Pattern Descriptions, IEEE CCGrid 2004.
WP-8, ZIB
Visualization context
Grid = “virtualization” of resources
Broad progress toward virtualization of hardware resources (compute power, disk space, network)
Less common: virtualization of graphics hardware, software
“Virtualization” of visualization resources
Scientific visualization has demanding hardware and software requirements, unavailable to many users
Software is frequently complex and “heavy-weight”
Our approach:Automation for core visualization operations
Provision of grid-based “visualization services”
Develop simplifying mobile services & devices
WP-8, ZIB
Visualization and mobile devices
Mobile devices• Access to viz services
from mobile phones, PDAs
• Development of new grid-enabled “mobile”viz devices
Visualization• Access to grid resources
from existing viz applications
• Access to existing viz applications from the grid
WP-8, ZIB
Visualization service
1D
2D
3D
WP-8, ZIB
Mobile + viz = “viz artifacts”
MotivationAdvantages of stereo, collaborative interactionImportance of the meeting room, teleconferencingSimplifying interaction + gaining real estate by breaking out into the physical world
WP-8, ZIB
Data cards
Represent online data, user credentials, remote users, parameters, etc.
WP-8, ZIB
Interaction pads
targ
auth
sens?
net
source target
source
targ
auth
sens?
net
upper left upper right
center
lower left lower right
targ
auth
sens?
net
targ
auth
sens?
net
param 1 param 2
“binding pad” “placement pad”
“parameter pad” “control pad”
WP-8, ZIB
Remote File Access (I)
Needed forVisualization of remote files (intermediate results of large jobs)Job input files (extract and transmit only data of interest)…
Existing mechanismsGridFTP remote partial file access (read, seek, read, seek, .. )
I/O wrapper for local and remote access methods (to be used in GAT)
But does not fit Grid requirements, because
too many small remote operations!
WP-8, ZIB
Remote File Access (II)
Need to execute high-level operations on the server side
File format specific approach(HDF5)HDF5 – structured binary file format.
Datasets, Groups, Attributes
Forwards HDF5 calls to the remote side
Plugin for the Globus GridFTP server
Generic approach (much better ) next slide
WP-8, ZIB
Remote File Access (III)
Generic approach
Compact access pattern description: Pitfalls
Speedup of several orders of magnitude achieved!
Example: Access part of a Cube: (339, 65874, 262144, (0, 511, 2048, 32,
(0, 511, 2048, 32, (0, 3, 16, 32)))
just 1 (in words: one!) remote call
WP-8, ZIB
Replica Management Components
Replica catalogSupports hierarchical data organization
POSIX filesystem like interface
Uses C++ with SOAP/GSI interface
Adaptor inside of GAT
File movement serviceUsed for replication
Based on GridFTP
Adaptive service (WP6)Ranks replica locations
Optimizes GridFTP parameters
WP-8, ZIB
StorageBox Metadata Service
Unified storage for metadata of persistent Grid objects (archived files, job input/output, messages/notifications)
Any number of namespaces, configurable via pluggable access policies as public or private archives, group collaboration spaces or private annotations for public data
Public Sensor Data Archive
User ‘A’: Sensor Data Annotations and Private Messages
Project ‘X’: Collaboration
Space
User ‘B’: Sensor Data Annotations and Private Messages
AB
WP-8, ZIB
StorageBox Metadata Service
Portlet as user interface
Integrated with WP12’s MessageBox service
SOAP web service interface
Pluggable database backendswith PostgreSQL implementation
Portable C++ implementation Win32, Linux, Mac OS X
Planned: Improved query language / processing
Remote administration interface
More access policies
WP-8, ZIB
Continuing plans
Visualization, mobile devicesFurther integration with portal
Implement additional viz operations
Deploy, test both services & devices w/ end users
Data managementProvide a flexible namespace for organizing files
Remote File Access: Generic vs. File Format Dependant
WP-8, ZIB
Questions?
WP-8, ZIB
Visualization and mobile devices
Mobile devices• Access to viz services from mobile phones, PDAs
Completed + demoable from mobile phones, iPAQCompleted + demoable from mobile phones, iPAQ• Development of new grid-enabled “mobile” viz devices
Demoable: use of new devices to load remote content, Demoable: use of new devices to load remote content, activate and control video conference sessions activate and control video conference sessions
Visualization• Access to grid resources from existing viz applications
Completed: remote data access from AmiraCompleted: remote data access from Amira• Access to existing viz applications from the grid
Demoable: grid automation, presentation of reports Demoable: grid automation, presentation of reports integrating content from gnuplot, OpenDX, Amira;integrating content from gnuplot, OpenDX, Amira;grid access to streaming Amira contentgrid access to streaming Amira content
WP-8, ZIB
Videoconferencing example
24.02.04, Berlin-Munich, surgical planning
Top Related