ppt. format

July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UPervasive Knowledge Center, Indiana Universityniversity

11

Web Services for Web Services for VisualizationVisualization

Dr. Gordon ErlebacherDr. Gordon Erlebacher

School Comp. Sci. Info. Tech.School Comp. Sci. Info. Tech.

Florida State UniversityFlorida State University

July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 22

ContentsContents

ObjectivesObjectives

LanguagesLanguages

Streaming videoStreaming video

Video creationVideo creation

Remote control of visualization packageRemote control of visualization package

Interactive Web MapsInteractive Web Maps

ConclusionsConclusions


IssuesIssuesScientific visualization and information Scientific visualization and information visualization are natural interfaces to visualization are natural interfaces to complex and large datasetscomplex and large datasetsThe range and size of display devices is The range and size of display devices is increasing: PDAs, Powerwalls, E-paper, theincreasing: PDAs, Powerwalls, E-paper, the retina)retina)The proliferation of interaction devices make The proliferation of interaction devices make it difficult to reliably and consistently interact it difficult to reliably and consistently interact with the information displayedwith the information displayedThe interaction devices are often expensive The interaction devices are often expensive and non-standardand non-standard


(Picture from www.amiravis.com)

•Each user should be able to query the image differently

•Augment image with information from WEB

•Store GUI on PDA


Visualization / CollaborationVisualization / Collaboration

2D/3D navigation with multiple users2D/3D navigation with multiple users

Synchronization issuesSynchronization issues

InteractivityInteractivity

Information exchangeInformation exchange

Data analysis / mining Data analysis / mining

Feature extractionFeature extraction


Rear projection16’x8’Stereo2 projectors, blended image

Powerwall at FSUPowerwall at FSU


Partial Wish ListPartial Wish List

User Interaction should be natural, easy to useUser Interaction should be natural, easy to useGUIs should combine voice and graphicsGUIs should combine voice and graphicsGUIs should minimize unnecessary informationGUIs should minimize unnecessary informationUser interfaces should be configurable by usersUser interfaces should be configurable by usersUser interface should be portable and User interface should be portable and transportabletransportableUsers should move Users should move freelyfreely with respect to the with respect to the display and themselvesdisplay and themselves


Multiple-PDA DeviceMultiple-PDA Device

•Slice through data

•Query the Internet

•Fuse data

•User-user exchange


CPU power of PDA CPU power of PDA infinity infinity Memory of PDA Memory of PDA infinity infinityWireless bandwidth Wireless bandwidth infinity infinityWireless bandwidth 1/10Wireless bandwidth 1/10thth LAN LANDisplay resolution of PDA Display resolution of PDA infinity infinitySize of PDA screen remains limitedSize of PDA screen remains limited

Trends and AssumptionsTrends and Assumptions2000-20052000-2005

≤


ApproachApproach

Conform to standards (Corba, Java, python, XML)Conform to standards (Corba, Java, python, XML)Allow for extensionAllow for extensionMaximize generalityMaximize generalityAllow for multiple protocols, operating systems, Allow for multiple protocols, operating systems, languageslanguagesDatasource:Datasource:– SupercomputerSupercomputer– File serversFile servers– Data feeds (i.e., sattelites)Data feeds (i.e., sattelites)

802.11b standard (2001), 802.11a/g (2002-?)802.11b standard (2001), 802.11a/g (2002-?)Currently, wireless network only serves to get the Currently, wireless network only serves to get the information from and to the PDAinformation from and to the PDA


Proposed ArchitectureProposed Architecture

PDA PDARemote PDA

Server Server Server

Application Application Application

PC

Local Remote


Heat convection in the Planetary Mantle

Complex FlowsMining/Extraction/Analysis


Multiple ObjectivesMultiple Objectives

Integrate visualization with Web ServicesIntegrate visualization with Web ServicesBuild tools to enhance collaboration Build tools to enhance collaboration through visualizationthrough visualizationFacilitate feature extraction and Facilitate feature extraction and information sharing between usersinformation sharing between usersDevelop tools to query datasetsDevelop tools to query datasetsUse of system should be transparent to Use of system should be transparent to users (independent of location or users (independent of location or hardware)hardware)


Approach(es)Approach(es)

Work with standardsWork with standards– Should function on Windows/Linux/UnixShould function on Windows/Linux/Unix

Develop for ease of maintenanceDevelop for ease of maintenance– Students only stay for short periodStudents only stay for short period

Insist on web documentation of everythingInsist on web documentation of everything– Only partially successfulOnly partially successful– Success is a cascading effectSuccess is a cascading effect


Languages: C++Languages: C++

Fast ExecutionFast ExecutionCompiledCompiledWell-supportedWell-supportedReasonably portableReasonably portableStandard Template Library (not yet fully Standard Template Library (not yet fully standardized)standardized)Operator overloading (very useful)Operator overloading (very useful)Object-orientedObject-orientedStrongly typedStrongly typedSupported on all platformsSupported on all platforms


Languages: PythonLanguages: Pythonhttp://www.python.orghttp://www.python.org

Fast Prototyping Fast Prototyping Essentially same capabilities as PerlEssentially same capabilities as PerlRather well supportedRather well supportedNice interface to Java (JPython)Nice interface to Java (JPython)Weakly typedWeakly typedObject-OrientedObject-OrientedExtremely flexible (sometimes dangerous)Extremely flexible (sometimes dangerous)Extremely easy to useExtremely easy to useExceptionally clear code (e.g., forces indentation)Exceptionally clear code (e.g., forces indentation)Support on all platforms (including PDA)Support on all platforms (including PDA)


Languages: ACE/TAO Languages: ACE/TAO http://http://

www.cs.wustl.edu/~schmidt/ACE.htmlwww.cs.wustl.edu/~schmidt/ACE.html ACE: Adaptive Communication EnvironmentACE: Adaptive Communication EnvironmentTAO: The ACE ORB (CORBA Support)TAO: The ACE ORB (CORBA Support)Totally based on design patternsTotally based on design patternsWraps streaming, sockets, mutexes, etc. in high Wraps streaming, sockets, mutexes, etc. in high level classeslevel classesPortable across platforms and operating Portable across platforms and operating systems (we ported it to Linux on PDA, support systems (we ported it to Linux on PDA, support for WinCE)for WinCE)Problem: large memory footprint (4-5 Mbytes on Problem: large memory footprint (4-5 Mbytes on PDA)PDA)


Streaming VideoStreaming Videohttp://www.csit.fsu.edu/~dongchenhttp://www.csit.fsu.edu/~dongchen


Video StreamingVideo Streamingwith waveletswith wavelets

VisualizationServer

Frame

Wavelettransform

Encode

VisualizationIpaq

Frame

Wavelettransform

Decode

Color animations at 4 frames/sec on Ipaq (320 x 200) and 802.11b wireless network

CORBA/SOAP GUIIpaq


Flowchart of ACEFlowchart of ACE

Courtesy D. Schmidt


Streaming VideoStreaming VideoActive Object:

• Had its own thread(s)

• Does not block calling method

Passive Object:

• Uses parent thread

• Blocks calling method


Streaming VideoStreaming Video

EncoderCImageBuffers *ServerDgramGrabberConvertRgbToYUV ( )waveletTransform ()informSibling ( )

GrabberCImageBuffers *grabImage ( )informSibling ( )

ServerDGramCImageBuffers *informSibling ( )

CImageBuffersStreaming buffers (2)YUV/Wavelet buffers (2)RGB image buffers (2)

A

C

D

B

EncoderCImageBuffers *ServerDgramGrabberConvertRgbToYUV ( )waveletTransform ()informSibling ( )

GrabberCImageBuffers *grabImage ( )informSibling ( )

ServerDGramCImageBuffers *informSibling ( )

CImageBuffersStreaming buffers (2)YUV/Wavelet buffers (2)RGB image buffers (2)

A

CC

DD

B


0

5

10

15

20

25

30

35

80 120 160 200 240

video size (pixels x pixels)

frames/second

teapot random noise constant color

0

2

4

6

8

10

12

80 160 240 320 400 480 560 640video size (pixels x pixels)

User data rate Mbps


0

0.3

0.6

0.9

1.2

1.5

80 120 160 200 240video size (pixels x pixels)

User data rate Mbps


0

10

20

30

40

80 160 240 320 400 480 560 640video size (pixels x pixels)

frames/second


PC PDAPC PC

10 fps 5 fps

random random


Online Video CreationOnline Video Creation

Scientists use visualization tools to Scientists use visualization tools to analyze and navigate their large datasetsanalyze and navigate their large datasets

Videos are created for dissemination and Videos are created for dissemination and archivingarchiving– Sequence of frames is stored by viz programSequence of frames is stored by viz program– Frames are transformed into appropriate Frames are transformed into appropriate

video formatvideo format– Video format is converted to one or more Video format is converted to one or more

additional formats (for easy accessibility)additional formats (for easy accessibility)



Generation of videos is usually a manual Generation of videos is usually a manual processprocess

Videos usually created on Windows or Videos usually created on Windows or expensive Unix systemsexpensive Unix systems

Visualizations usually created on LinuxVisualizations usually created on Linux

Conclusion: Conclusion: – Need technology to create videos directly on Need technology to create videos directly on

Linux systemsLinux systems


Image files

Movie files

Input parameters

Choice of Codec


Input parameters

Preset parameter combinations

Codec



Created by J. F. BoisdetCreated by J. F. Boisdet

http://vector.csit.fsu.edu:8081/~boisdet/tehttp://vector.csit.fsu.edu:8081/~boisdet/temp/one.pymp/one.py


AmiraAmirahttp://www.amiravis.comhttp://www.amiravis.com


AmiraAmira

Flowcharts are created interactively by the userFlowcharts are created interactively by the user

Each component has an associated user Each component has an associated user interfaceinterface

Software capitalizes on graphic hardware (SGI, Software capitalizes on graphic hardware (SGI, Onyx, Nvidia, ATI) to achieve good performanceOnyx, Nvidia, ATI) to achieve good performance

Flowcharts, called networks, can be saved for Flowcharts, called networks, can be saved for later use.later use.

Developer version allows users to create their Developer version allows users to create their own modules for specialized visualization.own modules for specialized visualization.


AmiraAmira

Amira is a commercial packageAmira is a commercial package

I don’t necessarily recommend this I don’t necessarily recommend this packagepackage

However,However,– It has nice features, perhaps useful to the It has nice features, perhaps useful to the

visualization of static and time-dependent fluid visualization of static and time-dependent fluid structuresstructures


AmiraAmira

Read in 3D fileRead in 3D file

Generate several planar cross-sectionsGenerate several planar cross-sections

Generate an iso-surfaceGenerate an iso-surface

Generate a volumetric plotGenerate a volumetric plot

Combine techniquesCombine techniques

Demonstrate data querying (line cut, Demonstrate data querying (line cut, pointwise, etc.)pointwise, etc.)


Amira FeaturesAmira Features

Very InteractiveVery InteractiveManipulatorsManipulators– Interact with the dataInteract with the data

ExtensibleExtensible– Users can write own extension modulesUsers can write own extension modules– API is very sophisticatedAPI is very sophisticated

Highly advanced algorithms to do Highly advanced algorithms to do – Isosurface, volume rendering, vector Isosurface, volume rendering, vector

visualizationvisualization– Combinations of the aboveCombinations of the above


Heat Convection between Two Heat Convection between Two Plates (Amira)Plates (Amira)

Data, courtesy David Yeun

643

subsampling

2573 dataset

Heat flow between two plates at constant temperature


Remote Control AmiraRemote Control Amira(Yunsong Wang)(Yunsong Wang)

http://vector.csit.fsu.edu:8081/~yunsong/chttp://vector.csit.fsu.edu:8081/~yunsong/cgi-binbac/remote_amira.pygi-binbac/remote_amira.py

Creation or loading of Amira scriptsCreation or loading of Amira scripts

Automatic initiation of AmiraAutomatic initiation of Amira

Retrieve bitmap from serverRetrieve bitmap from server

Working on retrieving x,y,z coordinatesWorking on retrieving x,y,z coordinates


Amira Tcl Scripts

Amira Output

Amira Server

Script Creation

Amira Display


Remote Control AmiraRemote Control Amira(Yunsong Wang)(Yunsong Wang)

http://vector.csit.fsu.edu:8081/~yunsong/chttp://vector.csit.fsu.edu:8081/~yunsong/cgi-binbac/remote_amira.pygi-binbac/remote_amira.py


Interactive Web MapsInteractive Web Mapshttp://www.csit.fsu.edu/~garbowza/WDI/http://www.csit.fsu.edu/~garbowza/WDI/

Built by Zachary Garbow Built by Zachary Garbow – (Minnesota Supercomputer Institute)(Minnesota Supercomputer Institute)

Store large datasets on a serverStore large datasets on a server

Clients operate on the datasetClients operate on the dataset– Zooming, histograms, mean/avg/stddevZooming, histograms, mean/avg/stddev

C++ on the ServerC++ on the Server

Java Applet on the ClientJava Applet on the Client

Challenges: balance between computation on Challenges: balance between computation on client and server, and networking considerationsclient and server, and networking considerations


Interactive Web MapsInteractive Web Maps


Temperature field: 2D grid: 3400x500

Ra = 3×107

Ra = 3×108

Ra = 109

Ra = 1010


Two-way flow of information!!

Map of data

Histogram


Interactive Web MapsInteractive Web Mapshttp://www.csit.fsu.edu/~garbowza/WDI/http://www.csit.fsu.edu/~garbowza/WDI/

Built by Zachary Garbow (Minnesota Built by Zachary Garbow (Minnesota Supercomputer Institute, works with D. Supercomputer Institute, works with D. Yuen)Yuen)


Visualization UbiquityVisualization Ubiquity

Collaboration through visualizationCollaboration through visualization

Office walls become visualization displays Office walls become visualization displays (E-Ink: thin, pliable medium capable of (E-Ink: thin, pliable medium capable of electronic encoding)electronic encoding)

Exchange of visual data becomes as Exchange of visual data becomes as ubiquitous as exchange of text documents ubiquitous as exchange of text documents in 2001in 2001


An Ideal Visualization SystemAn Ideal Visualization System

Reusable Reusable modulesmodules

FlexibleFlexible

Ease of useEase of use

Low memoryLow memory

ExtensibleExtensible

ScriptableScriptable

Good debuggingGood debugging

Portable Intelligent defaults Changeable

defaults Interpreted and

compiled modes Novice and expert

modes Mathematical text

editor


Future trends in VisualizationFuture trends in Visualization

Use of Object-Oriented design patterns for Use of Object-Oriented design patterns for reusabilityreusability

Plugin technology on distributed systemsPlugin technology on distributed systems

Extensive use of visualization across the Extensive use of visualization across the network network

Increased intelligence in software Increased intelligence in software

Insertion of new algorithms without Insertion of new algorithms without recompilationrecompilation


Wireless SpeedsWireless SpeedsPresent and Near FuturePresent and Near Future

Present: 802.11bPresent: 802.11b– Range: 150 m, 10 Mbit/secRange: 150 m, 10 Mbit/sec

11stst quarter 2002: 802.11a quarter 2002: 802.11a– Range: 150 m, 54 Mbit/sec Range: 150 m, 54 Mbit/sec – Not compatible with 802.11bNot compatible with 802.11b

33rdrd quarter 2002: 802.11g quarter 2002: 802.11g– Range: N/A, 54 Mbit/secRange: N/A, 54 Mbit/sec– Compatible with 802.11b!!Compatible with 802.11b!!


OQO: true mobile computing?OQO: true mobile computing?Fall 2002Fall 2002

Up to 1 GHz Up to 1 GHz

Crusoe chipCrusoe chip

256 Mbytes memory256 Mbytes memory

10 Gbyte hard disk10 Gbyte hard disk

• Touchscreen• USB/Firewire• Windows XP• 4” screen


Future WorkFuture Work

Integrate these packages into consistent Integrate these packages into consistent framework framework – They are currently developed independentlyThey are currently developed independently

Increase support for XMLIncrease support for XML

(hopefully) integrate some of these packages (hopefully) integrate some of these packages into frameworks developed in Pervasive Group into frameworks developed in Pervasive Group (do not reinvent the wheel)(do not reinvent the wheel)

Investigate interoperability Java/PythonInvestigate interoperability Java/Python

Integrate Streaming with Zope or Java AppletsIntegrate Streaming with Zope or Java Applets


ConclusionsConclusions

We have developed several web-based We have developed several web-based services related to visualizationservices related to visualization

Objective is to access and manipulate Objective is to access and manipulate data from remote sitesdata from remote sites

Hope is to allow multiple users to Hope is to allow multiple users to manipulate the same data concurrentlymanipulate the same data concurrently

Difficulty: integrating multiple languages Difficulty: integrating multiple languages together without sacrificing efficiencytogether without sacrificing efficiency

ppt. format

Documents

Transcript of ppt. format