Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of...

18
2004 All Hands Meeting Neuroterrain + MBL Atlas Normalization Image Processing Pipeline & Interactive tools to support atlas normalization 2005

Transcript of Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of...

Page 1: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

2004 All Hands Meeting

Neuroterrain + MBL Atlas Normalization

Image Processing Pipeline & Interactive tools to support atlas normalization

2005

Page 2: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Pipeline For Atlas NormalizationDatabase Driven• Use PostgreSQL (open-source RDBMS)

Useful native types for spatial processing (2D objects & methods) + straight-forward C API for adding new types (3D & atlas specific classes)]Complex native types used directly in query predicates to filter data

• Highly granular data model for tracking all processing detailUse of Materialized Views when large scale de-normalization required

• Thorough collection of all algorithmic I/O requirementsAlgorithm ontology - I/O data types, run-time HW/SW requirements, etc.Assemble pipeline definitions from atomic algorithm objects

• Job Controlassignment of available processing “jobs” across a distributed collection of heterogeneous runtime environments driven & monitored via the RDBMS

• Audit trail of algorithm/pipeline runsUsed for many purposes - including round-trip QC & adjustment of algorithm efficacy.

Page 3: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Pipeline For Atlas NormalizationJ2EE architecture• Open-source tools• Enterprise Java Beans [EJB] (JBoss)

server-based collection of re-usuable logicUsed by ALL other systems - Servlets, WS, Applications & Applets

• Servlets + Portlets - for Web UIs (Tomcat)• Web Services - for networked remote procedure calls [RPC] (Axis)• Aspect-Oriented Paradigm

servers provide functionality accessible to programmer via minimal API (e.g., do not need to build centralized authentication de novo)application-specific customization via XML config files - no direct coding required.Examples:

• JNDI - directory services for access to RDBMS (no JDBC) & other resources• Centralized authentication & authorization• Fail-over & multi-serving scaling

• GRID connection - processing & storage (Globus)

Page 4: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Pipeline For Atlas Normalization

J2SE - Applications & Applets for complex end-user tools• Web start when practical (client-side platform problems)• Use web services both for database and image repository access.• Provides ontology based querying to select MBL data

Again - via WS-connection to ontology repository• Heavy use of JAI/Java 2D/Java 3D for UI image processing

Couldn’t use ImageJ• imaging implementation not sufficient performance for some of the GUI-based

image processing we need to do• ImageJ = Java imaging circa 1997 when images were simply static JPEGs or

GIFs to serve up in applets• Absolutely no image processing framework in Java at that time - Wayne R.

used his extensive experience developing NIH Image to perform Herculean task of creating a full-featured image manipulation Java App at a time when Java didn’t support such functionality. Even GUI elements such as “progress bars” & line drawing he needed to implement from scratch. This legacy code still lies at the core of ImageJ.

Page 5: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Pipeline For Atlas Normalization

Two major applications for MBL Brain Image Processing• Neuroterrain-MBL Normalization Pipeline QC Tool• Neuroterrain-MBL Integrated Brain Atlas Viewer

Java atlas viewer (NetOStat) derived from original Mac app. MacOStat

Brain Atlas Server• Gustafson C, Tretiak O, Bertrand L, Nissanov J. (2004) Design and

implementation of software for assembly and browsing of 3D brainatlases.Comput Methods Programs Biomed. 2004 Apr;74(1):53-61.

• Uses unique 3D data manipulation paradigm (“Macrovoxel”) to enable real-time atlas slicing via the net.

• Manages multiple atlas datasets & associated brain region VOIs.• Mac-based (being ported to Unix)

Page 6: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Neuroterrain-MBL Normalization Pipeline QC Tool

Thumbnail View• 3 way split frame

file system view to find directory with image stack.• Familiar drill up & down for rapid navigation

Thumbnail view of image stack• Scroll to rapidly view entire stack - thumb size adjustable• Easy click-n-drag to re-order sections - auto-shift entire groups of

images to account for wide-spread stack misalignment• Contextual menu - delete an image, add a blank to substitute for lost

sectionsFull view of individual image

• Grid & scale to assist image review• Simple contrast adjustments to enhance detail when necessary• Contextual menu - re-orient by flipping relative to x or y axis, quick

switch to full slide image view

Page 7: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Neuroterrain-MBL Normalization Pipeline QC Tool

Full Slide View - shows section in context of original slide• Section Navigation

“movie” controls - run stack as “movie” - very fast way to do final confirmation on QC (“play”, “pause”, “stop”, “FF”,”rewind”, “step+”, “step-”)Slider to quickly jump to specific imageOriginal Slide images

• Pair of original slide images for specific brain presented as horizontal mosaic• Using controls listed above automatically shifts outline control over appropriate

image in slide• Outline is “active” navigation tool - move outline to new section updates full

view• Section Image basic info

Lists status info (“deleted”, “flipped”) - active controlsLists original geometric location of section on original slide - geometric relation of section to slide must be maintained to support investigator using atlas to select brain regions for high-rez cell stack imaging.

• Image Manipulation“knife” tool to separate concatenated sections“sewing machine” tool to concatenate sections split by the auto-segmenter.

Page 8: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Pipeline QC Tool - Thumbnail View

Page 9: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Pipeline QC Tool - Slide View

Page 10: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

2004 All Hands MeetingMBL ImageRepository

P4/Lnx ClusterG4/Mac Cluster

Nrtrn J2EEApp Svr

Slide SegmentationCoarse Alignment

Auto-segment ROIsWarp

Pathfinder

Coarse AlignmentWarp

Inet, Inet2 &Grid

Router

Switch

MBL Brain Image Stack IP Pipeline

Page 11: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:
Page 12: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:
Page 13: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:
Page 14: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Neuroterrain-MBL Integrated Brain Atlas Viewer

Atlas Viewer• Select from multiple atlas data sets• Select & draw brain region ROI

assign color/ROItoggle ROIs on & off

• Use wire-frame “virtual knife” navigator for rapid navigationKnife position info - x,y,z rotation - cutting plane translationSelect cutting plane (coronal, horizontal or sagittal)Select whether mouse movement translates knife along cutting plane or rotates along x,y, or z.

• Atlas position info - x,y,z location

Page 15: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Neuroterrain-MBL Integrated Brain Atlas Viewer

MBL Section Image Viewer• Stack navigation - slider, step buttons or “jump to”

Coupled to atlas viewer - selecting new brain section adjusts atlas virtual knife to equivalent registered location.

• Use web-service based connection to MBL image repositorypose an ontology-based query against MBL RDBMS to define list of brains to view (e.g., “retrieve all brains for female BXD5 mice in age range 20 -40 days of strain X”)Image stacks delivered over the net via WS (SOAP + attachments) - either as individual images or a compressed archive - archive(s) are decompressed in the background, while investigator views another stack.

• Select individual MBL brain for viewing from result set via menu• Multi-mode viewing - brains for which multiple “views” of stack exist

(e.g., “raw”, “linear affine adjusted”, “non-linear warped”, etc.) - can select which set to view

Page 16: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Neuroterrain-MBL Integrated Atlas Viewer

Page 17: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Neuroterrain-MBL Integrated Atlas Viewer

Page 18: Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of available processing “jobs” across a distributed collection of ... Examples:

Neuroterrain-MBL Integrated Atlas Viewer