SOURCEIssue 44 A PUBLICATION FOR SOFTWARE DEVELOPERS

Quantifying Anatomical Shape with SlicerSALT

p.3

Kitware Source contains information on open source software. Since 2006, its articles have shared first-hand experiences from Kitware team members and those outside the company’s offices who use and/or develop platforms such as CMake, the Visualization Toolkit, ParaView, the Insight Segmentation and Registration Toolkit, Resonant and the Kitware Image and Video Exploitation and Retrieval toolkit. Readers who wish to share their own experiences or subscribe to the publication can connect with the Kitware Source editor at comm@kitware.com.

Kitware Source comes in multiple forms. Kitware mails hard copies to addresses in North America, and it publishes each issue as a series of posts on https://blog.kitware.com.

GRAPHIC DESIGNER Steve Jordan

EDITORS Sandy McKenzie Mary Elise Dedicke

GRAND OPENING PHOTOGRAPHER Elizabeth Fox Photography

This work is licensed under an Attribution 4.0 International (CC BY 4.0) License.

Kitware, ParaView, CMake, KiwiViewer and VolView are registered trademarks of Kitware, Inc. All other trademarks are property of their respective owners.

COVER CONTENT Stanford Bunny image generated with SlicerSALT’s Shape Analysis Module. See “Quantifying Anatomical Shape with SlicerSALT,” which begins on page three, for Stanford bunny meshes.

CONTENTS

2

QUANTIFYING ANATOMICAL SHAPE WITH SLICERSALT

KITWARE NEWS

p.3

p.5

p.8

3D SLICER AND VIRTUAL INSECT DISSECTION

3

QUANTIFYING ANATOMICAL SHAPE WITH SLICERSALT

Two years ago, the National Institute of Biomedical Imaging and Bioengineering funded an initiative to create open source software to enable biomedical researchers to generate shape analysis measurements from their medical images. This software is called Slicer Shape AnaLysis Toolbox (SlicerSALT). It is designed to re-engineer state-of-the-art shape modeling and analysis research code into a robust, bundled toolkit. More specifically, SlicerSALT aims to enhance

the intuitiveness and ease of use of research code for shape analysis studies. It also aims to help researchers find changes in shape with higher statistical power.

This article demonstrates how to use SlicerSALT to compute a spherical harmonic representation of a random object of spherical topology (the Stanford bunny). The data for the example is available through the link below, and SlicerSALT is available as source code and as binary distributions for all major platforms (e.g., Linux Redhat/CentOS, Windows and MacOS). Links to the source code and distributions reside on salt.slicer.org.

Beatriz Paniagua

Stanford BunniesSlicerSALT’s Shape Analysis Module densely samples landmarks based on the geometric properties of spherical shapes. It was used to create the screenshot to the right that shows corresponding meshes of two Stanford bunnies: an original and another with a cut ear.

Shape Analysis Module contains SPherical HARmonics-Point Distribution Model (SPHARM-PDM). SPHARM-PDM is a hierarchical, global, multi-scale boundary description that can only represent objects of spherical topology that are based in spherical harmonics. SPHARM-PDM can be computed from binary segmentations or surfaces.

In SlicerSALT, the binary segmentations or surfaces are pre-processed by a filter to ensure they have spherical topology. More specifically, the filter smoothes and fills small interior holes. The pre-processed inputs are then converted to quadrilateral surface meshes. From the meshes, SlicerSALT computes spherical parameterization with area-preserving, distortion minimizing spherical mapping. SlicerSALT implements this mapping to fit a spherical harmonics basis, which it uses to compute a SPHARM description for each input dataset. SlicerSALT then samples the SPHARM description to generate triangulated surfaces (or SPHARM-PDM representations) via icosahedron subdivision. These representations have geometric correspondence between structures, as they contain analogous points in similar parts of their geometry.

In medical applications, SPHARM-PDM representations of the same anatomical structure can be used to quantify shape differences. SlicerSALT, for example, can use such representations to quantify a defect of a known magnitude between two models. In the case of the Stanford bunnies, the defect is a cut ear.

Example Data: http://bit.ly/kwsource-slicersalt-example-data

Screenshot showing corresponding meshes of two Stanford bunnies.

4

Quantifying Anatomical Shape With Slicersalt / Issue 44

Beatriz Paniagua is a technical leader on the medical computing team at Kitware, where she leads craniomaxillofacial and musculoskeletal image analysis projects as well as morphometry analysis projects. Her main research interests are computational anatomy and the translational aspects of science.

Meet the Author

In particular, two modules in SlicerSALT compute and visualize point-to-point correspondent vectors that quantify the localized differences between shapes. These modules are Model to Model Distance and Shape Population Viewer.

For the Stanford bunnies example, SlicerSALT finds that the ear of one bunny was cut roughly three millimeters. This simple example illustrates the precision of SlicerSALT’s quantification methods, which can be used to measure shape defects in anatomy in the presence of disease or treatment.

Acknowledgement Research reported in this publication was supported by the National Institute Of Biomedical Imaging and Bioengineering of the National Institutes of Health under Award Number R01EB021391. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

SlicerSALT comparing the Stanford bunnies.

SlicerSALT analyzing the ear of a Stanford bunny.

5

3D SLICER AND VIRTUAL INSECT DISSECTIONHollister Herhold

Insects are arguably the most successful terrestrial life form. With an evolutionary history stretching over 400 million years, the world’s oceans remain the sole domain without a significant insect presence. Numbering somewhere around 1.5 million species, their vast diversity remains largely unparalleled.

Scientists have been studying the structure, or morphology, of insects for hundreds of years. The overwhelming majority of studies, however, concentrate only on external anatomy. Investigation of internal structures has usually required one of two techniques: dissection, which distorts the position of internal organs by cutting into and prying open the exoskeleton; or serial sectioning, an extremely labor-intensive process where an insect is embedded in clear epoxy and thinly sliced, one section at a time, to show internal anatomy.

New imaging modalities, in particular micro-CT scanning, have ushered in a renaissance in insect morphology. Here at the American Museum of Natural History in New York, our micro-CT scanner in the museum’s interdisciplinary Microscopy and Imaging Facility allows us to peer into areas previously unstudied.

Lasioglossum, a genus of sweat bee, with abdominal air sacs shown. From Herhold et al., American Museum Novitates 3920, Feb 2019.

Volume rendered in 3D Slicer.

During an investigation of bee anatomy, we came across some unexpectedly large muscles in the abdomens of some “sweat bees,” so called because of their habit of occasionally drinking sweat off desert hikers. Muscles are found in the abdomens, the typically bulbous hind ends, of nearly all insects. These muscles are usually thin and flat, arranged along the inside of the body wall, and are used to pull segments together for abdominal pumping—basically, what looks like insect breathing. What was seen in these bees, however, were large, cylindrical muscles, arranged to pull top-to-bottom—a structure often seen in the thorax (the middle part where the legs and wings attach) for powering the wings, except these were in the abdomen. The actual purpose of these newly discovered muscles is a subject of ongoing research.

6

3D Slicer and Virtual Insect Dissection / Issue 44

High-resolution, volumetric datasets of insect internal structures are only as useful as the tools used for analysis. Voxel sizes for insect scanning are quite a bit smaller than the typical human-sized scan; for example, the 16 species scanned for the bee study ranged from 4.8 microns to 19.4 microns. We have been using 3D Slicer in our analysis pipeline since 2016. In addition to its vast array of packaged functionality, we have taken advantage of the Python scripting capabilities of 3D Slicer to develop custom analysis tools for various studies, such as area and volume measurement and plotting.

Tube removal is an essential yet simple technique. Insects are typically placed in a plastic Eppendorf tube for scanning. Scan data includes the sample holder as well as the specimen. Removal of the tube is necessary to successfully volume-render the insect, otherwise the result is a great-looking, high-resolution image of an Eppendorf tube. Masking and cropping of the dataset is done using a “fill between slices” technique, where the desired sample area is marked at representative slices spaced throughout the volume along the head-to-tail axis of the insect. Approximately 20-40 slices are marked, depending on the complexity of the specimen and its proximity to the inside surface of the vial. The masking volume is then created using the “Fill between slices” feature of the “Segment Editor Extra Effects” module. Any required cleanup of this volume is done using the standard Segment Editor tools such as Paint and Scissors. Creating this masking volume for tube removal typically takes about an hour, depending on the specimen.

When this masking volume is applied against the original volume with the “Mask Volume” feature, the tube is removed, leaving only the insect. At this point, the masking segmentation is saved as a separate segmentation file to be used later in case missing areas in the cropped volume are noticed during analysis and need to be added back.

Large dorsoventral, or top-to-bottom, muscles seen in a cutaway of Dieunomia nevadensis, a sweat bee. Dorsal-longitudinal muscles (dlm), oblique muscles (om), dorsal-ventral muscles (dvm) and spiracles (sp) are highlighted. From Herhold et al., American Museum Novitates 3920, Feb 2019.

Volume rendered in 3D Slicer, muscle color added in Adobe Photoshop.

7

Volume renderings of Megalopta sp. (Halictinae: Augochlorini), detailing cropping method used to show metasomal muscle morphology. A. Full specimen. B. Full specimen with location of clipping box shown. C.

Clipped specimen, shown in same location. Arrow denotes direction of view in D, toward posterior. D. Rotated and scaled posterior view, showing details of internal structures.

Once the volume is masked, standard volume-rendering techniques are used to view the specimen. For the sweat bee study, enabling Cropping and moving the Crop ROI around allowed real-time virtual dissection for detailed examination of internal structures.

A compelling reason for our use of 3D Slicer is interoperability. The cross-platform and open-source nature of the project means data sharing with colleagues is simple. Platform and file format issues are simply nonexistent, as anyone can download and install on any system to view and contribute.

One particular feature seldom highlighted in product marketing is the user community. The Slicer forum is a tremendous resource for users old and new, providing invaluable assistance with using the program, or development assistance for those wishing to create their own extensions and modules.

Insects provide an amazing resource for the study of evolution, and continued improvements in micro-CT scanning and analysis tools will help us learn more about these fascinating animals.

References[1] Herhold, Hollister W., Steven R. Davis, Corey S. Smith, Michael S. Engel, and David A. Grimaldi. 2019. “Unique metasomal musculature

in sweat bees (Hymenoptera, Apoidea, Halictidae) revealed by micro-CT scanning.” American Museum Novitates 2019, no. 3920 (February): 1-28. https://doi.org/10.1206/3920.1

Hollister Herhold is a Visiting Scientist at the American Museum of Natural History, working in the Entomology and Paleoentomological Collections. His research includes studying evolution via insects (fossils as well as living insects and other arthropods), primarily using micro-CT to study anatomy and physiology.

Meet the Author

8

KITWARE NEWSKitware Grand OpeningKitware was proud to host the grand opening of its new headquarters in Clifton Park, New York on March 5, 2019. CEO Lisa Avila and CTO Bill Hoffman led the event with opening remarks, followed by a ribbon cutting ceremony featuring Lieutenant Governor Kathy Hochul. Refreshments rounded out the festivities along with plenty of photo ops. Many thanks again to Abele Builders, the Town of Clifton Park Industrial Development Agency, Empire State Development, Spatial Solutions, Bast Hatfield, Keystone Architectural Services, Central New York Electrical Contractors, Merkel Donohue, and FASTSIGNS.

Lt. Governor Kathy Hochul, CEO Lisa Avila, and CTO Bill Hoffman cutting the ribbon.

Clockwise from top left: the Kitware lobby; shared office space; the break room; a themed conference room

OUR NEW HEADQUARTERS FEATURES:

• over 40,000 square feet with space to hire

• meeting rooms with geographical themes such as the Adirondacks and Antarctica

• a rooftop patio with over 2,000 square feet

• lots of natural light

• a variety of office configurations to suit the needs of our teams, including open work spaces and casual seating

• a break room/auditorium with more than 2,500 square feet for hackathons, tech lunches, training course, and company celebrations

• treadmill offices for employees wanting some exercise while they work

9

Grand Opening DemosAs part of the grand opening, guests were led throughout the building on an interactive tour featuring demos by the technical teams on a variety of topics, such as detecting fake videos, learning how to spot fake images, how CMake is integral to the development of many software applications, visualizing scientific simulations on mobile devices, and virtual training for laparoscopic surgeons. See https://www.kitware.com/demos/ for details on each demo.

DEMONSTRATING CMAKE

Millions of copies of Kitware’s CMake are downloaded each year by software developers who use the tool to manage the building and testing of their software across multiple platforms. Kitware builds and maintains a suite of open source solutions for use in the software development process that facilitate high- quality software development.

During this demo, our visitors were shown some familiar and notable applications that use CMake, as well as a visual display of its impressive download statistics.

DETECTING THREATS

Using a collection of cameras and sensors combined with deep learning, Kitware’s THREAT X is a software system designed to be deployed on field conditions to provide squad commanders an up-to-date picture current threat conditions.

Our tour demo featured the unblinking and untiring THREAT X, which deploys a collection of cameras and uses the latest deep learning techniques to identify scene elements (such as windows and doors), track motion, identify elements that look like people, and identify weapons out to distances as great as one kilometer.

SPOTTING FAKE IMAGES AND VIDEOS

Modified and fake images and videos have become a serious challenge for validating and authenticating visual media content in news stories, social media, and other sources. Kitware is developing media forensics techniques to detect fake or edited images and videos obtained from cell phones, surveillance cameras, satellites, and other sources.

Demo attendees had the opportunity to look over a variety of images and videos and guess which was real and which was fake.

10

PERFORMING VIRTUAL LAPAROSCOPIC SURGERYLaparoscopic surgery is a minimally invasive procedure where the surgeon must guide the tools based on the camera feed displayed on a video monitor.

However, this procedure requires significant training in the skills required to use the tools. Our virtual task trainer for fundamental laparoscopy training, designed and tested in collaboration with RPI, Buffalo, and UCA, implements core training tasks including suturing, pattern cutting, and camera navigation. Demo attendees were able to see the haptic devices in action and even try out some pattern cutting.

VISUALIZING SCIENCE ON CELL PHONESNumerical simulations have a large impact on our lives though many of us don’t realize their importance. They are used to create new pharmaceuticals to combat disease; investigate new surgical procedures; design new fuel-efficient cars; track and predict changes in our climate and biosphere; develop lighter, stronger materials; and explore new ways of generating energy.

Visitors got a taste of what our technology can do by running simulation demos on their mobile devices using VTK.js and ParaView Glance. Check out https://www.kitware.com/demos/#scientific for the QR codes for these simulations.

The Clifton Park team posing in the auditorium at the new Kitware headquarters.

11

North Carolina Office Delivers Free CoursesOur Carrboro, North Carolina office hosted its second Biomedical Image Analysis and Visualization Course on March 12-14. Members of our Medical Computing and High-Performance Computing and Visualization teams taught clinical researchers and technologists how to use the open source tools VTK, ParaView, ITK, and 3D Slicer, which are widely used in the medical imaging research community. We had close to 30 participants ranging from nearby University of North Carolina to as far away as Seattle Children’s Hospital.

Participants learning all about the Visualization Toolkit (VTK).

A lunch-time demo of the SlicerVR was well attended.

Kitware Welcomes New Team MembersWe’re pleased to welcome the newest Kitware employees at their respective locations!.

In Clifton Park, NY:

In Arlington, VA:

In Carrboro, NC:

In Lyon, France:

• Anthony Barricelli, R&D Intern

• Izabella Colon, Annotation Specialist

• Mary Elise Dedicke, Technical Proposal Writer

• Hannah DeFazio, R&D Intern

• Jae Fish, Annotation Intern

• Todd Friedman, Compliance Associate

• Matthias Kirchner, Staff R&D Engineer

• Andrew Matrai, R&D Engineer

• Michael McDermott, R&D Engineer

• Aimee Messier, Accounting Associate

• Najam Syed, R&D Engineer

• Bhavan Vasu, R&D Engineer (former intern)

• Branden Wagner, IT Security Manager

• Brandon Davis, R&D Engineer

• Christopher Funk, R&D Engineer

• Patrick Avery, R&D Engineer (former intern)

• Ramraj Chandradevan, R&D Intern

• Suraj Maniyar, R&D Intern

• Gabriel Devillers, R&D Engineer (former intern)

• Edern Haumont, R&D Engineer (former intern)

• Mhamed Lmarbouh, R&D Intern

• Marie Pujol, Admin & Accountancy Assistant

• Antoine Robert, R&D Engineer

• Jan-Michael Rye, R&D Engineer

• Max Zeyen, R&D Engineer

Check out https://blog.kitware.com/events/ for future course offerings and other events.

We are HiringKitware is a research and development organization focused on providing state-of-the-art scientific computing, computer vision, medical computing, data analytics, and software process solutions to academic institutions, government organizations, and commercial corporations. Through Kitware’s commitment to open source, your efforts will have a global impact. Our most popular open-source tools, such as CMake, VTK, ParaView, ITK, 3D Slicer, and KWIVER, are used by tens of thousands of researchers and software developers to fuel the next generation of scientific discovery.

Apply now at jobs.kitware.com

Santa Fe, NMClifton Park, NYArlington, VA

Carrboro, NCLyon, France