An Implementation of Curvilinear Reformatting on VTKWallace Loos, David Gobbi (Intern Supervisor) and Wu, Shin-Ting (Adviser)

Department of Computer Engineering and Automation (DCA)School of Electrical and Computer Engineering

University of Campinas (Unicamp)P.O box 6101, 13083-970 – Campinas, SP, Brazil

{wloos,ting}@dca.fee.unicamp.br

dgobbi@ucalgary.ca

Abstract – This report presents an implementation of curvilinear reformatting tool using VTK and OpenGLon the platform Mac. This task was performed in the Calgary Image Processing and Analysis Centre (CIPAC) atUniversity of Calgary within the cooperation program between CIPAC and the Interactive Visualization Group (IVG)at Unicamp.

Keywords – VTK, Curvilinear Reformatting, Mesh.

1. IntroductionThe curvilinear reformatting is a non-invasive toolthat allows exploring the human brain. It consistsin peeling the brain in the direction perpendicularto the scalp [8]. The utility of this tool for ana-lyzing the cortical surface and the opportunity oflearning a new development environment made usto set as a one-time project its implementation ontop of VTK and OpenGL running on OS X Mav-ericks/Mac during my three month Internship Pro-gram at the Calgary Image Processing and AnalysisCentre (CIPAC).

An implementation of the curvilinear refor-matting algorithm has been accomplished by theInteractive Visualization Group (IVG) at Univer-sity of Campinas (Unicamp) [2]. It is built on topof the wxWidgets GUI library [10] and the opensource Grassroots DICOM library is used for read-ing and parsing DICOM medical files [3]. GPU re-sources are extensively explored for real-time inter-active renderings. The C++ code is runnable on bothWindows and Linux platforms which are the ma-jor brazilian desktop platforms. The software devel-opment environment at CIPAC differs greatly fromthe brazilian environment. Their desktops run Ap-ple Macintosh and their applications are built on topof the cross-platform visualization toolkit VTK [9]and the Qt GUI library [1].

This short report details a learning experi-ence at CIPAC for migrating our curvilinear refor-matting implementation to the Mac platform. It isorganized as follows. A brief description of the al-gorithm and some differences between the imple-mentation are described in Section 2. Result of theimplementation is presented in Section 3. Finally,

the conclusion and future works are presented inSection 4.

2. Curvilinear Reformatting AlgorithmCurvilinear reformatting algorithm can be dividedinto four pre-processing steps:

1. to select the region of interest (ROI) on thescalp;

2. to build the mesh from the samples of ROI;3. to build a series of intersection-free offset

meshes from the mesh;4. to voxelize the offset meshes and the voxels

are tagged with their depth with respect tothe scalp.

A user selects ROI by brushing with mouseon the head’s surface (Figure 1(a)). When the in-teraction is finished, the region is sampled (Fig-ure 1(b)) and a mesh is built. The mesh is a collec-tion of vertices, edges and faces that approximatethe scalp’s geometry. It plays an important role inthe curvilinear reformatting, because it allows us toapply known mesh oriented geometric proceduresfor computing intersection-free laminar layers. Aboundary-based data structure is employed for man-agenvironmenting the mesh topology.

In fact, the mesh is displaced in the direc-tion of the normal vector until a previsouly speci-fied depth is reached. And for each displacement itis voxelized in the same resolution of the raw brainvolume. The voxelization is used to find the vox-els which the mesh intersects, and these voxels aretagged with their depth relative to the scalp. Thevoxelization algorithm is an adaptation of the depthmap based algorithm proposed in [5]. We explore

(a) Marks are drawn while the user delineates ROI. (b) The region of interest is sampled and the mesh is built.

Figure 1. Selecting the ROI and building the mesh.

the framebuffer object (FBO) architecture availablein OpenGL [7] to off-screen render the depth mapsof a mesh from three orthogonal views and to re-construct from these maps the 3D coordinates of themesh.

At the end we have a set of depth-taggedvoxels from which we may modify the visibilityof regions of a data volume and create the visualeffect that a curvilinear reformatting is performedon the brain data. Because of distinguishing infra-structures, the way that these depth-tagged voxelsare organized at Unicamp for displaying curvilin-ear reformatted volume should be changed in theCIPAC. In the following subsections we explain thereasons and how these changes were performed.

2.1. IVG’s ImplementationThe tagged voxels are organized as a selection vol-ume [4] and this selection volume is representedby a 3D texture silmilar to that for the brain vol-ume. Each entry corresponds to the voxel’s depthlevel. Exploring the multitexturing facilities ofGPUs, both the original data volume and the selec-tion volume are sent to GPU. The standard render-ing algorithm has been slightly modified by includ-ing a comparison of the depth value from the se-lection volume and the user-defined depth value ina fragment shader. If a voxel has depth value lessthan the user-defined one, the voxel is consideredinvisible and its value does not contribute to the fi-nal fragment color. A slider widget is employed forinteractively controlling the cropping depth.

A requirement for porting the curvilinearreformatting algorithm to the image analysis and

data management environment in the CIPAC is toexplore all facilities already available in the VTKlibrary. VTK supports several volume renderingtechniques including the standard GPU-based ray-casting one. How to adapt in a three-month in-ternship program the IVG’s implementation to theVTK-based implementation? The anwser to thisquestion is presented in the next subsection.

2.2. Implementation in CIPACUnder the infra-structure constraint, three problemsarose after a careful analysis: management of meshtopology, creation of the FBO-based depth maps,and reuse of the GPU-based ray-casting renderingavailable in the VTK library.

Although it is easy to port the module thatmanages the mesh topology, we decide, for learn-ing purpose, to use the 2D Delaunay triangulationclass available in the VTK library. From a list ofinput samples of the painted samples, a 2D Delau-nay triangulation is built and further processings canbenefit from the underlying data structure.

To reuse the voxelization algorithm imple-mented by IVG, we should create FBO. AlthoughVTK supports FBO, no researcher in the CIPAC hadexperienced its usage. Hence, we decided to di-rectly access FBO via the GL API as in the originalLinux/Windows implementation. The problem isthus reduced to the creation of an OpenGL graphicscontext on a Mac platform. A quick search amongthe references shows that the MacOS X operatingsystem offers Apple-specific OpenGL APIs for cre-ating OpenGL rendering contexts and for associat-ing them with the OS X windowing system [6]. We

Mesh(number of faces) Time(s)546 0.2960141121 0.488908

Table 1. Curvilinear Reformatting’s timing.

only need to take care about the context integratywhen we switch between the VTK and OpenGLcontexts.

The GPU-based ray-casting algorithmavailable in the VTK is a standard one. It doesnot access an extra user-specified texture but thebrain data volume. To overcome this limitation, westore the tagged voxels as an adjacency list on theCPU side and use them to “retouch” the brain datavolume that is sent to GPU. The retouching consistssimply in setting to zero the tagged voxels.

3. ResultsTwo curvilinear reformattings of a data volume areshown in the Figures 2 and 3. In Figure 2 thecropping mesh has 546 faces, while in Figure 3the mesh size is of 1121 faces. Table 1 presentsthe time spent to perform the construction of off-set meshes and their voxelization. The evaluationplatform was a desktop Intel Core i5 3,2GHz CPUwith 8GB RAM 1600MHz DDR3 and a NVIDIAGeForce GTX 675MX with 1024 VRAM. Note thatthese times are dependent on the mesh size.

The complexity to traverse the adjacencylist is O(D + V ), where D is the maximum depthof the crop and V is the number of labeled voxels.In other words, the complexity to traverse the adja-cency list is linear and the interactivity of the curvi-linear reformatting is ensured even the retouchingoccurs on CPU and the brain volume must be resentto GPU.

4. Conclusion and Future WorksA learning experience in the CIPAC has shortly re-ported. In three months the programmed internshiptask has been suceessfully accomplished. A ver-sion of curvilinear reformatting algorithm has beenimplemented on top of VTK in the Mac platform.During this period we contribute with an implemen-tation of a curvilinear reformatting algorithm devel-oped in our laboratory, while the CIPAC contributeswith the knowledge in the software development onthe Mac platform.

This internship program provides us a goodopportunity to learn from practice the difference be-tween building an application with VTK, as in theCIPAC, and building one with a set of basic data andgraphics management tools available in separate li-braries, as in the IVG. VTK provides a wide rangeof algorithms that satisfies demand of most applica-tions. The great advantage of using VTK is savingthe development time. Once it is based on an object-oriented programming model, most of concepts andimplementations that are difficult to understand arehidden behind the prebuilt objects. Therefore, nodetailed knowledge or skills are required to con-struct a standard application.

However, the VTK’s programming ap-proach does not offer us flexibility in creating com-pletely novel solutions as the curvilinear reformat-ting algorithm. Combining our own code withVTK can be very challenging, because it requiresthat graphics pipelines are re-written according toVTK’s rules. Because of steep learning curve forcoding with these rules in mind, we constructed “ar-tifacts” among the VTK’s prebuilt objects to makeit works as we expected: multiple transfers of thebrain volume to GPU for ray-casting rendering andanother graphics context to have immediate-modeaccess to the framebuffer objects for efficient off-screen rendering.

Since we are always looking for new tech-nologies and needing complete flexibility for eval-uating different ideas we plan to keep the raw ma-terial development approach in IVG. Nevertheless,facing the increasing number of Mac desktops, weplan to include the Mac in our cross-platform codedevelopment in mnext code revision.

References

[1] Qt project. https://qt-project.org/. Accessed on April 2014.

[2] Reformatacao curvilinea. http://www.dca.fee.unicamp.br/projects/mtk/loos/index.html.Accessed on April 2014.

[3] Grassroots DICOM. http://gdcm.sourceforge.net/wiki/index.php/Main_Page. Accessed on April 2014.

[4] Markus Hadwiger, Joe M. Kniss, ChristofRezk-salama, Daniel Weiskopf, and Klaus En-

(a)

(b)

(c)

Figure 2. Curvilinear reformatting.

gel. Real-time Volume Graphics. A. K. Peters,Ltd., Natick, MA, USA, 2006.

[5] Evaggelia-Aggeliki Karabassi, Georgios Pa-paioannou, and Theoharis Theoharis. A fastdepth-buffer-based voxelization algorithm. J.Graph. Tools, 4(4):5–10, dec 1999.

[6] OpenGL on the Mac Platform.https://developer.apple.com/library/mac/documentation/graphicsimaging/conceptual/opengl-macprogguide/opengl_pg_concepts/opengl_pg_concepts.html#//apple_ref/doc/uid/TP40001987-CH208-SW1. Accessed

(a)

(b)

(c)

Figure 3. Curvilinear reformatting.

on April 2014.[7] the cross-platform graphics API OpenGL.

http://www.opengl.org. Accessed onAugust 2011.

[8] Wu Shin-Ting, Clarissa Lin Yasuda, and Fer-nando Cendes. Interactive curvilinear re-formatting in native space. IEEE transac-tions on visualization and computer graphics,18(2):299–308, February 2012.

[9] Visualization Toolkit (VTK). http://www.vtk.org/. Accessed on March 2014.

[10] wxWidgets. https://www.wxwidgets.org/. Accessed on April 2014.