My name is John ffrench and I am the Director of Visual Resources at the Yale University Art Gallery. I oversee the imaging department which photographs the collecBon and also events, exhibiBons, etc as well as overseeing the Rights and ReproducBons office who deal with the disseminaBon of images for external requests. With the project I am discussing today, I was more of an organizing parBcipant or intermediary if you will between the conservaBon staff and the computer science group. It is important to know a few things before we begin. I am NOT a computer scienBst – though I can claim to have several as good friends. And I am not a conservator, though my group does provide treatment photography for the conservaBon department. Also I happen to be married to a conservator so that helps!
Recently, new two and three dimensional imaging modaliBes have been found useful in gaining insight into the restoraBon of damaged painBngs. Non-‐Invasive Imaging is parBcularly useful in restoraBon as it provides extensive informaBon about a work without physical contact. However, combining the results of different modaliBes is extremely difficult, and conservators generally use informaBon from each mode in isolaBon. In this project, soSware was developed to overcome the barriers in combining data and to create an intuiBve interface for conservators to examine works. The soSware allows the conservator to combine images in a common view and idenBfy the same region in the work in mulBple images simultaneously. The soSware further allows the conservator to combine data values to idenBfy materials
1
In an interdisciplinary collaboraBon, art and computer imaging experts from the Yale University Art Gallery and the Department of Computer Science began a project in 2011 to examine selected Early Italian panel painBngs combining a variety of imaging techniques that include digital photography, 3D scanning, tomography and a novel form of photography called Polynomial Texture Mapping (PTM). At that Bme no soSware existed that allowed for the various imaging modaliBes to share common coordinate systems. For example, it was not possible to overlay the PTM data on the correct corresponding secBon of a 3D model. Irma Passeri, a conservator at the Art Gallery’s laboratory, working with Holly Rushmeier, Professor of Computer Science, established that significant imaging data potenBally crucial to the work of restoring damaged painBngs, could be improved by leveraging the combined strengths of mulBple modaliBes. Professor Rushmeier therefore aimed to undertake an exploratory project in collaboraBon with Ms. Passeri, with the assistance of Post-‐Doc students in the CompuBng and the Arts department of Computer Science at Yale, to design new soSware that would allow these modaliBes to be used together. When fully refined and tested, the soSware applicaBon was be made available as an open source product.
2
In 2011, while there were means to display informaBon showing various conservaBon image types, there was no way for a conservator to do comparaBve analysis of different file types without using several programs.
There were certainly programs such as the Ghent altarpiece Project, however these were more a means of educaBon for the public than that of a research tool.
Also Yale developed West Campus, a off-‐site arts and sciences area which has, or will have shared and collaboraBve imaging labs, as well as conservaBon labs. The parBcipants of those spaces were looking for ways in which to create new means of Campus collaboraBon and a project like this was a good test-‐bed.
3
In extreme short, the project aimed to provide a means for varying images to be overlaid with each other in a computer environment to beaer enable conservators/curators the ability to study works of art (image sources were tradiBonal photography -‐ historic images and modern, mulB-‐spectral imaging, UV, IR, xray, CAT scans, PTM, and 3D laser scans to name a few).
This was seen as an exciBng collaboraBon with the Computer Science department and one of the cultural insBtuBons on campus as start of many more collaboraBon projects on campus.
IniBal 1 year grant was awarded in the amount of $80,000 which would cover the acquisiBon of capture, equipment for the capture and research of objects (NextEngine 3D scanner, materials to build a hyper-‐spectral camera, computer systems to store data and build out the soSware interface), and funds to cover 50% of the salary of a programmer/imaging scienBst.
Yale University Art Gallery would provide the object(s), studio space and 20% of staff Bme of a conservator and imaging specialist.
4
I’ll aaempt to run the soSware at the end of this talk, but for now will step you through a few of the features of the program. We all know how live-‐demo scenarios oSen go and given that this program is processor intensive. Here you can see a screenshot of what the interface looks like. There are four sample files loaded into the system, a 3D model created from a NextEngine scanner, a CAT scan of a panel painBng, a Hyper-‐spectral image, and another view of the CT scan.
The rogram is Mac/PC based – though admiaedly it performs best on a PC.
5
Here you can see the various menu opBons available. In the top menu you can have the opBon to open a 2D or 3D file directory.
Open
The two leSmost icons allow easy access to the dialog box for opening files. The first opens 2D and 3D images, and the second opens CT imaging data.
Render
The following five icons are only usable when working with 3D meshes. • Lightbulb: Toggles the direcBonal light.
• Points: Displays the mesh as a series of points or verBces.
• Wireframe: Displays the mesh in wireframe mode. • Surface: Displays the surface polygons on the mesh.
• Texture: Toggles the texture display.
View
These two icons alter what you see in the image panel when displaying an image. • Spiral: Toggles the use of interpolaBon on the picture or texture.
• Info: Toggles the display of computer, image, and perspecBve informaBon in the
6
CT Image Control This toolbox is only acBve when working with CT scanning data. You can use it to navigate through a series of 2D image slices or to select display opBons when working with 3D models.
2D Image Stack Control
These opBons are only accessible when the “2D Rendering” mode is selected. From here, you can see the current slice that is displayed in the image panel, as well as the total number of slices available to scroll through. You can also change the view in the image panel to a front, side, or top perspecBve and flip the image upside down. Perhaps most importantly, you can navigate through the full range of slices by dragging the slider at the boaom from beginning to end and back again. If your image has many slices and the slider is not precise enough to navigate to your desired slice, you can also move more slowly through the slices with the leS and right arrow keys.
7
Spectrum The Spectrum toolbox is most useful when working with hyperspectral images and textures; it can be used for standard RGB images, but will only contain three meaningful data points—the red, green and blue reflectance values. The toolbox has no effect when working with 3D meshes with no texture or medical imaging data in grayscale. To use the Spectrum toolbox, simply right-‐click on a given pixel or cell to display a graph of the available wavelengths ploaed against the normalized reflectance values. The ultraviolet spectrum consists of those wavelengths to the leS of the violet line on the graph, while the infrared spectrum appears to the right of the red line.
8
To view movie navigate to: hap://www.youtube.com/watch?v=txlwwlSRIQw
9
The source code and supporBng documentaBon is posted on Sourceforge
At the recent VAST conference a collaboraBve paper was wriaen on the project
There has been limited demonstraBon of the product to the audience it was IniBally desBned for. At last years MCN it was shown to a few people aaending (hence the interest and request to present our findings this year). It was going to be presented at IS&T however a conflict of interest in reporBng on the same findings as announced at the VAST conference required us to pull out at the last minute.
10
Currently the program is being further developed with new CS programmers and there is focus on a new imaging project underway to study medieval manuscripts and the pigments used. Through that project, addiBonal tools/features are being added. These new tools, while potenBally useful to the iniBal parBcipants, are more aimed at the needs of the 2nd phase supporters of the project than further establishing the core needs of the original group.
Unfortunately the iniBal partners of the project are not using the program as intended the new direcBon of the program is more in support of a previously unknown need,
11
WHAT NEXT?? Projects take Bme and money to complete, but we don’t always plan into project Bme for usability tesBng, or promoBon of products. It is an important step to factor in.
Gewng this into more test-‐users hands, ideally conservators who have the need, but can also provide valuable feedback and suggest further development.
While we are starBng to see some level of conservators and imaging scienBst express interest in the program, sharing this program through conservaBon circles, such as AIC, would be the next logical step and/or finding partners outside of Yale to work with on the collaboraBve development of such programs.
Ideally trying to work closer with another group interested in, or working towards similar ends would be logical.
More so in recent years, Yale is keen to openly share informaBon, resources and is looking for wider collaboraBon. But we as a community need to find beaer ways in which to bring awareness of such programs.
12
Top Related