KDD for Science Data AnalysisIssues and Examples

Contents

Introduction Data Considerations Brief Case Studies

Sky Survey Cataloging Finding Volcanoes on Venus Biosequence Databases Earth Geophysics Atmospheric Science

Issues and Challenges Conclusion

Data Considerations

Image Data Time-series and sequence data Numerical Vs Categorical values Structured and sparse data Reliability of Data

Brief Case Studies

Sky Survey Cataloging Finding Volcanoes on Venus Earth Geophysics Atmospheric Science Biosequence Databases

Sky Survey Cataloging

The survey consists of 3 terabytes of image data containing an estimated 2 billion sky objects

The basic problem is to generate a survey catalog which records the attributes of each object along with its class: star or galaxy

To achieve this scientists developed the SKICAT system

Reasons why SKICAT was successful

The astronomers solved the feature extraction problem Data mining methods contributed to solving difficult

classification problems Manual approaches were simply not feasible.

Astronomers needed an automated classifier to make the most out of the data

Decision tree methods proved to be an effective tool for finding the important dimensions for this problem

Finding Volcanoes on Venus

Data collected by Magellan spacecraft The first pass of Venus using the left looking radar

resulted in 30,000 1000 x 1000 pixel images To help geologists analyze this data set, the JPL

Adaptive Recognition Tool (JARtool) was developed

Motivation for using Data mining methods

Scientists did not know much about image processing or about the SAR properties. Hence they could easily label images but not design recognizers

There was little variation in illumination and orientation of objects of interest. Hence mapping from pixel space to feature space can be performed automatically

Geologists did not have any other easy means for finding the small volcanoes, hence they were motivated to cooperate by providing training data and other help

Earth Geophysics

Two images taken before and after an earthquake and by repeatedly registering different local regions of the two images, it is possible to infer the direction and magnitude of ground motion due to the earthquake.

Example of a geoscientific data mining system is Quakefinder which automatically detects and measures tectonic activity in the earths crust by examination of Satellite data

Atmospheric Science

Data mining tool used is called CONQUEST Parallel testbeds were employed by Conquest to

enable rapid extraction of spatio-temporal features for content based access.

Some of the goals of the this tool is the development of “learning” algorithms which look for novel patterns, event clusters etc.

Retrieved Sea Level Pressure Fields

Biosequence Databases

The largest DNA database is GENBANK with a database of about 400 million letters of DNA from a variety of organisms

The pressing data mining tasks for biosequence are

Find genes in the DNA sequences of various organisms.

Some of the gene finding programs such as GRAIL, GeneID, GeneParser, Genie use neural nets and other AI or statistical methods

Issues and Challenges

Feature Extraction Minority Classes High degree of Confidence Data mining task Relevant domain Knowledge Scalable machines and Algorithms

Conclusions

KDD applications in science may in general be easier than applications in business, finance, or other areas. This is due to the fact that science end users typically know the data in intimate detail.