A quick tour of the datasets for VLDB 2008 (does not include datasets already in the UCR archive)

A quick tour of the datasets for VLDB 2008

(does not include datasets already in the UCR archive)

Number of training objects 80

Number of testing objects 2320

Number of classes 8

Length of time series 1024

Euclidean Distance accuracy 95.05%

Some Name

The dataset came from blah blah blah blah

Why is difficult?

• Blah blah• Blah blah• Blah blah

Formatting Note

This is the one nearest neighbor, Euclidean distance accuracy for just the training set, measured using leaving-one-out.

I measured the accuracy of 1NN-ED on the training set (only).

This was to make sure we do not have any formatting misunderstandings

You should test the 1NN-ED on the training set (only), and see if you get the same answers. Do this first, otherwise we may waste time.



Number of classes 8



This figure is from [a]. The only change we made was to flip the data left to right, (and z-normalization)

This dataset is described in Mallat, S. G. (1998), A Wavelet Tour of Signal Processing, San Diego: Academic Press. However the data we used was donated by Jeong [a].

The data was obtained by randomly choosing 55 objects for the training set and choosing the rest for the testing set. Each time series was also reversed.

[a] M. K. Jeong, J. C. Lu, X. Huo, B. Vidakovic, and D. Chen (2006), "Wavelet-based Data Reduction Techniques for Process Fault Detection," Technometrics, 48(1), 26-40. http://web.utk.edu/~mjeong/

MALLAT TECHNOMETRICS

Why is difficult?

• Many classes • Some classes are globally similar, and have only local differences. • Small training set (In [a], using 1024 instances for training, a decision tree got 96.87% accuracy. Since this was too easy, we reduced the size of the training set significantly).



Number of classes 2


Euclidean Distance accuracy 95.522

See Keogh ICDM06

Eamonn Keogh, Li Wei, Xiaopeng Xi, Stefano Lonardi, Jin Shieh, Scott Sirowy (2006). Intelligent Icons: Integrating Lite-Weight Data Mining and Visualization into GUI Operating Systems. ICDM 2006.

ItalyPowerDemand (3 years)

TaskDistinguish days from Oct to March (inclusive) from April to September

Why is difficult? • Borderline days (late Sep vs early Oct)• Unusual days (soccer games etc)• Under sampled data?• August is radically different to the rest of the summer months.

1 3 5 7 9 11 13 15 17 19 21 23

From Keogh ICDM06



Number of classes 4



CinC_ECG_torso

TaskData is taken from ECG data for multiple torso-surface sites. There are 4 classes (4 different people)

Why is difficult? • See gray strip on figure. Depending on location on the body, the peak can be positive, neutral or negative. Similar remarks apply to all features. • The figure shows aligned data, but the challenge data is slightly out of alignment.



Number of classes 5



Haptics

TaskData is taken from 5 people entering their “passgraph” on a touchscreen. We only consider the X axis.

Why is difficult? • Small training set• I think (but have not checked this) that the high variability at the beginning and end of the time series is just noise.• We are just looking at the X-axis for simplicity, we should also be looking at Y-axis, pen pressure, pen acceleration…

Novel Shoulder-Surfing Resistant Haptic-based Graphical PasswordBehzad Malek, Mauricio Orozco, Abdulmotaleb El Saddik

4 sample time series (before normalizing)

0 200 400 600 800 1000 120040

60

80

100

120

140

160

180

200



Number of classes 6



Symbols

TaskThirteen people participated in this experiment. They were asked to copy the randomly appearing symbol as best they could. There were 3 possible symbols, each person contributed about 30 attempts.

Why is difficult?

• Individuality of the 13 individuals• Each of the 6 classes looks only at the X or Y axis, we really should have 3 classes looking at the X and Y axis• Two of the symbols are very very similar on the Y-axis• Small training set

This dataset was created for the contest by Jill Brady, a grad student at UCR. We gratefully acknowledge her.

0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400

X-axis Y-axis



Number of classes 10



MedicalImages

TaskThe data are histograms of pixel intensity of medical images. The classes are “different human body regions.”

Why is difficult?

• It is not clear that treating the raw data as time series is the best overall approach for this problems, but the original authors due report success with a “time warping” measure.• Original time series are of different lengths, some are very short, making them all the same length may have introduced artifacts

This dataset was donated by Joaquim C. Felipe, Agma J. M. Traina and Caetano Traina Jr.

0 10 20 30 40 50 60 70 80 90 100



Number of classes 2



SonyAIBORobotSurface

TaskThe robot has roll/pitch/yaw accelerometers, here we looked at just X-axis. The task is to detect the surface being walked on.

Why is difficult?

• Noisy data• Small training set. See figure at left, with enough data it looks easy.

This dataset was donated by Manuela Veloso and Douglas Vail of Carnegie Mellon University

Red: Cement. Blue Carpet



Number of classes 2



SonyAIBORobotSurfaceII

TaskThe robot has roll/pitch/yaw accelerometers, here we looked at just Z-axis. The task is to detect the surface being walked on.

Why is difficult?

• Noisy data• Small training set. See figure at left, with enough data it looks easier.

This dataset was donated by Manuela Veloso and Douglas Vail of Carnegie Mellon University

Red: Cement. Blue Carpet or Field



Number of classes 2



TwoLeadECG

TaskTime series is taken from MIT-BIH Long-Term ECG Database (ltdb) Record ltdb/15814, begin at time 420, ending at 1019. The task is to distinguish between signal 0 and signal 1.

Why is difficult? • Subtle distinctions • Small training set• Beat extractor does not produce perfect alignment, but after using EM to align the signal (figure at left) it is clear that certain parts of the signal are more informative.



Number of classes 3



StarLightCurves

TaskTime series are star light curves falling into three classes.

Why is difficult? • Two of the three classes are quite similar. • Large dataset (but the real datasets have billions of these!)• Phase was aligned using standard astronomy tricks. However we tried circular shift invariant Euclidean distance (see [a]) our accuracy improved, suggesting the alignment is not perfect.

1 - CEPH2 - EB 3 - RRL

[a] Eamonn Keogh, Li Wei, Xiaopeng Xi, Sang-Hee Lee and Michail Vlachos (2006) LB_Keogh Supports Exact Indexing of Shapes under Rotation Invariance with Arbitrary Representations and Distance Measures. VLDB 2006.



Number of classes 4



DiatomSizeReduction

Task“Each successive generation of a clonaly reproducing diatom is slightly smaller than its forebears .”[a]

Why is difficult? • Small training set • Possible errors caused by image processing step.• Change in scale of diatoms shows up as “warping”.

[a] http://rbg-web2.rbge.org.uk/DIADIST/index.htm?srseries.htm&main[b] Xiaopeng Xi, et al (2007). Finding Motifs in Database of Shapes. SDM'07

(many omitted)

Eun

otia

ten

ella

Gomphonema augur

Fragilariforma bicapitata

Stauroneis smithii

0 200 400 600 800 1000 1200

[b]



Number of classes 2



Motes

TaskSensor data used in paper [b].Here the task is to distinguish between sensor q8calibHumid and sensor q8calibHumTemp. The raw data has dropouts, which I left in.

Why is difficult? • Small training set.• Lots of dropouts (however, when noise is removed, should be very easy).• Here the dropouts had value zero. But after z-normalization these values changed. It would have been easier to do smart smoothing if the data was not normalized.

[a] Raw data from Carlos Guestrin (CMU), Classification version by Keogh [b] Jimeng Sun, Spiros Papadimitriou, Christos Faloutsos: Online Latent Variable Detection in Sensor Networks. ICDE 2005: 1126-1127

0 50 100 150 200 250 300 3500

5

10

15

20

25



Number of classes 3



ChlorineConcentration

TaskSensor data used in paper [b].Multiple sensors have spatial correlation, which I arbitrarily divided into 3 sets

Why is difficult? • The borderline cases are hard to classify. However with more data it would be easy. For example, when I randomly sample k items from the labeled test set, and do INN ED classification, I get…

1000 -> 76.5% accuracy2000 -> 89.85% accuracy3000 -> 96.8% accuracy

[a] Stacia Thompson and Jeanne M. VanBriesen (CMU) Classification version by Keogh [b] Jimeng Sun, Spiros Papadimitriou, Christos Faloutsos: Online Latent Variable Detection in Sensor Networks. ICDE 2005: 1126-1127

0 20 40 60 80 100 120 140 160 180-0.2

0

0.2

0.4

0.6

0.8

1



Number of classes 2



ECGFiveDays

TaskData is from a 67 year old male. The two

classes are simply 1) ECG date: 12/11/1990 2) ECG date: 17/11/1990

Why is difficult? • Wandering baseline was not removed,

this shows up as linear drift.• Beat extractor does not produce

perfect alignment, but after using EM to align the signal (figure at left) it is clear that certain parts of the signal are more informative.

Wandering baselineExcerpt of Class 1



Number of classes 7



InlineSkate

TaskThis data was been collected from

experiments with inline speed skaters on a treadmill.

Each time series represents an angular measurement of the ankle during one movement cycle.

Cycles were of different lengths, we made them all the same length.

Why is difficult? • Lots of “warping”• Long time series (for algorithms that

scale poorly in dimensionality).• The “cycle” extraction algorithm might

not be perfect (this was done before we saw the data)

The data was provided by Fabian Moerchen and Olaf Hoos.

0 200 400 600 800 1000 1200 1400 1600 1800 2000






FacesUCR

TaskThis data consists of faces of grad students

transformed into “time series”

Why is difficult? • Variation of head angle and expression.• Some have glasses/no glasses versions• All grad students look alike (well, some

do).• The transformation algorithm is a little

brittle (we have since found more robust techniques).

Photographs by Chotirat "Ann" Ratanamahatana, image conversion by Xiaopeng Xi and Eamonn Keogh






WordsSynonyms

TaskThis dataset consists of word profiles for

George Washington's manuscripts. This dataset is the “50-words” dataset,

remapped to 25 classes.The data was flipped left-right so that it

would not be recognized.

Why is difficult? • There are two ways to be a member of

each class.• In this case, length normalization

clearly does throw away useful info.• Errors from the difficult task of OCR on

old documents

0 50 100 150 200 250 300 350 400 4500

0.5

1

The data was provided by Toni M. Rath and R. Manmatha.

The time series representation of words is known to be very competitive with other representations [a].

Here the results might not be competitive because we are only using one (of four) time series per word, we are normalizing, and we have small training sets.

[a] Word spotting for historical documents. Toni M. Rath and R. Manmatha International Journal on Document Analysis and Recognition. Volume 9, Numbers 2-4 / April, 2007

A quick tour of the datasets for VLDB 2008 (does not include datasets already in the UCR archive)

Documents

Transcript of A quick tour of the datasets for VLDB 2008 (does not include datasets already in the UCR archive)