Data OrganizationData Organization

Data filesDatabases

Data streams

Data filesDatabases

Data streams

The organization of data

The organization of data

Data/information must be organized

Computer/network accessible Stored or streaming

Data/information must be organized

Computer/network accessible Stored or streaming

Stored informationStored information

Spreadsheet Worksheet oriented Cells, columns, rows Can be set up to look like a table

Database Table oriented Column headings describe a row Rows are records Tables can be related (relational database)

Spreadsheet Worksheet oriented Cells, columns, rows Can be set up to look like a table

Database Table oriented Column headings describe a row Rows are records Tables can be related (relational database)

Spreadsheets in ExcelSpreadsheets in Excel

Limited in size to 64K rows, 256 columns

Slow to compute changes Accessed via GUI or Visual Basic

Can be exported to relational database table

Limited in size to 64K rows, 256 columns

Slow to compute changes Accessed via GUI or Visual Basic

Can be exported to relational database table

Excel WorksheetExcel Worksheet

Relational DatabasesRelational Databases

Licensed Databases MS Access, MS Sequel Server, Oracle

Free databases MySQL, PostgresSQL, MaxDB, Firebird, Ingres (http://www.geocities.com/mailsoftware42/db/)

Interfaces: GUI, SQL, Programming languages (Java, Perl, PHP, C++, VB)

Licensed Databases MS Access, MS Sequel Server, Oracle

Free databases MySQL, PostgresSQL, MaxDB, Firebird, Ingres (http://www.geocities.com/mailsoftware42/db/)

Interfaces: GUI, SQL, Programming languages (Java, Perl, PHP, C++, VB)

Relational TableRelational Table

Relational Table Elements

Relational Table Elements

Relating TablesRelating Tables

Relating TablesRelating Tables

Relating TablesRelating Tables

One to One RelationshipOne to One

Relationship

One to Many RelationshipOne to Many Relationship

Many to Many RelationshipMany to Many Relationship

Data StreamsData Streams

Not stored in a file Serial data Real-time Underlying model to discover changes over time

Drift in the data mining/analysis tools needed

Incremental adaptation of the learning algorithm

2006 inaugural conference http://www.lsi.us.es/~aguilar/ds/

Not stored in a file Serial data Real-time Underlying model to discover changes over time

Drift in the data mining/analysis tools needed

Incremental adaptation of the learning algorithm

2006 inaugural conference http://www.lsi.us.es/~aguilar/ds/

Data Stream ExamplesData Stream Examples

Network Packets AP news wire Live audio / video Control system data Customer click streams Telephone records Large sets of web pages Retail transactions

Network Packets AP news wire Live audio / video Control system data Customer click streams Telephone records Large sets of web pages Retail transactions

SummarySummary

Data is stored or streaming Data is organized Data is accessible programmatically

Need to know How data is organized How to access the data How to apply DI tools to the data

Data is stored or streaming Data is organized Data is accessible programmatically

Need to know How data is organized How to access the data How to apply DI tools to the data